Neutralizing misinformation through inoculation: the conclusion that could not be a conclusion

via Trust, yet verify
http://ift.tt/2kf6j5P

This is already the third post on the “Neutralizing misinformation through inoculation” paper of Cook, Lewandowsky and Ecker (2017). This post will focus on one paragraph in the general conclusions of this paper (my emphasis):

The ongoing focus on questioning the consensus, in concert with the gateway belief status of perceived consensus, underscores the importance of communicating the consensus [68,69]. However, positive consensus messaging is not sufficient, given recent findings that misinformation can undermine positive information about climate change [33,56]. As a complement to positive messages, inoculation interventions are an effective way to neutralize the influence of misinformation.

Although this is a nice sounding conclusion, I have one problem with it: these could never ever be one of the conclusions that can be drawn from these two experiments described in the paper…

Experiment 1 looked at misinformation in the form of “false balance media coverage” and claimed that “the greatest effects were seen in perceived consensus” and were discussed on basis of that variable. As described in previous post, these were the findings (in ascending order of perceived consensus in a scale of 0 → 100):

,

If we only look at that data, then I think we can conclude that:

• The misinformation only message has a negative effect on the perceived consensus (which makes sense)
• The consensus only message did just fine on its own (it showed the highest value of perceived consensus of all groups)
• When a consensus message was combined with an inoculation message, it did a bit less well (but still high and the difference with a consensus only message could well be not significant)
• The inoculation only message did remarkably worse than the consensus only message or the combined consensus/inoculation message (it was only marginally better than the control group – also here, that difference could well be not significant)

So, I am not really sure how they could reach the conclusion that a “positive consensus messaging is not sufficient” or “as a complement to positive messages, inoculation interventions are an effective way to neutralize the influence of misinformation” from the results of experiment 1. According to their own findings on false balance media coverage, the consensus message is more than sufficient and an inoculation message is found not to be necessary (or could even affect the consensus message negatively, depending whether the difference is significant or not).

I didn’t write about experiment 2 yet. In short, it investigated the misinformation in the form of “fake experts casting”. In this experiment the participants were split up into four groups: control, misinformation only, inoculation only and inoculation + misinformation. The results from the dependent variables were not found to be significant or only marginally significant. For example, if we want to compare with the results from experiment 1, the perceived consensus values looked like this (in ascending order of perceived consensus in a scale of 0 → 100):

If we just compare with the previous experiment, it is clear that inoculation is hardly effective (the perceived consensus situated below the control group, somewhere in the middle of the misinformation group and the control group).

How did they then demonstrated the neutralization by inoculation in experiment 2? They managed to extract from the data that the misinformation had a polarizing effect on “climate attitudes” (people with low free-market support increased “climate acceptance”, while people with high free-market support decreased “climate acceptance”) and that inoculation apparently neutralized this effect. The inoculation + misinformation group showed “slightly less” polarization than the control group, demonstrating that the polarizing influence of misinformation was neutralized by the inoculation.

Whatever the case might be, it is clear that there was no group in which the participants received the consensus only message, nor was there a group that got a combined inoculation + consensus message, so no conclusions could be drawn about these two aspects on basis of the results of experiment 2 either.

These two statements are most probably the conclusions of the two referenced papers (numbered 33 and 56), but it could never be one of the conclusions of the “neutralizing misinformation through inoculation” paper. I would expect such statements in a “caveat” section or in the “discussion” section with the appropriate mention that this was not found in this paper, but they were put in the “conclusions” section of the paper, without even mentioning that this paper didn’t confirm the findings of those two referenced papers.

Concluding, unless I understood something completely wrong, the statements that a “positive consensus messaging is not sufficient” and “as a complement to positive messages, inoculation interventions are an effective way to neutralize the influence of misinformation” could never be one of the conclusions of this paper. After all, experiment 1 showed that the:

1. consensus message on its own is more than enough (it performed best of all groups)
2. combined inoculation + consensus message did not perform better than the consensus message alone (maybe even a tad less)

and experiment 2 didn’t research these two aspects at all.

via Trust, yet verify http://ift.tt/2kf6j5P

June 9, 2017 at 11:24PM