How not to measure conspiracy beliefs #3

Here is a brief update to my two previous posts on the flawed study published in Psychological Medicine. To recap, the study found that almost half of the respondents in a UK sample agree that the “[c]oronavirus is a bioweapon developed by China to destroy the West”.

In a new study, John Garry, Rob Ford and Rob Johns find that this number is most likely closer to 30% than 50% of the UK population (in a representative sample from Deltapoll). The authors address the criticisms raised in the previous posts and find that the belief in COVID-19 conspiracy theories is indeed lower when we take some of these methodological limitations into account.

It is great to see additional studies tackle these survey design issues and provide estimates on the “true” proportions. I highly recommend that you read the study.

I do not have a lot to add here except for two points. First, I am surprised by the numbers in Garry et al., i.e. that even when we use a “best practice” approach, the numbers are very high. The authors argue that the “prevalence of support for coronavirus conspiracies is only around five-eighths (62.3 percent) of that indicated by the Freeman et al. approach”, but I am not sure I would use “only” here. This is still a lot. In other words, if anything, I am surprised that the flaws in the original study did not matter more, especially when comparing the results to that of Douglas and Sutton (though they rely on a convenience sample).

Second, while the study also aims to provide more valid estimates on the causal effect on beliefs in conspiracy theories in compliance, I am not convinced the authors can say anything meaningful about this. Take this argument in the paper: “Of course, estimates of any causal effect of conspiratorial beliefs on compliance requires not just good measurement but also a move beyond bivariate correlations. By taking a step in that direction with controls for trust in various actors, we have provided a more restrained estimate of the potential effect of conspiracy beliefs on adherence.” Specifically, I am not convinced that simply controlling for trust in various actors will provide better estimates (there are multiple potential pathways between trust in actors, conspiratorial beliefs and compliance that cannot easily be addressed by adding covariates to a multiple linear regression model).

Again, it is great to see additional empirical attention to the question of how many people actually hold conspiratorial beliefs. The numbers in the original study were extreme, but maybe not as extreme as I would have initially thought.