Last year I wrote a post on the endless supply of contextual factors that (might) explain support for Brexit. In brief, I argued that a potential problem with the different studies examining the different factors is that they only contribute to the illusion of cumulative science. That is, just because a new study is able to predict support for Brexit with a new variable, we are not necessarily better off understanding or explaining Brexit.
A new study, published in Political Behavior, finds that wind speed can predict the Brexit remain vote. Specifically, the higher levels of wind speed on Election Day in councils in the UK, the greater the Remain vote share in said councils. The paper presents a sensible theoretical case for why wind speed should make people more likely to be against Brexit (i.e., vote remain). In addition, the paper provides evidence from other cases (where wind speed is related to more ‘prevention‐focused voting’), including experimental evidence, and all the data is publicly available (you love to see it).
In the grand scheme of things, while the paper in question introduces a theory for why wind speed should matter for Brexit support, I am not convinced we are better off understanding Brexit in the light of this study. We are at best able to confirm the conclusion that there are many causes of Brexit, and it is unclear whether the study can explain Brexit better than regional differences. Worst case scenario, we are introducing yet another variable that will not improve our understanding of why people voted ‘Remain’ or ‘Leave’.
The paper is explicit about wind speed not being a true natural experiment (i.e., wind speed on Election Day in councils is not as-if random). For that reason, the statistical models in the paper control for, among other things, socio-demographic characteristics. However, interestingly, none of the models control for what part of the UK a council is located in. I find this interesting as there is substantial within-region variation in wind speed in different councils. Here is a figure showing the relationship between ‘Remain’ vote (in %) and Election Day wind speed (in metre per second):
You see the highest (average) wind speed values in regions such as Northern Ireland, Wales, Scotland and London, and the lowest in regions such as the East Midlands and West Midlands. My concern is that if controlling for the region of the council, any statistically significant effect will be … gone with the wind. Alas, that seems to be the case. In a simple bivariate OLS regression, the wind speed coefficient is not only no longer statistically significant, but the sign of the effect is in the opposite direction (i.e., a negative effect on the Remain vote share). Similarly, the effect of wind speed is no longer statistically significant if you include a dummy control for whether the council is in England or not.
I am not saying that this is conclusive evidence against the hypothesis of interest (the authors might have a sensible argument for not including the region variable as a covariate in the models that I have missed). However, I do find the findings worth discussing for two reasons. First, the authors are not only interested in predicting the Brexit vote share, and some of the concerns highlighted here might be of interest for the other studies in the paper. Second, the focus on how the weather (might) have shaped support for Brexit is a good case to discuss the many causes of Brexit (even if the findings were robust when controlling for the region of the unit of analysis, i.e., the council).
This is not the first study to look at the relevance of the weather on Election Day for the Brexit referendum. A study published in Political Geography found that “if the referendum had taken place on a sunny day, there would have been a small increase in the margin of victory for Vote Leave”. The study found that most Remain votes were lost to rainfall in Hackney (in London), and the most Leave votes were lost to rainfall in Hillingdon (also in London). However, would these conclusions still be the same if we model the influence of wind speed? And what does the DAG look like for such a model of the world?
This brings us to the great paper by Jonathan Mellon on the hundreds of potential exclusion-restriction violations for studies using weather as an instrumental variable. In the most recent version of the paper, there are 14 examples of studies using wind speed as an instrument. Again, wind speed is not a true experiment and if wind speed also affects other outcomes, we need to be concerned about the exclusion restriction.
We now have a few studies examining the relevance of weather for the Brexit outcome (see also this study that is cited in the paper by Jonathan Mellon). These studies rely on different theories to make their argument(s) and different data sources to find empirical support. I am sure it is only a matter of time before we will see yet another study with an argument for how and why the weather is (causally) related to Brexit support, and I believe that should make us (re)consider how much we can infer about the relevance of such events (and other events) in explaining support for Brexit (or worse, the outcome of the referendum).
For all the studies we have now on the causes of Brexit, including the few studies interested in the relevance of the weather on Election Day, we have a similar amount of concerns related to how to aggregate and make sense of all these studies – if they are all true. In sum, I am not fully convinced that the weather on Election Day had a significant impact on how people voted (but maybe on whether people voted in the first place).