Did welfare policy reforms directly cause Brexit? That’s the conclusion in this article: “Tory austerity and welfare cuts directly caused Brexit, according to a ground-breaking new academic study.”

The results in the new academic study, ‘Did Austerity Cause Brexit?‘, “suggest that the EU referendum could have resulted in a Remain victory had it not been for a range of austerity-induced welfare reforms”. In this post, I outline why I do not believe this conclusion is warranted on the basis of the evidence presented in the study.

I should emphasise that I enjoyed reading the working paper and I sympathise with the empirical approach. The study is interested in a complex topic and combines multiple data sources in order to gather as much evidence as (humanly) possible. Accordingly, regardless of what you might think of my comments on the study, I can highly recommend reading it.

Brexit or UKIP?
While the study is interested in explaining Brexit, it is not really looking at Brexit. Instead, it focuses on the support for the UK Independence Party (UKIP), a party strongly in favour of leaving the European Union. The study presents a lot of results, including some interesting difference-in-difference estimates, but there is only a cross-sectional analysis on the determinants of people’s propensity to vote Leave (available in Table 6 in the manuscript).

There is nothing impressive about these cross-sectional models that provides strong evidence for the story that specific welfare reforms caused Brexit. While the models do not speak against such an interpretation per se, I do believe the manuscript is making too strong claims about Brexit on the basis of such an analysis. In other words, I would say that the manuscript is more about UKIP voting than Brexit. The author is just making it about Brexit as that’s the hot potato at the moment.

Model estimates and aggregate predictions
Despite the fact that the model estimates are interested in UKIP, the author makes the interpretation that three welfare reforms caused Leave to win the Brexit referendum: “Due to the tight link between UKIP vote shares and an area’s support for Leave, simple back of the envelope calculations suggest that Leave support in 2016 could have been up to 9.51 percentage points lower and thus, could have swung the referendum in favor of Remain.” (p. 3) A similar interpretation is available at the LSE British Politics blog: “Had austerity not happened, Leave support could have been up to 10% lower”.

I have three main issues with this interpretation. First, if the manuscript is interested in providing reliable estimates on whether or not welfare reforms caused Leave to win, I would like to see such an analysis be based on more than ‘simple back of the envelope calculations’. Or, alternatively, not be made with such confidence at all.

Second, there is no way I believe in an effect size of 10 percentage points lower support to Leave. From the research I have read on the topic, I know of no single study that find similar effect sizes of specific policy reforms on social outcomes, let alone election outcomes (more on this point below).

Third, while I understand the temptation to look for monocausal explanations of an inherently complex social outcome, the interpretations made in the manuscript are misleading. As my colleague Matthew Goodwin points out, people will read the paper and think “austerity is THE factor”. Such an interpretation, i.e. that austerity is the key driver, would be too simplistic and one should in general be cautious when taking specific model estimates and using them to make aggregate counterfactuals that are unrealistic overestimates at best and heavily misleading and wrong at worst.

What’s the treatment?
Due to a lot of moving parts in British politics, it is difficult to say that one specific policy (our “treatment”) had a causal effect on whether a person voted Leave or Remain. Accordingly, the key challenge is to identify the exact welfare reforms to study and ensure that no other factors related to the reforms drive the results.

In the manuscript, the author describes that he will “focus on three smaller welfare reforms – the abolishment of council tax benefit, the so-called ‘bedroom-tax’ and the introduction of Personal Independence Payments replacing Disability Living allowance” (p. 19). Well, three smaller welfare reforms. Everything is relative but this definitely adds to my skepticism, i.e. that specific welfare reforms caused the Leave vote to win.

Importantly, the manuscript also pays attention to aggregate data, but these analyses are more difficult to link to specific reforms. The question here is: What is the actual treatment!? Take the following two results from the paper on the correlates of UKIP support (after 2010):

1. Human capital: “[…] support for UKIP gradually trends up as a function of the share of the resident population with low educational attainment. The correlation between support for UKIP and the measure of low human capital only becomes sharply stronger after 2010.” (p. 14)
2. Employment shares: “Areas with larger employment shares in Retail, and Manufacturing saw significant increases in electoral support for UKIP after 2010. To get a sense of the magnitude, for the Manufacturing sector (ca. 15.4% of employment in 2001), the point estimate of 0.53 in 2015 suggests that the average area saw an expansion in support for UKIP by 2015 by 8.1 percentage points.” (p. 15)

These findings are not related to specific policies. However, the first empirical analysis in the manuscript concludes that UKIP gained “support after 2010 in areas with low skilled, working in routine jobs or the retail sector remain intact.” (p. 15)

I see no reason to question this finding, but human capital and employment shares are not only linked to austerity-induced welfare reforms but a series of factors. The interesting question is whether we would still believe the main findings (i.e. those based on the individual-level analysis) if we did not find a stronger correlation between support for UKIP and human capital only after 2010. This is not necessarily entering the garden of forking paths, but we are heading towards that territory. There is simply no convincing evidence for an effect of specific policies in this part of the analysis.

However, this points to two more serious issues with the inferences made in the study. First, we should question the parallel trend assumption in relation to the exposure to welfare reforms. The specific reforms are much more likely to affect poorer places (one of the studies cited in the manuscript shows that) and I see no reason to believe that nothing else happened in the poorer places that would not also correlate with UKIP support. In other words, a person that is more likely to be affected by austerity-induced welfare reforms is also more likely to be affected by other social, political and economic changes compared to a person not affected by the reforms.

Second, and returning to the point on the effect sizes, for the three reforms being studied, only 10% of the sample experienced one of the three reforms. This is not a problem as it is similar to the proportion in the population (10% of all UK households). However, if we are working with strong effects that could basically change the outcome of the referendum, this should mean that a nontrivial proportion of the affected group should change their vote choice to Leave (not taking differential turnout into account and assuming that they would vote Remain otherwise) as a result of the reforms – and not other factors.

What’s the mechanism?
This might come as a huge surprise for the economists reading this post, but here we go: Political scientists have been studying the impact of welfare policies on political behaviour for decades… (Including my recent study on the impact of welfare retrenchments on government support.)

This is my main issue with the study. There is too much going on data-wise with limited theoretical reflections and no engagement with the relevant literature in political science. One of the main findings in the political science literature is that there are multiple different mechanisms linking welfare policies to mass publics. There are no reflections on these different mechanisms in the manuscript. A good starting point to this literature is the seminal research by Paul Pierson from the early 1990s, including his book Dismantling the Welfare State? Reagan, Thatcher and the Politics of Retrenchment.

However, we do see the acknowledgement that there is something going on that might explain why welfare reforms are linked to the Brexit vote. Specifically, the study looks at “Perception of politics more broadly”, or as political scientists call it, political efficacy (the manuscript does not mention political efficacy at all). These measures include the perceptions that “Public officials do not care”, that respondents “Don’t have a say in what government does” and that “your vote is unlikely to make a difference” (p. 34). Again, there is a lot of relevant literature within political science on how welfare policies matter for political efficacy.

In a forthcoming article in Policy Studies Journal, I review 65 studies interested in estimating the impact of policies on mass publics. None of the 65 studies are cited in the manuscript. However, for people interested in the topic, I will recommend a few that are of interest in this context. First, for some studies using strong and interesting research designs to study how policies matter for political participation, see Davenport (2015), Flavin and Griffin (2009) and Flavin and Hartney (2015).

For the specific study of welfare reforms and the Brexit vote, three studies come to mind. First, Watson (2015) looks at the evolution of conditional welfare programs in Britain from 1996 to 2013 and uses panel data to trace the behaviour of individuals affected by these policies over time. Second, Gingrich (2014) looks at how welfare policies, and in particular how visible they are to the public, matter for whether people link their policy preferences to voting for a right-wing party. Third, in Spain, Muñoz et al. (2014) use panel data to study how an austerity package shaped political participation.

Last, for some additional studies interested in the impact of welfare policies on political efficacy and participation, see Guo and Ting (2015), MacLean (2011), Mettler and Stonecash (2008), Soss (1999), Soss and Schram (2007) and Swartz et al. (2009). All of these studies can help inform the manuscript in question on the mechanism(s) linking austerity-induced welfare reforms to the Brexit vote.

Economic versus cultural explanations
Altogether, this brings us to another issue with the manuscript, namely the focus on a single explanation for the Brexit vote rooted in economics. This issue goes beyond the specific manuscript and is symptomatic for a trend in the literature, i.e. to make it about a horserace between economic and cultural factors in understanding populist sentiments.

The literature is still in an early stage of disentangling the interplay between different factors, and it might as well be that the individual propensity to vote Remain was shaped by the interplay between economic and cultural factors. This is a possibility Pippa Norris outlines in a new working paper: “These theories can also be regarded as complimentary rather than rivals, for example if economic deprivation catalyzed resentment about immigrants and the rejection of open borders.”

Still, academics tend to reduce these complex questions to the simple horserace. A recent example is the new study in PPNAS by Diana Mutz, arguing that “Status threat, not economic hardship, explains the 2016 presidential vote”. I am sure the study is great (although see this paper), but simply ruling out the potential direct and/or indirect role of economic hardship based on a couple of regressions seems unfair to the theory.

This is why I am not convinced that the manuscript provides convincing evidence that the absence of specific welfare reforms would have resulted in a Leave win (the counterfactual claim). There is no data in the manuscript on immigration/culture that allows the author to study these alternative explanations or their respective role in shaping the Brexit vote. In other words, even if we believe in the results in the study (including the simple back of the envelope calculations), it is not possible to conclude that “austerity measures, not X” caused Brexit.

Last, the author of the study told HuffPost UK “that the withdrawal of welfare in poor areas allowed the Vote Leave campaign to “exploit” underlying worries about EU immigration and claims that billions were being spent on Brussels rather than at home in Britain.” While this is all plausible it yet again points to the fact that we are working with multiple potential mechanisms linking policies to political behaviour, and going for a simple economic explanation unrelated to any cultural factors is too simplistic.

Wrapping up
In my opinion, the evidence presented in the manuscript is not strong enough to make the point that welfare reforms caused Brexit. Did austerity-induced welfare reforms play a crucial role in British politics with implications for the Brexit referendum? Maybe, but I am not convinced.

The manuscript makes a good case for why we need to take welfare reforms serious in order to understand political behaviour, and I do believe there are reasons to expect that certain reforms made people, directly or/and indirectly, more likely to vote Leave. However, I have yet to see convincing evidence that “welfare cuts directly caused Brexit”.