Erik Gahner Larsen

Which party do you think is most likely to agree?

In a new poll, JL Partners surveyed more than 2,000 respondents to understand how the public perceives the Labour Party and the Conservative Party on a series of different “woke” topics. The Daily Mail uses the poll to conclude that “Sir Keir Starmer’s Labour Party is out of touch with public opinion”. Is that what the poll finds? No.

The question wording was “For each of the following issues, please indicate which of the main two parties is more likely to agree with it.” and the ideas include statements such as “Statues of historical figures being taken down” and “Shorter sentences for criminals”. The figure below shows the results together with information on the percentage of the public supporting these ideas.

The Campaign For Common Sense, which has nothing to do with common sense, concluded that “the Conservatives can confidently take a stand on these issues knowing that they are more in tune with the British public.” However, it is not possible to use the data in the survey to draw such a conclusion.

The problem is that we cannot use the question to understand 1) whether any of the parties actually support the ideas (both parties might reject them) or 2) whether both parties actually support the ideas (both parties might support them).

To understand why, consider if the idea had been “An income tax rate of 75%”. Most people would say that Labour would be most likely to agree with this idea, but that is not the same as Labour would support such an idea. And most voters would most likely say, if asked, that Labour is not currently supporting such an idea. For that reason, it is not possible to use the answers in the survey to say anything meaningful about whether any of the parties are more in tune with the British public in relation to the specific ideas.

If the Campaign For Common Sense was interested in showing what party was most in tune with the British public, they would need to ask questions about whether the public believes that the parties support certain ideas – not about which party is most likely to agree.

The Effect of Question Wording: Climate Edition

It is well known that the question wording in a survey may influence the respondents answer. Take as an example the pioneer study (pdf) on framing effects by Amos Tversky and Daniel Kahneman, published in Science, which shows that presenting the same action with different words, in this case whether people will be saved or die, affects the respondents action choice.

A lot of different studies investigate the effects of different wordings in survey questions. An interesting paper by Jonathon P. Schuldt, Sara H. Konrath and Norbert Schwarz shows, that presenting respondents with either the words “global warming” or “climate change”, influence the evaluation of whether the climate is changing or not – but only for republicans. From the abstract:

Republicans were less likely to endorse that the phenomenon is real when it was referred to as “global warming” (44.0%) rather than “climate change” (60.2%), whereas Democrats were unaffected by question wording (86.9% vs. 86.4%).

The experimental design (as well as the whole paper) is simple (and easy to read), but I would like to see a clearer investigation of the causal mechanism, in order to explain the individual differences (being more than simply a democrat or republican), maybe by using some sort of within subjects design.