While opinion polls are great they are also subject to a multitude of potential systematic errors. Some of these errors are related to the fact that polling firms rely on specific methods that might shape the results (so-called ‘house effects’). Some firms, for example, rely on internet panels when they recruit respondents, whereas other firms call people on their phones. Such differences might affect the results in opinion polls.
In an analysis of all Danish opinion polls on the public support for political parties from 2010 to 2017 (n=1,062), Zoltán Fazekas and I examined whether such house effects are present for the national political parties. In doing this, we relied on the Bayesian approach described in Jackman (2005) to estimate house effects for each of the 10 parties (90 estimates in total given the 9 polling firms).
Figure 1: House effects in Danish opinion polls, 2010-2017
Figure 1 presents the results with the 10 parties on the vertical axis and the nine polling firms on the horizontal. If there are no house effects in the polls for a party, we will see no circles on the horizontal line next to a party. The greater a house effect is for a party in the polls from a specific polling firm, the greater the circle will be (the size of the circle is proportional to the magnitude of the house effect). When the house effect is negative, i.e. the polling firm estimate a lower support for a party, the circle is red. The blue circles are for positive deviations. The gray color is for effects where 0 falls within the 95% credible interval.
TV 2 covered the analysis (in Danish) with additional interpretations of the results. Subsequently, the analysis was also covered by Mandag Morgen (also in Danish).