I came across this article by Nate Silver published back in October last year. In the article, Nate Silver formulates a very important point on the difference between all opinion polls and the polls highlighted in the media:
A lot of the value the models provide, as I mentioned, is in looking at all the polls and not just the ones that get highlighted in the media, which are often a highly nonrepresentative sample.
I agree with the above statement that opinion polls in the media often are a nonrepresentative sample of all opinion polls (especially because our article is used as a reference to the statement). That is, if you only look at the media polls reported in the media, you will not be looking at a random (unbiased) sample of opinion polls.
However, while it is fair to call opinion polls reported in the media for a nonrepresentative sample, I would not call it a highly nonrepresentative sample. If you collected all opinion polls reported in the media, you would end up with a different sample, and not have access to all opinion polls, but it will not look fundamentally different.
This is relevant as the point in our research is that biases are present at all stages in the reporting of opinion polls, from the selection of opinion polls to cover in traditional media outlets to how they are shared on social media. Specifically, while there is a difference in the probability of an opinion poll being picked up as a function of the change it shows (even if the change is within the margin of error), this is only one way opinion polls become “nonrepresentative”.
Specifically, it is not only that opinion polls showing greater changes are more likely to end up in the media, they are also more likely to attract attention from pundits and readers. For example, a point we devote more attention to in our book is that opinion polls get shared more on Twitter if they show greater changes.
Accordingly, while opinion polls covered in the media differ slightly from the ‘population of opinion polls’, I would not go as far as to call it a ‘highly nonrepresentative sample’. Or, in other words, I would not put all the blame on media outlets for why and how we end up with biased perceptions of what opinion polls show.