Pollsters and political opinion polls: why so many prediction errors?

Why pollsters get it wrong so often? How reliable are polls? We discuss the main causes of prediction errors. Participate in our debate and invite other to join

Debate created by Kenneth Bunker

Pollsters and political opinion poll errors

In 2017, the Conservative party won the UK general elections but they obtained considerably fewer MPs than most opinion polls and poll aggregators had predicted. In fact the snap election was to a great extent called by Theresa May based on the advantage that opinion polls gave her earlier in the year. That is not the only or worst example of pollsters errors. In 2016  the polls got it wrong in predicting two major events: the Brexit referendum and the US presidential election. Most opinion polls predicted a victory of the Remain camp. Although most pollsters gave a narrow advantage to those in favor of staying in the EU, some of the polls conducted by telephone calculated an 18-point lead. The previous year, during the 2015 British general election, the pollsters' prediction errors were of even a higher magnitude. None of them expected David Cameron's Conservative Party to enjoy such a landslide victory and majority in the Parliament. The American presidential elections have demonstrated again that pollsters get it wrong. Poll-based forecasts and exit polls failed to predict a Donald Trump victory.

These are only the last two polling errors in a series of elections in which pollsters have failed to predict the correct result. This would hardly be a problem if election polls weren’t so influential in electoral cycles. The numbers polls report and the media replicates have echoes in issues as diverse as internal factors such as campaign planning and everyday political affairs, as well as external issues such as international political relations and foreign investment. 

But how reliable are polls? When pollsters say that their predications are only off by a small margin (usually around 3%), media, voters, investors and other interested parties all believe them. These recent failures fuel the idea that election polls can no longer be trusted. Indeed, trusting polls when they are wrong can change the results of elections by creating incentives for voters to stay home if they believe their candidate will win anyway, or of potential foreign investors of thinking twice before initiating a business relation.

It is implicit that polls are important tools for anticipating electoral results. So it is crucial to understand why they are failing to bring them back to their standards. 

Why do pollsters get it wrong? 

There seem to be three major reasons that could explain why polls fail: 

• One reason is that there are serious involuntary methodological faults. It could be that polls do not estimate correctly the demographics and thus do not weigh results according to the true distribution in the population. It could also be that the sampling methods are incorrect, and polls are not reaching a representative group of people. This could be due to the financial restrictions of pollsters that have led to preferring online or telephone poll’s rather than face-to-face nationwide polls. 
• A second reason is that there is an intentional manipulation of results. This suggests that pollsters have an agenda. It means that there is no serious fault in the methods, but a political purpose of distorting results. While this is hard to prove, since pollsters do not normally release their data, it is possible. Pollsters could, for example, change the weight that they give to different portions of the population, say age, race or geographical regions, in a manner which is convenient to them. However, this would only make sense if pollsters are confabulated or herding.
• A third reason is that voters are not answering pollster’s questions honestly. It is perfectly possible that voters are declaring that they will not vote when they finally do vote, or that voters who declare that they will vote are giving false answers. This could explain why the point estimates and the margins of error are far off the actual result of elections. Polls are built on the assumption that respondents are honest. If respondents lie, or change their opinion after the poll, then it is only natural that polls will not be able to predict results accurately. 

Dr Kenneth Bunker is a political scientist at Universidad Central de Chile and director of the electoral and political analysis website Tresquintos 

Watch these videos about polling mistakes

You may also find interesting our debates: Should 16 year old be allowed to vote? and Causes of political polarization in America

Vote to see result and collect 1 XP. Your vote is anonymous.
If you change your mind, you can change your vote simply by clicking on another option.