Why pollsters get it wrong so often? How reliable are polls? We discuss the main causes of prediction errors. Participate in our debate and invite other to join
Debate created by Kenneth Bunker
Pollsters and political opinion poll errors
In 2016 the polls got it wrong in predicting two major events: the Brexit referendum and the US presidential election. Most opinion polls predicted a victory of the Remain camp. Although most pollsters gave a narrow advantage to those in favor of staying in the EU, some of the polls conducted by telephone calculated an 18-point lead. The previous year, during the 2015 British general election, the pollsters' prediction errors were of even a higher magnitude. None of them expected David Cameron's Conservative Party to enjoy such a landslide victory and majority in the Parliament. The American presidential elections have demonstrated again that pollsters get it wrong. Poll-based forecasts and exit polls failed to predict a Donald Trump victory.
|STATE||AVG. POLLING MARGIN||ELECTION RESULT||OVERPERFORMANCE|
These are only the last two polling errors in a series of elections in which pollsters have failed to predict the correct result. This would hardly be a problem if election polls weren’t so influential in electoral cycles. The numbers polls report and the media replicates have echoes in issues as diverse as internal factors such as campaign planning and everyday political affairs, as well as external issues such as international political relations and foreign investment.
But how reliable are polls? When pollsters say that their predications are only off by a small margin (usually around 3%), media, voters, investors and other interested parties all believe them. These recent failures fuel the idea that election polls can no longer be trusted. Indeed, trusting polls when they are wrong can change the results of elections by creating incentives for voters to stay home if they believe their candidate will win anyway, or of potential foreign investors of thinking twice before initiating a business relation.
It is implicit that polls are important tools for anticipating electoral results. So it is crucial to understand why they are failing to bring them back to their standards.
Why do pollsters get it wrong?
There seem to be three major reasons that could explain why polls fail:
- One reason is that there are serious involuntary methodological faults. It could be that polls do not estimate correctly the demographics and thus do not weigh results according to the true distribution in the population. It could also be that the sampling methods are incorrect, and polls are not reaching a representative group of people. This could be due to the financial restrictions of pollsters that have led to preferring online or telephone poll’s rather than face-to-face nationwide polls.
- A second reason is that there is an intentional manipulation of results. This suggests that pollsters have an agenda. It means that there is no serious fault in the methods, but a political purpose of distorting results. While this is hard to prove, since pollsters do not normally release their data, it is possible. Pollsters could, for example, change the weight that they give to different portions of the population, say age, race or geographical regions, in a manner which is convenient to them. However, this would only make sense if pollsters are confabulated or herding.
- A third reason is that voters are not answering pollster’s questions honestly. It is perfectly possible that voters are declaring that they will not vote when they finally do vote, or that voters who declare that they will vote are giving false answers. This could explain why the point estimates and the margins of error are far off the actual result of elections. Polls are built on the assumption that respondents are honest. If respondents lie, or change their opinion after the poll, then it is only natural that polls will not be able to predict results accurately.
Dr Kenneth Bunker is a political scientist at Universidad Central de Chile and director of the electoral and political analysis website Tresquintos
Watch these videos about polling mistakes
If you change your mind, you can change your vote simply by clicking on another option.
Pollsters and political opinion polls: why so many prediction errors?
New to netivist?
Join with confidence, netivist is completely advertisement free. You will not receive any promotional materials from third parties.