Why the Polls Got It Wrong: Deliberate Versus Accidental Survey Bias

To understand ‘deliberate bias’ among census and surveys, we can look all the way back to the Jewish Bible.
The Book of Numbers in the Jewish Bible is named as such because it begins with a partial census of the Israelites. Partial because Moses tallies up only fighting men over the age of 20 while ignoring women and children. This was deliberate bias.
Now, let’s look at ‘accidental bias.’ We can look at major events like ‘Brexit’ and the recent Trump victory in the U. S. election.
The consensus was overwhelmingly wrong because of accidental survey bias.
It’s the difference between theory and reality, and it gets tripped up in the basics of methodology. Current approaches follow a wisdom-of-the-crowds approach where multiple predictions are gathered, the extremely divergent results get dumped, and the more common result gets used. We see it in the Olympics where the low score and the high score get dumped.
But it keeps failing:
Polls missed Brexit Polls missed Trump win Betting websites miss both The common problem: the surveys are biased in favor of white collar workers.

This post was published at FinancialSense on 11/23/2016.