With a backdrop of companies struggling to get both consistent and credible polling, it is interesting to analyse how different polling methods can skew the answers.
FiveThirtyEight generally takes an inclusive attitude towards polls. Our forecast models include polls from pollsters who use traditional methods, i.e., live interviewers. And we include surveys conducted with less tested techniques, such as interactive voice response (or “robopolls”) and online panels. We don’t treat all polls equally — our models account for the methodological quality and past accuracy of each pollster — but we’ll take all the data we can get.
This split, however, between live-interview polls and everything else, is something we keep our eye on. When we launched our general election forecasts in late June, there wasn’t a big difference in the results we were getting from polls using traditional methodologies and polls using newer techniques. Now, it’s pretty clear that Hillary Clinton’s lead over Donald Trump is wider in live-telephone surveys than it is in nonlive surveys.
We don’t know exactly why live-interview polls are getting different results than other types of surveys; there are a lot of potential causes and it’s something we’ll be digging into.
Online polling will become preferred due to the relatively low cost. But unless the results are of a reasonable quality, live interview polls will remain the more credible. We saw this recently when some rag commissioned a poll by a never-before-heard-of US company who had never polled on politics New Zealand previously. The results were ridiculous. Read more »