Live Polls And non-Live Polls yield different results

With a backdrop of companies struggling to get both consistent and credible polling, it is interesting to analyse how different polling methods can skew the answers.

FiveThirtyEight generally takes an inclusive attitude towards polls. Our forecast models include polls from pollsters who use traditional methods, i.e., live interviewers. And we include surveys conducted with less tested techniques, such as interactive voice response (or “robopolls”) and online panels. We don’t treat all polls equally — our models account for the methodological quality and past accuracy of each pollster — but we’ll take all the data we can get.

This split, however, between live-interview polls and everything else, is something we keep our eye on. When we launched our general election forecasts in late June, there wasn’t a big difference in the results we were getting from polls using traditional methodologies and polls using newer techniques. Now, it’s pretty clear that Hillary Clinton’s lead over Donald Trump is wider in live-telephone surveys than it is in nonlive surveys.

We don’t know exactly why live-interview polls are getting different results than other types of surveys; there are a lot of potential causes and it’s something we’ll be digging into.

Online polling will become preferred due to the relatively low cost.  But unless the results are of a reasonable quality, live interview polls will remain the more credible.  We saw this recently when some rag commissioned a poll by a never-before-heard-of US company who had never polled on politics New Zealand previously.  The results were ridiculous. 

Nonlive polls may have presented a false signal of competitiveness in some red states. You probably remember hearing about how Kansas and Utah might be in play? That was mostly based on nonlive telephone interview polls. A forecast of just nonlive polls had Clinton within 5 percentage points of Trump in both states on July 1. The live-interview-only forecast, in contrast, had her down double-digits in both states. Since that time, the nonlive polls have come more in line with the live-interview polls — Trump’s regained a double-digit lead in both states even in the nonlive interview forecast.2

As the cases of Utah and Kansas suggest, I’d put more faith in the live-interview polls than in other types of surveys, all else being equal. Indeed, our forecast models do just that. If the gap between live-interview polls and everything else persists, though, we’ll need to explore what might be causing the split and which is more likely to be right.

 

FiveThirtyEight

  • JEL51
  • gaz thewoodsman

    “Hillary Clinton’s lead over Donald Trump is wider in live-telephone surveys than it is in nonlive surveys” – really??

    So why are Trumps events packed to overflowing while Clinton’s events are all empty seats? ABC television ran a live poll regarding Clinton vs Trump
    preference. When the live results showed Trump was ahead by something like 70% vs Clinton 17% they took the poll down.

    It seems none of them can be trusted.

55%