Which pollsters can you trust, and which are bought and paid for?

We should have something like this here.


In the process of being an analyser of polls, Nate Silver has had to figure out which are actually delivering reliable and honest results, and which are essentially fronts that get you the answer you’re paying for.

We know that Colmar Brunton are one of the most variable and have a clear bias towards Labour, for example.

As for Horizon, they had the Conservative party in government as part of the coalition with National right now.

Whenever the Fraser House spy tells me that Labour are polling in the low to mid-thirties, and then paid-for polls come out, you can reliably subtract 4-6 points, every time.


– 538

Hacking the system: how Trump took the lead

Nate Silver writes at FiveThirtryEight about how Trump hacked the system.

As is usual with Nate Silver there is plenty of data and explanations before he gets to his conclusions:

Most of the media’s self-criticism of its Trump coverage has focused on whether Trump’s dominance of the news cycle reflects a craven desire for higher TV ratings or Web traffic numbers. It’s fine to debate that — although these criticisms are sometimes being evinced through crocodile tears given the record ratings and traffic Trump is bringing to news organizations of all kinds.12

But this critique avoids some thornier questions. For instance, with his ability to make news any time he wants with a tweet, news conference or conveniently placed leak, Trump has challenged news organizations’ editorial prerogative. Should the press cover a candidate differently when he makes trolling the media an explicit part of his strategy, on the theory that some coverage is almost always better than none?    Read more »

Nate Silver leaves NY Times, takes his blog to ESPN

This is big news. Nate Silver has packed up his blog at NY Times and moved to ESPN. His traffic will go with him. That is the nature of blogs where personalities are followed not mastheads.

Nate Silver, famous for his eerily accurate election predictions, is dumping the Gray Lady for the network of Keith Olbermann. The math wizard is taking his FiveThirtyEight blog — which was a must-read during the 2012 presidential election — and jumping ship to ESPN, reports his former co-worker Brian Stelter in The Times. Silver will now write and crunch numbers for the sports network while also “most likely” contributing to Keith Olbermann’s new show, according to theTimes report. In what is a classic Times-ian understatement,  Stelter writes, “[Silver’s] departure will most likely be interpreted as a blow to the company.”

To which one might say: ya think?   Read more »

One Man Traffic Machine

Nate Silver’s approach has been vindicated and he is a one man traffic machine for the New York Times, watch this video as he explains the polls:

America goes to the polls

America votes. Here are some last minute thoughts from pundits and commentators.

Ezra Klein says the polls are right:

The polls will prove to be right. President Obama will win with 290 electoral votes. I’m not extremely confident in the precision of that estimate: Some swing states are close enough that it’s entirely possible for a good ground game to tip, say, Florida into Obama’s column, or Colorado into Romney’s. Virginia is basically tied, and I’m giving it to Romney based on the assumption that challenger wins in a tie, but it could easily go the other way. So if Obama ends up winning with 303, I won’t be surprised.

That said, 290 is what a conservative read of the polls says, as of this moment. And I trust the polls more than I trust my intuition, or the fragmented, impressionistic reporting on the two GOTV efforts. So I’m going with that.

Nate Silver says;

In order for Mr. Romney to win the Electoral College, a large number of polls, across these states and others, would have to be in error, perhaps because they overestimated Democratic turnout. It’s this possibility, more than the chance of a successful hail-mary in a state like Pennsylvania, that accounts for most of Mr. Romney’s remaining chances of winning the Electoral College.

There is also the chance that Mr. Obama could finish toward the higher end of the polling range in most states. If Mr. Obama has gained a point or two nationally because of Hurricane Sandy or other factors, then polls taken before it may underestimate his standing in the individual states as well.

The FiveThirtyEight forecast accounts for this possibility through its trendline adjustment, which is why our forecasts now are slightly more optimistic for Mr. Obama in some states than a simple average of polls. Had there been evidence of late movement toward Mr. Romney, the trendline adjustment would instead have worked in his favor.

But Mr. Romney’s chances of winning the Electoral College have slipped, and are now only about 8 percent according to the forecast model — down from about 30 percent 10 days ago.

He adds:

All of this leaves Mr. Romney drawing to an inside straight. I hope you’ll excuse the cliche, but it’s appropriate here: in poker, making an inside straight requires you to catch one of 4 cards out of 48 remaining in the deck, the chances of which are about 8 percent. Those are now about Mr. Romney’s chances of winning the Electoral College, according to the FiveThirtyEight forecast

As any poker player knows, those 8 percent chances do come up once in a while. If it happens this year, then a lot of polling firms will have to re-examine their assumptions — and we will have to re-examine ours about how trustworthy the polls are. But the odds are that Mr. Obama will win another term.

Michael Tomasky at the Daily Beast thinks Republicans are dreamin’:

There’s been just one presidential election in the history of modern polling–since 1952–in which Republican turnout matched Democratic turnout. That was 2004, when exit pols had both at 37 percent.

So Republicans matched the Democratic vote once in 15 elections, and now they’re going to be an unprecedented 6 percent higher? If that turns out to be true, I’ll be the first to say so. History would not suggest that it’s all that likely. The 2004 race represented the the biggest ground game the GOP has ever assembled, and it went to work against a Democratic one that at least on paper doesn’t come close to matching the Obama operations.

Fred Barnes reckons Mitt Romney will win…seriously he does:

In 2008, self-identified Democrats led Republicans in turnout by seven percentage points.  Gallup’s projection is that Republicans will have a 49-46 percent edge this year.  “The political environment and the composition of the likely electorate strongly favor Governor Romney,” Goeas says.  The Battleground Poll’s “vote election model” projects Romney with 51 percent.

Is it a tossup?

Not according to Nate Silver, and here is the math behind that:

There were 22 polls of swing states published Friday. Of these, Mr. Obama led in 19 polls, and two showed a tie. Mitt Romney led in just one of the surveys, a Mason-Dixon poll of Florida.

Although the fact that Mr. Obama held the lead in so many polls is partly coincidental — there weren’t any polls of North Carolina on Friday, for instance, which is Mr. Romney’s strongest battleground state — they nevertheless represent powerful evidence against the idea that the race is a “tossup.” A tossup race isn’t likely to produce 19 leads for one candidate and one for the other — any more than a fair coin is likely to come up heads 19 times and tails just once in 20 tosses. (The probability of a fair coin doing so is about 1 chance in 50,000.)

Instead, Mr. Romney will have to hope that the coin isn’t fair, and instead has been weighted to Mr. Obama’s advantage. In other words, he’ll have to hope that the polls have been biased in Mr. Obama’s favor.

1 in 50,000 is not good odds. Nate then discusses possibilities of errors in polling caused by statistical sampling errors and time factors involving snapshot polls. He explains how his model and other polling companies account for those errors. Then he discusses the biggest allegation about polling, that of bias:

This introduces the possibility that most of the pollsters could err on one or another side — whether in Mr. Obama’s direction, or Mr. Romney’s. In a statistical sense, we would call this bias: that the polls are not taking an accurate sample of the voter population. If there is such a bias, furthermore, it is likely to be correlated across different states, especially if they are demographically similar. If either of the candidates beats his polls in Wisconsin, he is also likely to do so in Minnesota.

The FiveThirtyEight forecast accounts for this possibility. Its estimates of the uncertainty in the race are based on how accurate the polls have been under real-world conditions since 1968, and not the idealized assumption that random sampling error alone accounts for entire reason for doubt.

To be exceptionally clear: I do not mean to imply that the polls are biased in Mr. Obama’s favor. But there is the chance that they could be biased in either direction. If they are biased in Mr. Obama’s favor, then Mr. Romney could still win; the race is close enough. If they are biased in Mr. Romney’s favor, then Mr. Obama will win by a wider-than-expected margin, but since Mr. Obama is the favorite anyway, this will not change who sleeps in the White House on Jan. 20.

My argument, rather, is this: we’ve about reached the point where if Mr. Romney wins, it can only be because the polls have been biased against him. Almost all of the chance that Mr. Romney has in the FiveThirtyEight forecast, about 16 percent to win the Electoral College, reflects this possibility.

Yes, of course: most of the arguments that the polls are necessarily biased against Mr. Romney reflect little more than wishful thinking.

So what about the claim that it is a toss-up:

Nevertheless, these arguments are potentially more intellectually coherent than the ones that propose that the race is “too close to call.” It isn’t. If the state polls are right, then Mr. Obama will win the Electoral College. If you can’t acknowledge that after a day when Mr. Obama leads 19 out of 20 swing-state polls, then you should abandon the pretense that your goal is to inform rather than entertain the public.

But the state polls may not be right. They could be biased. Based on the historical reliability of polls, we put the chance that they will be biased enough to elect Mr. Romney at 16 percent.


The Debates

The presidential debates kick off today. Many commenters claim that Romney is going to kick ass and teach Obama a thing or two in the debates.

He needs to…because it is his last hope. Those same commenters have used all manner of excuses to blind themselves to reality…that Mitt Romney is a plastic candidate with a wooden demeanour and a tits campaign. They claimed that Romney was waiting till confirmation to spend and that once the spending got underway things would change…it didn’t and hasn’t. They claimed that using Ronald Reagan’s mantra would work…except Romney can;t deliver the lines as convincingly as Ronald Reagan could. They claim that the media and polls are all bias. Now while I have some sympathy with the bias of the media the same cannot be said of the polls. That myth has been thoroughly debunked.

However they may be right when it comes to debates, because traditionally the polls favour the challenger after the first debate:

[T]he first debate has normally helped the challenger. In the nine elections between 1976 and 2008, there were only two years when the incumbent-party candidate gained ground relative to the challenger; these cases were 1976, when Gerald R. Ford halved his six-point deficit with Jimmy Carter, and 1988, when George H.W. Bush moved just slightly further ahead of Michael Dukakis.

But on average, the challenging-party candidate gained a net of one and a half percentage points on the incumbent-party candidate.

There is a slight catch though:

[T]he challenger’s gains have come mainly from undecided voters rather than from the incumbent himself.

If you look at the poll average for the incumbent and challenging candidates individually, the incumbent candidate held steady on average at 45.5 percent of the vote both before and after the debate. However, the challenger improved to 43.9 percent of the vote from 42.4 percent.

So a reasonable best guess, based on the historical precedent and without considering any factors specific to this race, is that Mr. Romney will gain a point or two in the polls by next week, while Mr. Obama’s number will hold steady.

Mitt Romney needs a blinder, he really needs to score big, unfortunately that just isn’t going to happen. He may get a modest bounce but it is unlikely to be enough.

If he doesn’t get a bounce though, as history suggests he should, then he is in deep, deep trouble.

[H]ere’s the bad news for Mr. Romney: no candidate who trailed by as much he did heading into the first debate went on to win the election. In the two cases where the lead reversed after the debate, 1980 and 2000, the trailing candidate was down only one or two points in the polls. The FiveThirtyEight “now-cast,” conversely, pegs Mr. Romney’s deficit at about 5 points instead. (Other methods put it at between three and four points.)

More bad news for Mr. Romney: although there has been a tendency for the challenging candidate to gain ground immediately after the first debate, there has not been any tendency for the challenger to gain over the remaining weeks of the election. On average during these years, the challenging candidate trailed by 1.5 percentage points in polls conducted just after the first debate — and the challenger eventually lost the election, on average, by 1.4 percentage points, a nearly identical margin.

Debunking Oversampling

Rabid Republican supporters have constantly attacked me for not following their particular party line in predicting a Romney win. I even have a bet with Leighton Smith that Obama will win, not that I want him to win but simply because the math, the polls and the facts do not support any other contention.

In effect it is me with the safe bet. Nonetheless I get constant emails pointing out the error of my ways, constant comments doing the same and people pointing me to discredited loons who have come up with a conspiracy that suggests that all polling companies with the exception of Rasmussen are colluding to keep Obama president in conjunction with the liberal media paymasters.

Me? I prefer facts. Nate Silver looks at the current conspiracy of the day…oversampling:

In 2004, Democratic Web sites were convinced that the polls were biased toward George W. Bush, asserting that they showed an implausible gain in the number of voters identifying as Republicans. But in fact, the polls were very near the actual result. Mr. Bush defeated John Kerry by 2.5 percentage points, close to (in fact just slightly better than) the 1- or 2-point lead that he had on average in the final polls. Exit polls that year found an equal number of voters describing themselves as Democrats and Republicans, also close to what the polls had predicted.

Since President Obama gained ground in the polls after the Democrats’ convention, it has been the Republicans’ turn to make the same accusations. Some have said that the polls are “oversampling” Democrats and producing results that are biased in Mr. Obama’s favor. One Web site,, contends that even Fox News is part of the racket in what it says is a “trend of skewed polls that oversample Democratic voters to produce results favorable for the president.”

People forget that these conspiracies follow the cycles. They also forget that they are wrong.

The criticisms are largely unsound, especially when couched in terms like “oversampling,” which implies that pollsters are deliberately rigging their samples.

But pollsters, at least if they are following the industry’s standard guidelines, do not choose how many Democrats, Republicans or independent voters to put into their samples — any more than they choose the number of voters for Mr. Obama or Mitt Romney. Instead, this is determined by the responses of the voters that they reach after calling random numbers from telephone directories or registered voter lists.

Pollsters will re-weight their numbers if the demographics of their sample diverge from Census Bureau data. For instance, it is typically more challenging to get younger voters on the phone, so most pollsters weight their samples by age to remedy this problem.

So what about the charge of partisan bias:

If the focus on “oversampling” and party identification is misplaced, however, FiveThirtyEight does encourage a healthy skepticism toward polling. Polling is difficult, after all, in an era in which even the best pollsters struggle to get 10 percent of households to return their calls — and then have to hope that the people who do answer the surveys are representative of those who do not.

So perhaps we should ask a more fundamental question: Do the polls have a history of being biased toward one party or the other?

The polls have no such history of partisan bias, at least not on a consistent basis. There have been years, like 1980 and 1994, when the polls did underestimate the standing of Republicans. But there have been others, like 2000 and 2006, when they underestimated the standing of Democrats.

What does Nate Silver do that is different?

We have an extensive database of thousands of polls of presidential and United States Senate elections. For the presidency, I will be using all polls since 1972, which is the point at which state-by-state surveys became more common and our database coverage becomes more comprehensive. For the Senate, I will be using all polls since 1990.

That is a pretty impressive amount of data, and one of the reasons why Nate Silver is the most accurate political statistician and why I follow his predictions. What does the data actually say, rather than the partisan hackery:

In the 10 presidential elections since 1972, there have been five years (1976, 1980, 1992, 1996 and 2004) in which the national presidential polls overestimated the standing of the Democratic candidate. However, there were also four years (1972, 1984, 1988 and 2000) in which they overestimated the standing of the Republican. Finally, there was 2008, when the average of likely voter polls showed Mr. Obama winning by 7.3 percentage points, his exact margin of victory over John McCain, to the decimal place.

In all but three years, the partisan bias in the polls was small, with the polling average coming within 1.5 percentage points of the actual result. (I use the term “bias” in a statistical sense, meaning simply that the results tended to miss toward one direction.)

Nate Silver also looked at state polls which showed similar traits. His conclusion over all is:

On the whole, it is reasonably impressive how unbiased the polls have been. In both presidential and Senate races, the bias has been less than a full percentage point over the long run, and it has run in opposite directions.

That does not mean the pollsters will necessarily get this particular election right. Years like 1980 suggest that there are sometimes errors in the polls that are much larger than can be explained through sampling error alone. The probability estimates you see attached to the FiveThirtyEight forecasts are based on how the polls have performed historically in practice, and not how well they claim to do in theory.

But if there is such an error, the historical evidence suggests that it is about equally likely to run in either direction.

Nor is there any suggestion that polls have become more biased toward Democratic candidates over time. Out of the past seven election cycles, the polls had a very slight Republican bias in 2010, and a more noticeable Republican bias in 1998, 2000 and 2006.

They had a Democratic bias only in 2004, and it was very modest.

Still, 2004 went to show that accusations of skewed polling are often rooted in wishful thinking.

I can almost taste my lunch now.

What is his path to victory?

Mitt Romney is in serious trouble. He has pretty much lost Ohio with Obama maintaining a lead that is outside the margin of error.

Obama’s convention bounce barely subsided, and in fact is recovering again. In Florida Obama is also polling consistently higher than Romney. So what is the path to victory and the 270 electoral college votes he needs?

Nate Silver asks that precise question and notes that his path is difficult, if not impossible.

So one sign that Mr. Romney’s team is preparing a “Plan B” to win the election without Ohio would be if they begin to place more emphasis on Iowa and Nevada. They would then have to hope that a shift in the national environment would carry states like Virginia and Florida back into their column.

It isn’t a great plan. But when you’re the Republican candidate and are down outside the margin of error in Ohio with six weeks to go, you don’t have any great plans.

Mr. Obama’s probability of winning the Electoral College rose to 79.7 percent on Tuesday in the FiveThirtyEight forecast, up from 77.7 percent on Monday.

Wisconsin has already flipped to the Obama side of the ledger…the numbers are thinning for Mitt Romney and his path to victory becomes ever more difficult. RealClearPolitics puts Obama electoral college count at 265, just 5 shy of victory. With just 82 toss ups remaining Mitt Romney needs to win 79 of those. Romney needs Florida with its 29 college votes, and that isn’t at all certain.

The math doesn’t work, especially if Florida moves from toss up to Obama…which is likely any day now based on the current polling where even Rasmussen is showing a lead to Obama. Without Florida with 29 electoral college votes Romney’s race is done.

Bounce for Obama, not for Romney

The evidence is mounting across all major pollsters plus with the poll of polls that RealClearPolitics runs and also from statistician Nate Silver that Romney did not get a poll bounce from his convention and that Obama has.

RealClearPolitics is showing all the polling and it is pretty much a sea of blue for the gains that Obama has made. The RCP Poll average Romney vs Obama is telling:

Nate Silver has been copping a bit of flack lately, as have I for publishing his analysis. People think he has loaded his model and that he should wait for more data before calling things. Nate explains why that isn’t going to work:

I’ve been picking up some sentiment from analysts and journalists in my Twitter feed recently, who correctly note that polling around the party conventions can be volatile. They suggest that we ought to wait for more data before concluding very much about the bounces that the conventions have produced.

I’d love to have more data. I’d love it if we had a dozen national tracking polls rather than four. I’d love it if we had a pollster who was spending tens of thousands of dollars to poll every single swing state every single day.

It’s also the case that we’ll know more about the state of the race in two weeks than we know now — and we’ll know more about in four weeks than we do in two.

But we publish our forecasts every day. The goal is to make what we hope is the most accurate possible forecast given the information available at that time.

Saying “wait for more data” sort of misses the point. What about the data that we have on hand already? Is it compelling enough to suggest that there has been some change in condition of the race? Or isn’t it?

I follow Nate because he is simply looking at the numbers…all of them…all the time. It isn;t about the politics it is just about the numbers. In some ways he is a freak…but it is just about the math for him. And that math isn;t good for Mitt Romney:

He has Obama with a 79.8% chance of winning and his Electoral College vote is resounding:

The popular vote appears close, but people forget that it is all about the Electoral College.

Nate Silver makes this observation:

Nevertheless, the polling movement that we have seen over the past three days represents the most substantial shift that we’ve seen in the race all year, with the polls moving toward Mr. Obama since his convention.

How far will Mr. Obama’s numbers rise, and how long will his bounce last? We don’t know that, of course. But the range of possible outcomes reads pretty favorably for him.

As I wrote on Saturday night, Mr. Obama’s polls could easily cool off quickly. If we return to the equilibrium where Mr. Obama is about two points ahead in the polls — about where they were for months on end heading into the conventions — then Mitt Romney’s position won’t be too badly damaged. Still, Mr. Romney will be the underdog, and he’ll have had two or three weeks of time run off the clock.

Or, if Mr. Obama’s bounce is more prolonged and more pronounced, Mr. Romney could be in pretty bad shape.

Our goal is not to err on the side of caution any more (or any less) than we think is warranted by the data. There is absolutely, positively the chance that we will jump too far ahead of the trend. I can all but guarantee you, in fact, that if we run these forecasts for enough years, there will be a few instances in which the model makes an aggressive move — and is totally and completely wrong about it.

Of course it ain’t over till it is over, but right now it is advantage Obama.