The Accuracy of Horizon polls, Ctd

Yesterday I blogged about how the minor parties could easily manipulate the Horizon poll to suit their aims. Now Andrew Balemi, who is a Professional Teaching Fellow in the Department of Statistics at The University of Auckland and a former Head of Marketing Science at market research company Colmar Brunton has joined the debate on the Stats Chat Blog:

It claims that it’s to be trusted as it has sampled from a large population of potential panellists who have been recruited and can win prizes for participation. The Horizon poll adjusts for any biases by reweighting the sample so that it’s more like the underlying New Zealand adult population – which is good practice in general.

However, the trouble is this large sampling frame of potential panellists have been self-selected. So who do they represent?

To illustrate, it’s hard to imagine people from more affluent areas feeling the need to get rewards for being on a panel. Also, you enrol via the internet and clearly this is biased towards IT-savvy people. Here the sampling frame is biased, with little or no known way to adjust for any biases bought about from this self-selection problem. They may be weighted to look like the population but they may be fundamentally different in their political outlook.

Panel polls are being increasingly used by market researchers and polling companies. With online panel polls it’s easier to obtain samples, collect information and transfer it, without all the bother involved in traditional polling techniques like CATI.

I believe the industry has been seduced by these features at the expense of representativeness – the bedrock of all inference. Until such time as we can ensure representativeness, I remain sceptical about any claims from panel polls.

I believe the much-maligned telephone (CATI) interviewing, which is by no means perfect, still remains the best of a bad lot.

Do you want:

  • ad-free access?
  • access to our very popular daily crossword?
  • access to Incite Politics magazine articles?

Silver subscriptions and above go in the draw to win a $500 prize to be drawn at the end of March.

Not yet one of our awesome subscribers? Click Here and join us.

As much at home writing editorials as being the subject of them, Cam has won awards, including the Canon Media Award for his work on the Len Brown/Bevan Chuang story.  And when he’s not creating the news, he tends to be in it, with protagonists using the courts, media and social media to deliver financial as well as death threats.

They say that news is something that someone, somewhere, wants kept quiet.   Cam Slater doesn’t do quiet, and as a result he is a polarising, controversial but highly effective journalist that takes no prisoners.

He is fearless in his pursuit of a story.

Love him or loathe him.  But you can’t ignore him.