More on Dodgy Polls, Ctd

The other day I blogged about the Conservative Party and Colin Craig’s dodgy polls that the media and he are touting predicting victory in Rodney and suggesting that Epsom was in play for anyone but John Banks.

I called into question the ethics and veracity of the poll for various reasons. On the matter of ethics I noted that the polling company was Research First and coincidentally their Managing Director, Roger Larkins, was a candidate for the Conservative party. I also called into question the veracity of the poll because no one knew and the neither the Conservatives nor Research First had published the questions that were used.

Roger Larkins took exception to my post and emailed me:

A colleague has drawn my attention to your entry about my connection to the Conservative Party, and they way you have linked this to the work we do at Research First. The suggestion that my personal political beliefs would somehow colour the quality of the work my company produces is an offensive one, and one I reject whole heartedly. I am happy to talk to you about the many steps taken in that particular piece of research to ensure robust results were achieved (and I welcome, in general, your critique of ‘dodgy’ and unscientific polls). In this case, however, your ad homenim dismissal of Research First’s work is misplaced and I would appreciate you publishing a retraction. Research First prides itself on professional and robust research, and in no way was the research conducted for the Conservative Party conflicted by my personal interest in the Conservative Party.

I replied to Mr Larkins that I stood by my claims especially about the ethics of himself, Research First and the Conservative Party in not declaring what I consider to be a clear conflict of interest.

On the matter of methodology I still could comment because to date neither Mr Larkins nor the Conservative party have published the questions asked. This is standard polling practice, the fact that they still hahd failed to do so just raises, properly, allegations of impropriety with the methodology.

I informed him that until the methodology used, and the questions asked is imparted to me I failed to see what I had to retract.

Mr Larkins responded:

I would like to clarify a couple of points with regard to the issue you have raised:

Firstly, timing.  Research First has been conducting research for and on behalf of Colin Craig since before the Auckland mayoral election of 2010. During that time, we have completed several projects for Mr Craig. The most recent project (being the research in question) was undertaken during the period September 3 – 11, 2011. This research happened BEFORE I was asked to stand as a candidate for the party (which was on Friday, September 30).

Secondly, specificity. Research First undertakes research on behalf of many clients. These have included in the past year two other political parties, with approaches to tender for two others. In other words, we do not solely provide research services for one political organisation.  As a professional research company, we have designed survey questionnaires to meet each client’s needs, and reported the findings in an appropriate manner.

Finally, design. Research First is committed to high quality surveys. All our telephone surveys are completed using randomised databases of contact numbers, with multiple call-backs, helping to ensure the resulting sample is reflective of the population being surveyed. In the case of the Rodney/ Epsom survey, the total number of survey completions was 305 completed surveys in Epsom (providing a maximum margin of error of +/-5.6% for a 50% sample at the 95% confidence interval); and 535 completed surveys in Rodney (providing a maximum margin of error of +/-4.2%). As such the Epsom data approached statistical validity for the response rate for given candidates, and the Rodney data can be considered statistically valid.

As noted previously, Research First has deep concerns for ‘dodgy’ surveying, and works constantly to ensure standards for research are maintained at the highest possible level.

These survey sizes are laughable, especially for the Rodney data where Colin Craig and the Conservatives have shopped the results to all and sundry to suggest that Colin Craig can beat Mark Mitchell. Mr Parkins still had not provided the scripts. I asked again for the scripts so that the stated results and claims could be compared against the framed questions.

This was the reply:

I’m concerned that even based on my response you will still stand by ‘everything’ you have said. Your concerns were that I had a conflict of interest between my professional role as a director of Research First and my personal interest in standing for election as a candidate for the Conservative Party.  I would have anticipated that on balance, the timing of being asked to be a candidate, the fact that Research First has conducted polling for several political parties and the fact that the research design was statistically robust would have demonstrated that our organisation is a bona fide market research company with ethical structures and robust designs, and that as a result, the perception of a conflict of interest was erroneous.

Bearing that in mind, to remove all doubt regarding the question structure, I can disclose the following:

Questions included both unprompted and prompted aspects, and sought to differentiate the responses regarding party voting intentions from candidate voting intentions.  Relevant questions  included the following, in order of being asked:

1. For your party vote, have you decided who you will be voting for in the election?

2. Which party do you currently intend to vote for?

3. Have you heard of the Conservative Party?

4. Have you heard of Colin Craig?

Participants were read a brief preamble to provide context: ‘Colin Craig is the leader of the Conservative Party of New Zealand. In the Auckland mayoral election, Colin came third with over 40,000 votes’. Then asked…

5. If Colin were to stand in as a candidate in your electorate, what would be the likelihood that you would vote for Colin to be your member of parliament (on a scale of 1 = definitely; 2 = likely; 3 = neutral; 4 = unlikely and 5 = very unlikely)?

Those who identified they were neutral or unlikely to vote for Craig were asked:

6 . Who do you intend to vote for?

There were additional questions relating specifically to how awareness was generated, as well as detailed policy questions. I’ve been advised these questions are relevant to the Conservative Party’s campaign strategy, and so details can’t be released.

To say the least I was gobsmacked with this response. Not only did it confirm the conflict of interest but it exposed the highly suspect methodology used by Research First in obtaining what they euphemistically describe as a “result”.

This is nothing short of a rigged poll. They lead respondents down a garden path and even prompted them as to what the required response should have been with poor questions. The poll surely is statistically invalid, in point of fact it is complete rubbish.

I stand by everything I have said about Research First, Roger Parkins especially in terms of ethics and reliability. The answers from Roger Parkins speak for themselves.

I believe that these polls, the results and the actions of the Conservative party and Colin Craig are nothing short of a attempt to manipulate an electoral result using a flawed and statistically irrelevant set of results that were contrived and manipulated to the point of being meaningless.

If Mr Parkins believes that his polls are correct then he should wager $1000 with me on his poll not being accruate and Colin not winning Rodney. Alternatively he could make a killing on iPredict which has Mark Mitchell and National to win at 95.1%.

  • Godzone

    One can only say that it appears as if National are slightly uneasy at the Conservatives apparent upsurge in support. Of course real values are like salt to a National witch…..Rodney has an educated and quite conservative population base. It is also the birthplace of what is left of the National Party that I grew up supporting….Mr. Craig’s family is well known in that community, in an electorate that has been shunned and ignored by Clockwork Smith for far too long. 
    As a member of a family that helped create the National Party and one that has lived in Rodney for a 100 years, I will now do all that I can to see that Colin Craig gets his opportunity, one that he deserves and will never take for granted. 

    Questioning the moral integrity of a well known and respected Polling agency and its founder, an actual employer, is just typical of a National’s desperation to cling to past glories…now long gone..

    • http://www.whaleoil.co.nz Whaleoil

      Glad to see that you have as little integrity as research first polls. You won’t be missed.

  • Pingback: That poll which shows Colin Craig ahead in Rodney, revealed | Kiwiblog()

  • Andrew McMillan

    I think this is called the best of both worlds; Conservative Party campaigning while doing a wee bit of “scientific” polling on the side.

    Good meaty story, and I see DPF’s covering it. You guys should really share an office :P

  • Warren Pohatu (Maoriboy)

    Obviously hit a nerve… hmmm… perhaps Mana should start doing their own polls too…. I conduct unscientific polls myself… I ask my 2 mates what they reckon and then I tally up the results… a bit worrying 57% Maori party… 14% Mana… and 15% unsure… try and figure that out… there are only 3 of us… (I admit I just make up the figures cos um useless at maths)

  • Apolonia

    What about the fact that the polling done in Epsom produced the same results as that done by other polling agencies.

  • jonno1

    I must have done the Epsom one, at the time when there was speculation that Colin Craig would run in Epsom.

    I agree the questions were directive, but so are other polls I’ve done. There was only one semi-open-ended question which asked you to rank your top three Conservative policies, so not exactly unprompted. I asked the pollster why Conservative opposed partial asset sales as this seemed inconsistent with their other conservative policies, but was told there was nowhere to record this comment/question.

    I also told them Colin Craig would be mad to stand in Epsom and split the right-wing vote still further – maybe that comment got through. :)

    In short, no better or worse than a Telecom or Vodafone poll that also tries to push you a certain way and doesn’t allow criticism. If I were running a poll for a business or a political party I would most want to know what I was doing wrong.

  • NZGroover

    Another one who forgot you’re a detail man…….and the devil’s always in the detail.

  • Jordan Thoms

    So they told them about a candidate, then asked them who they would vote for? I’m not a statistician but that’s so clearly wrong. If you wanted valid results, wouldn’t you ask them BEFORE you told them about CC?

  • Sting16

    Well research is research. You do not like Collin Craig? Give the Conservative party a chance.

104%