More on Dodgy Polls, Ctd

The other day I blogged about the Conservative Party and Colin Craig’s dodgy polls that the media and he are touting predicting victory in Rodney and suggesting that Epsom was in play for anyone but John Banks.

I called into question the ethics and veracity of the poll for various reasons. On the matter of ethics I noted that?the?polling company was Research First and coincidentally their Managing Director, Roger Larkins, was a candidate for the Conservative party. I also called into question the veracity of the poll because no one knew and the neither the Conservatives nor Research First had published the questions that were used.

Roger Larkins took exception to my post and emailed me:

A colleague has drawn my attention to your entry about my connection to the Conservative Party, and they way you have linked this to the work we do at Research First. The suggestion that my personal political beliefs would somehow colour the quality of the work my company produces is an offensive one, and one I reject whole heartedly. I am happy to talk to you about the many steps taken in that particular piece of research to ensure robust results were achieved (and I welcome, in general, your critique of ?dodgy? and unscientific polls). In this case, however, your?ad homenim?dismissal of Research First?s work is misplaced and I would appreciate you publishing a retraction.?Research First prides itself on professional and robust research, and in no way was the research conducted for the Conservative Party conflicted by my personal interest in the Conservative Party.

I replied to Mr Larkins that I stood by my claims especially about the ethics of himself, Research First and the Conservative Party in not?declaring?what I consider to be a clear conflict of interest.

On the matter of methodology I still could comment because to date neither Mr Larkins nor the Conservative party have published the questions asked. This is standard polling practice, the fact that they still hahd failed to do so just raises, properly, allegations of impropriety with the methodology.

I informed him that until the methodology used, and the questions asked is imparted to me I failed to see what I had to retract.

Mr Larkins responded:

I would like to clarify a couple of points with regard to the issue you have raised:

Firstly, timing.? Research First has been conducting research for and on behalf of Colin Craig since before the Auckland mayoral election of 2010. During that time, we have completed several projects for Mr Craig. The most recent project (being the research in question) was undertaken during the period September 3 ? 11, 2011. This research happened BEFORE I was asked to stand as a candidate for the party (which was on Friday, September 30).

Secondly, specificity. Research First undertakes research on behalf of many clients. These have included in the past year two other political parties, with approaches to tender for two others. In other words, we do not solely provide research services for one political organisation. ?As a professional research company, we have designed survey questionnaires to meet each client?s needs, and reported the findings in an appropriate manner.

Finally, design. Research First is committed to high quality surveys. All our telephone surveys are completed using randomised databases of contact numbers, with multiple call-backs, helping to ensure the resulting sample is reflective of the population being surveyed. In the case of the Rodney/ Epsom survey, the total number of survey completions was 305 completed surveys in Epsom (providing a maximum margin of error of +/-5.6% for a 50% sample at the 95% confidence interval); and 535 completed surveys in Rodney (providing a maximum margin of error of +/-4.2%). As such the Epsom data approached statistical validity for the response rate for given candidates, and the Rodney data can be considered statistically valid.

As noted previously, Research First has deep concerns for ?dodgy? surveying, and works constantly to ensure standards for research are maintained at the highest possible level.

These survey sizes are laughable, especially for the Rodney data where Colin Craig and the Conservatives have shopped the results to all and sundry to suggest that Colin Craig can beat Mark Mitchell. Mr Parkins still had not provided the scripts. I asked again for the scripts so that the stated results and claims could be compared?against?the framed questions.

This was the reply:

I?m concerned that even based on my response you will still stand by ?everything? you have said. Your concerns were that I had a conflict of interest between my professional role as a director of Research First and my personal interest in standing for election as a candidate for the Conservative Party.? I would have anticipated that on balance, the timing of being asked to be a candidate, the fact that Research First has conducted polling for several political parties and the fact that the research design was statistically robust would have demonstrated that our organisation is a?bona fide?market research company with ethical structures and robust designs, and that as a result, the perception of a conflict of interest was erroneous.

Bearing that in mind, to remove all doubt regarding the question structure, I can disclose the following:

Questions included both unprompted and prompted aspects, and sought to differentiate the responses regarding party voting intentions from candidate voting intentions.? Relevant questions ?included the following, in order of being asked:

1. For your party vote, have you decided who you will be voting for in the election?

2. Which party do you currently intend to vote for?

3. Have you heard of the Conservative Party?

4. Have you heard of Colin Craig?

Participants were read a brief preamble to provide context: ?Colin Craig is the leader of the Conservative Party of New Zealand. In the Auckland mayoral election, Colin came third with over 40,000 votes?. Then asked?

5. If Colin were to stand in as a candidate in your electorate, what would be the likelihood that you would vote for Colin to be your member of parliament (on a scale of 1 = definitely; 2 = likely; 3 = neutral; 4 = unlikely and 5 = very unlikely)?

Those who identified they were neutral or unlikely to vote for Craig were asked:

6?. Who do you intend to vote for?

There were additional questions relating specifically to how awareness was generated, as well as detailed policy questions. I?ve been advised these questions are relevant to the Conservative Party?s campaign strategy, and so details can?t be released.

To say the least I was gobsmacked with this response. Not only did it confirm the conflict of interest but it exposed the highly suspect methodology used by Research First in obtaining what they?euphemistically?describe as a “result”.

This is nothing short of a rigged poll. They lead respondents down a garden path and even prompted them as to what the required response should have been with poor questions. The poll surely is statistically invalid, in point of fact it is complete rubbish.

I stand by everything I have said about Research First, Roger Parkins especially in terms of ethics and reliability. The answers from Roger Parkins speak for themselves.

I believe that these polls, the results and the actions of the Conservative party and Colin Craig are nothing short of a attempt to manipulate an electoral result using a flawed and statistically irrelevant set of results that were contrived and manipulated to the point of being meaningless.

If Mr Parkins believes that his polls are correct then he should wager $1000 with me on his poll not being accruate and Colin not winning Rodney. Alternatively he could make a killing on iPredict which has Mark Mitchell and National to win at 95.1%.