Poll Wars Episode 2: Attack of the Clowns

In the lead up to the last election, a period which all seems such a long time ago – there were a number of opinion writers at The Oz whose polling commentary really was in a galaxy far, far away.

The political blogosphere reacted to such arrant nonsense being churned out on a weekly basis by giving the guilty a good clip ‘round the ears and an education in basic statistics. The fallout provided much mirth, not only to the blogospherical participants and readers, but among the journalistic peers of the guilty, lobbyists, parliamentarians, their staff, academics and the broader political insider set generally – the fallout from this little episode culminated in The Oz having a notorious hissyfit in an editorial about how they own Newspoll so the rest of us should just STFU.

It was a memorable, if unedifying spectacle.

This skirmish became known as the Poll Wars, where the first of our contributions here was titled Poll Wars Episode 1:The Phantom Metrics.

A little while ago over in The Oz, our favourite stenographer Dennis Shanahan revived the franchise with a B-grade sequel – a little wet lettuce bitchslap to the bloggers generally, but to this Possum particularly, over Nightwatchman’s single digit preferred PM rating in Newspoll. For those that haven’t yet read it (or have understandably forgotten about it), it’s probably a good idea to go and give it a bit of a squiz. I suggest it works best accompanied with its official soundtrack:

Dennis gets a little frisky over a number of issues in his page 2 column, the first being his opening statement “Where have all the poll pundits and, more particularly, the poll pedants gone?“, suggesting that we just aren’t paying enough attention to him anymore.

While it’s flattering, if a little disturbing, to know that Dennis seeks our eternal attention – I would never dare speak for the other pseph bloggers, but my reasons for not giving three fifths of five eighths of sweet FA’s worth of attention to whatever it is that Dennis is writing these days is pretty simple – the election is over. His type of agenda pollution of the mainstream media coverage of the most prominent and important poll in the country simply no longer has any consequences of any relevance to anyone but Dennis Shanahan.

It was certainly important at the time, but the time for some things eventually passes with history, much like credibility.

After this little piece of introductory attention whoring, the next bit of nonsense worth addressing, and the primary reason why any of it is even on the horizon of this place, is his following quip:

“After The Australian put the story, which I wrote, on the front page, it captured public attention and was reported, commented upon and retold in newspapers, radio, television and blogs.

As Possum Comitatus said on Crikey: “While this latest voting intention is hardly newsworthy, even to the poor poll junkies among us, it is the preferred prime minister figure that really grabs the attention.”

Ol’ Possum went on: “While records are meant to be broken, this one was obviously meant to be smashed. Brendan Nelson has stormed into the worst preferred prime minister result in the history of Newspoll with an astonishing 9 per cent.”

So far so good – but it’s at this point where Dennis Shanahan goes troppo, demonstrating that a very little knowledge is indeed a very dangerous thing. He states:

“Yet there was one key point missing from all the commentary that has previously cropped up in analysis of Newspolls: in Possum’s words, Nelson “stormed” to his rating by 2 percentage points.

Rudd’s record on preferred PM was also reached by a rise of 2 percentage points.

The margin of error for the Newspoll survey on a sample of 1140 is 3 percentage points. The leaders “stormed” to these records with movements of less than the margin of error.“

According to Dennis, what Nelsons Preferred PM rating was in the previous poll is somehow important when comparing the lowest individual scores of Opposition Leaders across a 20 year period. If he believes this to be the case, it really just reinforces why no one seems to pay a great lot of attention to him these days in the blogosphere.

Nelson didn’t “storm” into his dubious distinction of having the lowest ever preferred PM rating in Newspoll history from the last poll, he stormed into this dubious position by simply beating Simon Creans lowest score. He actually did it last month on 11%, but this months 9% just adds another increment to the historical minimum, and single digit figures are really quite a shocker.

This brings us to the most amusing thing about the article – the actual nature of his spiel itself. A well constructed strawman generally requires you to misrepresent the actual nature of an opponents argument into something slightly different, so that you can then proceed to not only burn it down with the fires of rhetoric, but where you can kick the seven shades of shit out of the smouldering embers with a barrage of well organised facts.

While Dennis might do misrepresentation well, his follow up left a bit to be desired.

You might notice in his article that Dennis waves around his “Margin of Error” like a drunken pirate with an acute middle ear infection might wave around a particularly large and cumbersome sabre.

The margin of error as reported from our largest pollsters is predicated on there being an approximate 50/50 split in the poll as the true value. This derives from the way the Standard Error of the poll is calculated via the formula:

Standard Error =

 

Where p is the population proportion we are trying to measure (like the proportion of the population that will vote for the Coalition, or the proportion of the population that prefer Nightwatchman as preferred PM) and n is the number of survey respondents.

In practical terms this means that when polling companies state that their Margin of Error is 3% for polls that measure the Two Party Preferred vote – it’s pretty close to being true as the TPP is usually between 50/50 and 60/40 thereabouts. Hence the value of p (the proportion of people that say they would vote for a given party) for the two party preferred vote is roughly 0.5, or 50%

But when a poll produces results that are way outside of that 50/50 split – like the 9% rating for Brendan Nelson, the statistical value of the margin of error actually reduces the farther away the reported result is from that 50/50 split, and reduces substantially.

By how much?

Well for Nelsons preferred PM result of 9%, we can be generous and say that p = 0.1 (the proportion of the population that prefer Nelson as PM vs. the 0.9 proportion that in some way do not), substitute that value into the equation along with the number of survey respondents in that particular Newspoll (n= 1140), and end up with a Standard Error of 0.889

To change a standard error into a Margin of Error for this poll, we need to build a thing called a 95% confidence interval – the 95% value being what all the pollsters use (which basically says that we can be 95% sure that the true result is within our margin of error). To do this is very easy – we simply multiply the Standard Error of 0.889 by the critical value 1.96 (which is the critical value associated with 95% confidence intervals for these things), multiply the whole thing by 100 to turn it into a percentage and Ta Da!

We have a Margin of Error on the Preferred PM poll of not 3% as Dennis states (because he’s just a goose mindlessly quoting Newspoll figures that he doesn’t actually understand), but a Margin of Error of 1.742%

We could be generously conservative to both Brendan and Dennis here and round it up to a nice 1.8%.

So when Dennis takes the flamethrower to that strawman of his where he invents me talking about Nelson storming from 11% to 9% and says “The leaders “stormed” to these records with movements of less than the margin of error.“, he cant even get the basics right.

If I were saying such a thing, which I’m not and I certainly wasn’t in that article he’s quoting from, in an episode of extreme irony it would still be correct because the exact opposite actually occurred – it was the MoE on that poll result which was actually LESS than the movement that occurred.

Not content with just getting the basics wrong in his piece of self-serving piffle, Dennis also found the need to pontificate about those of us in the blogosphere that actually do have an understanding of the polling statistics that so often confuse Mr Shanahan :

“Statistical bloggers forever complain about reports of movements of less than 3per cent and essentially want polls to be banished from newspapers and public debate except during an election.”

“Essentially want polls to be banished” eh?

Put it back in your pants Dennis.

I’ve heard some pretty idiotic things in my life, many from the columns of Mr Shanahan himself – but this one is a cracker.

No Dennis, we don’t want polls banished from public debate – far from it, we just want polling to be reported accurately, a feat which the overwhelming majority of the political journos in this country manage easily and with a high level of professionalism. Some, it would seem, just need excuses instead.