Newspoll Tuesday – Graphdump edition and new Pollytrack methodology

Another Tuesday, another Newspoll of psephy goodness via The Oz, this time showing the ALP down 2 on the primary and the Coalition steady for a 45/38 split -washing into the two party preferred for an ALP lead of 56/44, down from the 57/43 a fortnight ago.

Today we unveil a new Pollytrack methodology to plug these numbers into.

Previously we were using just phone polls, but over the last few weeks I’ve been experimenting around with different polling aggregations and comparing them to election results to find a more accurate way to do it (and a big thanks goes out to a few folk that gave me access to unpublished polling data to help with the calibration of this thing).

Instead of just using phone polls, we are going to do the obvious and use every piece of polling data we can find to give us rolling super samples calculated weekly. Each Pollytrack calculation is made up of a rolling all pollster average weighted by sample size, that operates over a 5 week maximum window, and where each pollster has only their latest poll contributing to the super sample at any given time.

As a pollster releases a new poll, their old poll result drops out of the rolling window, and gets replaced by their new polling data.

The 5 week window works best as ACNielsen (a monthly pollster) occasionally releases polls 5 weeks apart rather than the usual 4, simply because of timing issues. Using a 5 week window allows us to always have an ACN poll in the Pollytrack mix.

The current Pollytrack mix is made up of Newspoll, ACNielsen, Morgan face-to-face and Essential Research (EMC). Morgan phone polls will also be added when they pop up, but the last phone poll from Morgan was taken over 5 weeks ago, so it drops out of our Pollytrack window.

The beauty of this is that it gives us super samples. Our current sample is 6039, but it has been as high as 7450 in early July, while the lowest it’s been all year was 4158 in early May. That gives us minimum MoEs on this baby of between 1.1 and 1.5%.

On the primary and TPP Pollytrack charts, the minimum MoEs have also been added to the chart itself, giving us a much better handle on the current state of play that includes pure sampling error uncertainty.

Currently, Pollytrack has the primary vote on 45.7/37.2 to the ALP, with the two party preferred running at 56.6/43.4 to Labor. The minimum MoE for this week’s Pollytrack is plus or minus 1.3%.

That now makes Pollytrack by far and away the most accurate tracking poll in the country bar none. Nothing else comes within a bull’s roar of this baby.

(As always, just click the charts to blow them up)

The estimated seat changes are calculated using an assumption of a uniform swing on the national pendulum.

Essential Media have also been added to our Loess regression charts, which now give us:

Currently our local regression series has the ALP leading 45.6/36.4 on the primary vote for a TPP of 57.0/43.0.

Next up, we’ll take a squiz at the Newspoll qualitative metrics.

Finally a squiz at how all the pollsters have been traveling, including the new EMC polls we’ve added to the mix. There were a few numbers out of place in this series previously for the Morgan polls – I had a bit of a brainfart, but that’s all been fixed up.

Things might get a little bit all over the shop round these parts for the next few weeks as I prepare for the site move, so I might run a day or so late on stuff.

But good psephing to all, I’ve got to scurry off and do the latest US Intrade data that should have been completed yesterday.

Elsewhere: Pollbludger and Larvatus Prodeo

Advertisement

Major Announcement

This place will be ending and the whole thing moved across to a new home, Pollytics.com

I’ll have more tech capabilities, there’ll be bucketloads more content, it will be funded by advertising and I’ll even be decloaking for the process.

If we’re going to get serious, we may as well do it properly.

So folks, are there any new things you’d like to see, because just about anything will be possible?

Do State governments suffer with Federal Governments of the same party?

It’s probably an unanswerable question – sometimes they would seem to, other times not. But the difficulty with measuring it statistically is that you need to go so far back into the past with the data (simply to get enough observations) that you run into the problem of the society you are measuring back yonder being vastly different to the one we have today.

So instead, some food for thought – we’ll measure the changes that have occurred in State polling metrics over 2008, looking at the Primary Vote, Better Premier and Net Satisfaction Ratings.

A lot of these are undoubtedly down to local State government politics, but some may not be. Also note that for the Net Satisfaction value for WA, we couldn’t use the Newspoll released on Monday as it didn’t have any Opposition satisfaction ratings published in it (presumably because of Chairman Sniff standing down from the Opposition Leadership) – so instead we’ve used the April-June quarter reading for that WA entry. Also note that a lot of these results are less than the MoE on the polls – so take low numbers with a grain of salt, but it’s all food for thought anyway.

Victorian Newspoll – Eroding Satisfaction.

Hot on the heels of the NSW bimonthly Newspoll comes another of the Victorian variety – it’s in the treeware version of The Oz, but not yet online.

The Victorian polling since the election is a textbook case of how public political opinion usually behaves over the political cycle (as opposed to yesterdays NSW example). After the last election there was a spike for 12 months or so before slowly washing out of the system as the grind of day to day politics slowly eroded the governments standing.

There’s a couple of interesting things to note here, the first being the long term erosion of the Better Premier rating for the government over the last 5 years. The second is the deterioration in Net Satisfaction levels of both the Premier and Leader of the Opposition since the last election. If that trend of growing dissatisfaction with the political system were to continue, two things would be expected to happen if the Victorian experience plays out like it does nearly everywhere else in the polling; firstly, the minor party vote grows for fairly obvious reasons, but secondly, the political environment tends to change where politics becomes both more vicious and more superficial (think NSW over the last few years, or QLD 6 months before the last election).

It’s a strange relationship, but as the public becomes more cynical of the major parties, so too do the major parties seem to become more cynical in the way they treat the public. I’m sure Old Media has a large role in how that plays out, more easily justifying the cheap, sensationalist route they seem to pursue when the public starts becoming disenchanted with the political system as a whole.

Anyway, that’s enough of my pontificating (it’s starting to resemble a Gary Says moment from the Morgan polls ) – here’s the charts, just click on them to expand.

NSW Newspoll – Constipation more popular than the ALP

Newspoll has released their bi-monthly NSW poll via The Oz – worth having a squiz at.

Pick a chart, any chart – from voting intention, better premier, satisfaction ratings, net satisfaction ratings – any chart at all really and the story is the same; a government that stayed one election too long. The long term trends are about as awful as they get and it looks like the ALP TPP vote is just about to collapse.

What’s interesting here is that an election win is usually a medium term circuit breaker – after an incumbent retains government, the polls usually spike for 12 months or so before slowly eroding over the next few years as the political grind catches up with the government. But with the last NSW election, any spike Iemma may have achieved had not only washed out of the system within 9 months, but all of the metrics bar the Two Party Preferred vote estimates had collapsed.

Unless the NSW government defies 20 years of national polling history – an extremely unlikely event – the TPP will follow the qualitative metrics long term trend down, for it nearly always does.

If you ever wanted to see what the metrics of a textbook case of an untenable government leader looks like, I give to you Exhibit A – just click on them to expand.

The last one is the killer. It shows the relationship between Net Satisfaction and Net Satisfaction calculated with the undecideds removed. The pink line crossed the red line while the undecideds simultaneously decreased in number – that spells bad news.

The undecideds have broken against Iemma – they wont be coming back.

To give you another example of how this relationship has played out, the same thing happened to Nelson federally:

It’s why he’s untenable as well. So the question is whether NSW ALP has the plums to move against a Premier that can only be elected if the NSW Leader of the Opposition is, as the yank saying goes,  found with a dead girl or a live boy.

But even then, she’d be an even bet.

The other question is whether this is driven by public perceptions of ineffective leadership or by a broader public perception of an ineffective NSW Labor government as a whole (or worst of all, both). If it’s the former, there’s room for improvement with a new leader, if it’s the latter then the next few years will be a pathetic spectacle of deckchair rearrangement.

UPDATE:

Worth mentioning as well is the misleading nature of the two party preferred vote here. Because NSW has optional preferential voting, preference flows wont be as certain as we would expect them to be, say, Federally. The other problem is that with the ALP primary in the low 30’s, there’s a chance of the Greens outpolling the ALP in a few inner city seats as well as the possibility of Independents doing the same elsewhere – the very notion of TPP becomes a little shabby as a true indicator of political reality with such low ALP primary vote numbers.

With the ALP in the low 30s and the Coalition around 40 – that leaves just under 30% to be picked up by non-majors. But, as always, these things dont happen smoothly -the third party vote will be lumpy; low in some seats, but higher in others to make up the average, and probably high enough in some places to knock the ALP out of second spot at the moment, making the TPP a dubious metric.

ACNielsen and Pollytrack Expanded

The new ACN poll is out today with the ALP steady and Coalition down 1 in the primaries for a 43/39 split leading to the ALP up 1 for a 55/45 two party preferred headline number.

Plugging this into our Pollytrack series – the rolling 3 phone pollster average weighted by sample size – we get (just click on the charts to expand them):

Pollytrack currently has the primaries coming in at 45/39.2 to the ALP with the two party preferred running 55.9/44.1 the same way.

I’ve added two new charts to Pollytrack.The first is the Primary Vote Swing which shows how far the primary vote of each party has changed since the last election while the second shows the two party preferred swing since the last election as well as how many seats that would change according to a uniform swing on the national pendulum.

As always we have our all pollster loess regression series:

And finally, a look at how the ALP and Coalition have been traveling with all the pollsters this year.

The One Nation Effect – temporary or structural?

Continuing on from the other day where had a bit of a squiz at a chart of the Coalition primary vote and the havoc One Nation seemed to wreak on it, today we might have a closer look at how the One Nation effect has played out on both parties. We know that One Nation caused the Coalition enormous chunks of grief throughout their brief cameo appearance on the national political stage, to the point where the 1998 election result itself was determined by One Nation preferences rather than silly myths about well targeted marginal seat campaigns. If a handful of One Nation branches decided to preference the other way, or preference at all (or even consistently) in many cases, Australian politics could have had a rather different face over the last 10 years.

To start with, some basic facts would be handy; in the 1998 election, one Nation ran in 135 seats, picked up 8.4% of the national vote and their preferences flowed 53.7% to the Coalition and 46.3% to the ALP.

The tricky bit about identifying the One Nation effect is trying to nail the period of time it was operating while also controlling for any longer term trends that may have been running, as well as trying to pick out any permanent structural change that One Nation may have caused on the primary and two party preferred votes of the majors.

What follows isn’t a definitive look by any means, more of an exploration of the data so feel free to request any equations or relationships that you think might be pertinent here.

Firstly, there’s a big quadratic time trend running through the primary and TPP votes of the majors since Newspoll started back in 1985.

Why?

Dunno

Could be lots of reasons, demography being one of them. But we know it’s there.

So first we’ll run a series of basic regressions where we’ll regress the primary and TPP votes of the ALP and Coalition against a quadratic time trend and a One Nation dummy variable that starts in April 1997 (when the party was created) and continues through to the present. Essentially this dummy variable assumes that the One Nation effect was permanent on the vote levels of the ALP and Coalition. That’s not to say that this assumption is purely or even partially correct, just that it’s what we are assuming for now and we have to start somewhere. The equation for each of the vote types of the majors is given, followed by the visual represnetation. These series are all serially correlated up the wazoo so we can adjust the standard errors to accommodate, but removing the serial correlation itself is an impossible mugs game in survey time series because the underlying human behaviour is in large part driven by it. As usual, all the charts can be blown up by clicking on them.

Coalition Primary Vote:

Coalition Two Party Preferred Vote:

ALP Primary Vote:

ALP Two Party Preferred Vote:

From this, since One Nation was created (and after controlling for the quadratic time trend), the Coalition has lost an average of 6 points on their primary vote, but only 4.3 on their TPP vote. Yet, the ALP has gained 2.8 points on their primary and 4.3 on their TPP.

One possibility that fits well, is that the 2.8 points the ALP gained came directly (over the longer term)  from the LNP. That would leave the ALP needing 1.5 points of the TPP vote to pick up (The 4.3 they gained in TPP minus the 2.8 they gained on the primary). If we then look at the Coalition vote, they lost 6 points. If 2.8 of those went to the ALP, that leaves 3.2 going elsewhere (initially to One Nation, but to other parties since), of which 1.5 of those (46%) flowed back to the ALP in preferences. That would be consistent with the original1998 One Nation preference flows mentioned earlier, which may have anchored the broader pattern after 1998 as the One Nation vote dissipated elsewhere.

The other problem here though is that the One Nation effect may have only been temporary, and that any permanence of it may have simply washed into the larger prevailing quadratic time trend. For instance, if we limit the One Nation variable to operate only through the period of April 1997 when they were created, through to the 2001 election (after which they effectively ceased to be a force) and run the regressions again, we get:

Coalition Primary Vote:

Coalition Two Party Preferred Vote:

This looks visually interesting – with the Coalition losing 4.4 on the primary and 3.9 on the TPP. However, this specification of the One Nation effect throws up something a little weird for the ALP primary vote.

The TPP regression results for the ALP are the same (but opposite sign) to the Coalition (obviously) with an ALP gain of 3.9 on the TPP – but the primary vote ends up like this:

ALP Primary Vote

The ALP gained 4.4 points on their primary vote but only 3.9 on their TPP, suggesting that they were pulling primary votes off the Democrats and Greens without boosting their TPP vote.

The big question is trying to determine how much of the One Nation effect was structural (as in the first set of equations) vs. how much of the One Nation effect was temporary (the second set of equations)?

Finally, it was asked in comments of a recent post if the Coalition controlling the senate had an impact.

The Coalition took control in July 2005. It’s hard to tell if the decline started then, or at just after the 2004 election. You’ll notice a big drop just before January 2006 – Workchoices entered parliament in November 2005.

Newspoll Tuesday – ‘Where has the decent political Oz Rock gone’ edition

Another Tuesday, another Newspoll, another kick to The Nightwatchman.

The Nats are up 1 to 5%, bringing the Coalition up 1 to 38% on the primary vote, sitting behind Labor unchanged on 47%. The TPP remains unchanged from last poll on 57/43 to Labor.

Feeding it into our Pollytrack series – currently running a pooled sample of 3933 for a minimum MoE of 1.56% -we get:

Throwing last weeks polls into our larger local regression series, which has had this Newspoll as well as a new Morgan face to face added since last time, we get:

Currently our regression lines for the primary votes sit on 45.9/39.9 to the ALP, while our TPP lines sit on an ALP lead of 55.7/44.3.

Meanwhile, can someone tell me where all the decent political Oz Rock has gone? Maybe I’m just getting old and listening to the wrong things, but for mine I think its time the world brought back Spy vs Spy!

Polls – More than horserace commentary.

This was me in Crickey yesterday.

If you were to look at the betting markets for the NT election last week, you’d be forgiven for thinking that the ALP had a greater chance of being abducted by the latest outbreak of NT UFO’s than they had of being beaten by the CLP. Yet, with no major pollster running pre-election surveys in the Territory, should we be at all surprised that the markets got it so wrong in terms of the chance of Labor retaining government?

As much as political polling is scorned as reducing important political issues down to little more than horse race commentary, it fulfills one fundamentally important role – it stops people talking shit.

From politicians to columnists, from reporters to your average Joe – political polling encourages all but the learned types at The Australian to keep it in their pants.

With no major polls in the Territory election, information about the election itself was dominated by party propaganda on the one hand and political commentators staring deeply into their navels on the other – usually finding little more than lint as a result, but lint dressed up as profundity none the less. Without polling information, election campaign analysis becomes an exercise in either wishful thinking or what ought to happen — and as we’ve seen in the Territory, theories on what ought to happen were well formed, plentiful, but mostly wrong. There is no substitute for the type of empirical reality that only polling can provide.

It’s a pretty simple rule – you can’t really analyse what you don’t really know.

What makes betting markets valuable is their capacity to aggregate all available sources of information to predict a result, but without polling information anchoring the market to some semblance of reality, without that knowledge of what people are actually thinking on the ground, the betting markets were left drifting in the breeze, ostensibly being guided by lint powered column inches telling us that Labor was a shoe in because, well, that’s what ought to happen.

So should we really be surprised that without political polls running in the Territory campaign, the markets were so out of whack with the result? While the markets may have gotten the end result right, the magnitude of the victory will probably come down to a few hundred votes – hardly the landslide that was predicted, and certainly not justifying 1/14 odds that some markets were offering.

Good information makes good markets, and there is no better information than good, independent polling. It provides far more than fodder for horse race political commentary, it provides certainty and knowledge and evidence for observable reality. And at the end of the day, isn’t observable reality what all good political commentary should be about?

Elsewhere: Cross posted at Larvatus Prodeo

A Few Charts.

Just thought I’d whack up a few charts of things I’ve been looking at over two-party preferred margins lately.

First up, Newspoll TPP margins for every Newspoll going back to Novemember 1985. Newpsoll had a few periods where they didn’t calculate TPP results in their polling, so I’ve constructed a TPP series using the primary vote estimates and distributing preferences the further away from an election you get, the less the results get distributed according to that previous election and the more they get distributed on the basis of the results of the approaching election (click on these to blow them up).

It’s interesting how election results interrupt the longer running trends – sometimes it’s the dreaded narrowing, sometimes its the opposite.

Next up, if we measure the TPP vote of the winning party for every election back to 1949, as well as the  percentage of seats that the winning party achieved from that vote and chart them with a scatter plot and a trend line (quadratic trend) we get:

This gives us an alternative to the pendulum for projecting how many seats a given TPP vote would be expected to deliver to the victorious party. If we transform those percentage of seats won to actual seats in a 150 seat parliament (which we have today) we get:

Which is really just a bit of fun, although the cluster of results where a party won government by getting less than 50% of the vote makes things a bit more complicated here.

Finally, a little chart on the way that One Nation permanently reduced the Coalition primary vote. It’s again using Newspolls but I’ve also added a quadratic time trend through the results with a regression that has a One Nation dummy variable in it (which starts at the first Newspoll after the One Nation Party was created).

I hope you find it as visually interesting as I do in terms of the pronounced impact of One Nation on the Coalition vote.

UPDATE:

The US Election Intrade data has suffered its weekly update, showing Obama getting his second week of probability flight in a row.

All you need to know about the Lyne by-election

With acknowledgement to the Pollbludger of the same name

The yellow is the Federal electorate of Lyne, with the State electorates overlaid.

The soon to be independent member for Lyne, Rob Oakeshott, currently holds the State seat of Port Macquarie. Now by the word “hold” I don’t mean he scrapes in, I mean he HOLDS it, not only getting 67% of the primary vote, but winning every single booth on the primary vote as well – Rob Oakeshott doesnt know what a preference looks like.

So the Port Macquarie area of the seat, making up approximately half of the population of Lyne, is his -no ifs, no buts.

Yet it doesn’t stop there; Wauchope, once in the Port Macquarie electorate, has strongly supported Oakeshott before an earlier electoral redistribution at the State level and will again now at the Federal level, especially since the good denizens of Wauchope are rather pissed that the Mayor of the Port Macquarie Hastings City Council Rob Drew – now ironically the Nationals candidate for Lyne (oh dear) – proceeded to piss around $70 million of ratepayers hard earned up the wall on the Glasshouse development fiasco in Port, a piece of negligence for which the Council was subsequently sacked for incompetence much to the cheer of many Wauchopians. Mr Drew wont find much in the way of electoral solace in Timbertown.

One of the funny things about Lyne, and something I know about personally having grown up there and retaining large chunks of family in the electorate, is that there are large community ties between the Hastings Valley which makes up most of the Port Macquarie electorate by population, and the Manning Valley which provides the other large population centre of Lyne – Taree. The communities of Taree and Port Macquarie are politically similar, as are the smaller local satellite communities of Wingham and Wauchope.

With Port Macquarie and Wauchope supporting Oakeshott, Taree and Wingham will follow. We can already see how this will play out in Taree by looking at some of the booth results at the last State election. The coastal hamlet of Harrington, which is close to Taree and filled with people that work in Taree (yet has strangely been placed in the Port Macquarie electorate), gave Oakeshott 75% of the primary vote in the Harrington booth. I would be surprised if the Taree booths gave a result that strong, but with the ALP not running, a good chunk of their 30% and a bit vote they usually receive will flow to Oakeshott, and most Liberal oriented voters in the town (of which there are plenty – Vaile only beat a Liberal candidate in the 1993 election by a couple of hundred votes) will vote for Oakeshott – probably giving him around the 55% of the primary vote mark in the Taree booths, maybe a tad higher. Not as high as the 70%+ he will receive in some of the Hastings Valley booths with the ALP not running, but high enough to thump the seven shades of shit out of the Nats.

That’s three out of four of the largest population centres, leaving us Wingham – the place from where I actually originate. The burghers of Wingham are usually Nats voters, but went 25% for One Nation in 1998. I can’t see Wingham doing anything other than following their swamp donkey cousins in Taree and voting for Oakeshott as well, especially considering the 1998 predisposition to vote for a conservative that isn’t Lib/Nat.

The only real question over Lyne is the size of Oakeshott’s win. He wont go to preferences, but my money is on a primary vote of around 63%.

However, here’s a hot tip – the Nats will still poll well in the Comboyne booth, perhaps even coming close to Oakeshott on primaries. Considering the thumping they’re going to receive, 100 votes is nothing to scoff at

US Election Update – August Edition

Just a quick roundup of all the Intrade charts fit to print on the current state of the US Election according to the Intrade markets.

Below, in order, the current Electoral College Votes projected by Intrade for the Democrats, the headline “Democrat as President” market probability and the EV Probability Sum (which is the expected value of electoral college votes, where each state has its college votes multiplied by that States win probability and where they’re all summed up to give us the number)

States holding Democrat are:

CA NY IL PA OH MI NJ MA WA MD MN WI CO CT IA OR NM HI ME NH RI DE DC VT VA NV

Next up is the current Democrat win probabilities for each State as well as how those win probabilities have changed over the last week.

Next up is the 100K simulation of electoral college votes for the Democrats which, because of you fine folks out there, adjusts for state non-independence.

Next up is a tracking of the mean, median and modal values of those simulations since April, as well as a comparison between the headline market Democrat win probability and the simulated Democrat win probability (important to see if the State simulated markets are a leading indicator of the headline market -letting us know if the State markets collectively contain more information)

And finally, current State probabilities as a map, with the electoral college votes of each state given as well.