Poll Wars: Episode 1 – The Phantom Metrics
Posted by Possum Comitatus on July 11, 2007
“I was stimulated to consider whether Dennis Shanahan was right when he interpreted the turnaround in John Howard’s better prime minister rating as something more than merely encouraging. How could this be a turning point in the campaign if voter intention has not moved in spite of the Prime Minister’s improved ratings?
The question is whether the data supports the view that a turnaround in Mr Howard’s better PM rating presages an improvement in the Coalition’s electoral stocks. The short answer to this question has been yes in the past three elections.”
The fact that anyone could get stimulated over Baghdad Bob is concern enough – but I suppose that’s what Martin has got to say. But what is of real concern is that last paragraph. Let us focus on the phrase that is the glue of Martins piece; “whether the data supports…”.
We’ll get to some interesting regression models later on that completely tear this argument a new one, but first off let’s look at whether the Preferred PM ratings have clearly moved over the periods leading up to the last three elections: Below are the graphs of the Preferred PM rating for Howard and the Government Primary Vote over the 1998, 2001 and 2004 periods. The dashed vertical line is the election in each graph.
Well 2 out of three ain’t bad. Since we are looking at the effect of Preferred PM from June to July onwards for an election somewhere around November, and taking into account the margin or error on these polls which is plus or minus 3%, we be confident that in 1998 and 2001, Howards Preferred PM rating went up as the election approached, but in 2004 we simply cannot say that at all. In fact, in 2004 the only thing that can be said is that Howards PPM rating didn’t move in any way that could be classed as statistically significant between April and October of 2004.
Before we move on any further, I’d like to show you a couple of interesting things.
Firstly, what it is we are actually talking about:
In the 1993 election, Hewson was preferred PM over Keating in both February and March and lost. In the 1996 election, Keating was Preferred PM over Howard in February and March and lost. The theory of Preferred PM being the great deterministic force behind election victories isn’t looking too crash hot.
Second, have a squiz at Preferred MP ratings for the Opposition and Government overlaid against their respective satisfaction ratings (using Newspoll data):
From this we can see that PM satisfaction levels and his preferred PM status pretty much walk hand in hand, but the same cant be said for the Opposition. What is also worth taking a look at here is how the preferred PM rating from Howard became slightly decoupled from his satisfaction rating around mid 2005.Let’s take a closer look:
The reason for this is simple – there are more complex dynamics at play. The period directly after the 2005 budget saw around 5% of the Coalitions primary vote move over to the Minor Party+Undecideds camp.See Determining the Swinging Voter and Their Behaviour for a detailed explanation of this.
But what’s important here is to notice the consequences of that movement carry across into preferred PM ratings. Howards satisfaction ratings went down, people were shifting their primary vote to the minor parties and into the Undecideds camp but they still preferred Howard over Beazley. Howard kept the PPM status by a powerful lack of alternatives.Then along came Rudd and bang, PPM changed.
Now, keeping all that in your thought orbit, let us move on to test the other key argument made by Martin:
“The question is whether the data supports the view that a turnaround in Mr Howard’s better PM rating presages an improvement in the Coalition’s electoral stocks.”
Martin is saying that the Preferred PM rating is a leading indicator for the Coalitions primary vote.
This we can test – in depth. We’ll use the following variables:
C = a constant that is handy in these equations to measure the base value of the dependent variable (GOVPRIMARY in this case) that isn’t explained by movements in the values of the independent variables. This is automatically calculated by least squares regression software.
GOVPRIMARY = governments primary vote
PPMGOV = Preferred PM rating of Howard
And we’ll sometimes use (-1) or (-2) after those variables to represent a lagged variable i.e. PPMGOV(-1) is a variable that whose current value is last months value of PPMGOV.
And we’ll use the sample period of the Howard governments reign from April 1996 through to July 2007.
For those of you with a statistical bent – the following is a “cut down for a wider non-stats audience” affair, so please refrain from giving me grief. I’m sacrificing minutia for readability.
To start with lets run the obvious regression which explains how much of the government primary vote can be explained by changes in the previous months value of PPMGOV (which is Martins argument – what we are testing is whether PPMGOV(-1) is a leading indicator).
This suggests, very very superficially (I’ll demolish it in a minute), that Martin may be somewhat right.22.7% of the movement in the governments primary vote can be explained by movements in the previous months PPM rating.It suggests that a 1 point increase in this months PPM will lead to a 0.21555 increase in the governments primary vote next month.
Now let us run the obvious next regression which explains how much of the government primary vote can be explained by changes in the current months value of the PPM rating for Howard.
This suggests that 30% of the movement in the governments primary vote in any given month can be explained by changes in the value of the PPM for Howard in that month .So, superficially, this suggests that changes in PPM values walk hand in hand with changes in the governments primary vote rather than being leading indicators.
Now let’s run both variables together:
When we combine the dynamics of both the lagged value of PPM and the current value of PPM for Howard something stands out. The lagged value of PPMGOV called PPMGOV(-1) is not statistically significant. This means that is doesn’t actually explain any statistically significant movement in the government primary vote when taking into account the influence of the current value of the PPM rating for Howard.
There is another problem here tied into this. This equation is auto-correlated up the wazoo. This means that there is an awful lot of lagged explanatory power still laying around in the residuals of the equation that is messing with our explanatory variables. If we look at the correlogram for that last equation we get:
This means, in this particular case, that the previous value of the governments primary vote has a big influence on the current value of the governments primary vote (the inertia effect that you often find in voting estimation series) and is interfering with out modelling. We can make another regression which accounts for this using a one period lagged value of GOVPRIMARY:
This clears up all the autocorrelations and gives us a nice white noise process in the residuals which is exactly what we want.
What this new, clean equation tells us is that the 57-58% of the movement in the value of the governments primary vote can be explained by changes in the values of the 3 key variables of GOVPRIMARY(-1), PPMGOV and PPMGOV(-1).
However, far from Howards preferred PM rating being a leading indicator, it’s the opposite.
Put simply, what this means is that if we hold all the other independent variables constant, a one point increase in the last periods value of Howards preferred PM rating will lead to a 0.21 point reduction in the governments primary vote!
That is EXACTLY THE OPPOSITE of what Martins theory tells us should happen.
Preferred PM ratings for Howard are simply not a leading indicator for the governments primary vote, period. Their lagged effect is smaller than the effect of the PPM ratings current value, and that lagged effect is completely the wrong sign.
Howard’s PPM rating is covariant with the government primary vote, in that the two values move together through time, likewise the lagged values of the government primary vote are a leading indicator for futre values – but the lagged PPM values mean diddly squat.
I can appreciate that Martin tried to defend The Shanahans piffle over interpreting polls, but some piffle is simply not worth defending.
His last paragraph sums it up:
“So, will the PM drag the Coalition up with him on the strength of his personal rating and will that be enough to overcome Labor’s commanding lead? The next few Newspolls will answer these questions.”
Watch out for the next Newspoll! Buy the Australian to get it!
Same bat time, same bat channel! ;-)
Which is fair enough.