Analysing the Poll Bias: Morgan vs. Newspoll Part I
Posted by Possum Comitatus on July 6, 2007
There’s a lot of Hoo Har in the punditry over the differences between Morgan and Newspoll results. So let’s take an actual look at the differences between the two, and the size of the differences that are evident at different primary vote levels for the ALP and the Coalition.
For this, I’m using Newspoll and Morgan polling data for the primary vote, starting from the first poll published after the election of the Howard government in 1996 and including every published poll of each organisation since, right through to the present day. This gives us 315 Morgan polls to work with and 282 Newspolls.
To start with, let’s look at the Probability Density Functions for the primary vote estimations of the ALP and the Coalition, and compare the two results. For those that don’t know what a PDF is, you could zip over here for a brief explanation. Alternatively, you could just look at them as smoothed histograms with an area under the curve equalling 1.Basically they just tell us how common in each organisations polls various primary vote values are for each party.
At this stage we better add some basic statistics to go with it.
The stats are pretty self explanatory – although the mode score for the Newspoll ALP is strangely low, but that could be a bit of an artefact of chance more than anything else.
As you can see, the differences are fairly obvious. What stands out in those PDFs is that the difference between the Newspoll and Morgan estimates of the ALP primary vote are much larger around the centre than the differences for the Coalition. What’s also evident is that Morgan estimates lower scores for both parties more often than does Newspoll.
If we take the Cumulative Distribution Function of those PDFs, whereby we cumulatively add the PDF values for each party according to each polling group, we can pull even more info out in a manner that is much easier to explain.
The way to read these CDFs is simple. Take a primary vote value from the bottom axis, trace it vertically to where it intercepts either a Morgan (black) or Newspoll (red) curve, and trace horizontally from that intercept to the left axis to get the probability of that organisation producing an estimate of the primary vote that is less than or equal to the primary vote value you chose.
For instance, let’s use the ALP CDF and take a primary vote value of 40 on the bottom axis. If we trace that vertically we see that it intercepts the Morgan poll (black line) at about 0.37.That means that a Morgan poll will estimate the ALP primary vote being less than or equal to 40, thirty seven percent of the time. However (using the same technique), a Newspoll will estimate the ALP primary vote being less than or equal to 40, fifty percent of the time. Hence either Newspoll underestimates the ALP vote or Morgan overestimates the ALP vote at that value.
Armed with that, we can see a few more interesting things. Morgan and Newspoll get fairly identical results for the Coalition when their estimated primary vote is above 45%.But below that 45% level the two polls increasingly diverge in their estimates with Morgan estimating lower levels of support for the Coalition (compared to Newspoll). The lower the estimated vote, the larger the gap is between the two polling results for the Coalition.
From this we can conclude that either Morgan underestimates the Coalition primary vote when that vote is less than 45%, or alternatively that Newspoll over estimates the Coalitions primary vote when that vote is less than 45%.
If we look at the ALP CDF, we see that when the ALP estimated primary vote is between about 36 and 50, either Morgan overestimates the ALP primary vote or Newspoll underestimates it.
As for how large is the difference in estimation sizes between the two organisations – well there is no straight answer for that. The difference isn’t homogenous, but generally it’s in the ballpark of 1-2% for the primary votes. However, the difference between Morgan and Newspoll is generally greater when it comes to estimating the ALP primary vote than it is when it comes to estimating the Coalition primary vote.
UPDATE: I’ve had a ferret around the size of the differences between the polls and put some numbers to them in Part 2