Polling Analysis and Carr-ying on
Posted by Possum Comitatus on December 12, 2007
A long time ago in a poll far, far away – the September quarterly Newspoll breakdown to be precise, some people got a bee in their bonnet about such outrageous overanalysing of the polling data.
The key problem, despite many a clarification to the contrary both here, Poll Bludger and just about everywhere else in the known pollyjunkie universe, was a simple one where critics refused to listen to what was actually being said, preferring to make up their own interpretations of what the key figures produced actually meant. Explanations became pointless and the only way to address their particular problem was to simply wait for the election results and demonstrate the point with real world data.
So today we can use actual electoral data to repeat the exercise to achieve two things, firstly to test how this method stacks up against using the usual national pendulum approach when it comes to estimating the number of seats to fall from a given swing, and secondly to highlight using real world data why some critics completely missed the point.
The Newspoll quarterly breakdowns give us 2 sets of figures as ammunition for polling analysis, firstly they give us the State swings for NSW, Vic, Qld, WA and SA. Secondly they give us the swings in safe Coalition held seats, safe ALP held seats and marginal seats – where safe seats are defined as being held on a margin greater than 6%. So what we will use here is what we used last time, 139 seats in the 5 states that Newspoll measures (we’ll remove the two Independent seats from the mix).
So if go over to the AEC and extract just that data for the election result (simulating Newspoll quarterly data), we end up with the following:
Next we need to take the ratio of the Marginal Seat swing to the National Swing, which in this case is 5.1/5.6 = 0.91, then do the same for Safe Coalition Seats 6.08/5.6=1.09 and again for Safe ALP seats 4.79/5.6= 0.86
What we will do here is make the assumption that the ratio of the swing types will hold between States – meaning that in every state the average swing in the marginal seats for that state will be 0.91 multiplied by the State swing for that State. So the swing in NSW marginal seats will be 0.91*5.98 = 5.45. We then do that for every State and we end up with a populated swing matrix of:
This assumption may not hold exactly – but that’s OK, the differences should come out in the wash at the end, the point here is to estimate the number of seats to fall given the data that Newspoll quarterly breakdowns provide us with. It’s a pendulum within pendulums approach.
Next up we need to remove any over or under cooked feedback effects within states, between the movements in their 3 seat categories and their total state swing – so let me introduce to you a thing called a swing unit. A swing unit is simply the number of seats multiplied by a swing. If we have 10 seats, and applied a 5% swing, we would have 50 swing units.
So the number of seats and their type can be represented as:
To get the swing units for each seat type, we simply multiple, for example, the marginal NSW seat number (11) by the marginal NSW seat swing (5.45) to get 59.9 swing units for that type. If we do that for all seat types (and where we also multiple the totals of the State seats by their respective State swing) we get:
|Total swing units||276.7138||187.2184||224.2586||72.0785||31.8435|
|State swing units||287.04||194.62||218.68||74.36||31.95|
As we can see, the total swing units for NSW using the sum of the marginal and safe seats is 276.7, but the total number of NSW seats multiplied by the NSW swing produced 287 swing units. So what we want to do is adjust these swings by the ratio of those two numbers for all estimated swings.
So for NSW marginal seats, the swing becomes the original estimated marginal seat swing in NSW (5.45) multiplied by the ratio of total swing units for NSW (276.7) to State swing units for NSW (287.4).
Hence adjusted NSW Marginal Seat Swing becomes 5.45*(287.04/276.7)= 5.65
Doing that for all seat types gets us the following swing matrix:
Now it’s simply a matter of applying these swings to the relevant seats. The easiest way to do it is to simply list all 139 seats we are talking about and their pre-election margins – where positive margins represent ALP seats and negative margins represent Coalition seats. Then we just add these swings to the seat margins according to the type of seat e.g. NSW marginal seats all have 5.64 added to their margin, QLD safe Coalition seats all have 8.27 added to their margins and safe WA ALP seats all have 1.83 added to their margin.
The purpose of the end result is to try and get an accurate estimate of how many seats would fall given the data that Newspoll quarterly breakdowns provide us with. What isn’t important is the actual projected margin on any of the seats – that is entirely unimportant – and it’s where earlier critics lost the plot despite having it told to them repeatedly.
What is important is how many seats would be projected to fall using those numbers, not any given number itself.
This methodology is effectively a large number of pendulums all put together, pendulums within pendulums, so seats with a given projected margin will more than likely end up having either a greater or lesser actual margin than what was projected – but for every seat that ends up with a higher margin, another seat will end up with a smaller margin simply because swings tend to be normally distributed around a given mean swing. To give an example of this, if we look at the 148 seats where major parties were the victor, and show the size of the swings to the ALP as a histogram – we get a very normal looking distribution, a bell curve:
So armed with all that, and applying those swings to the relevant seat types we end up with the following projected number of ALP seats:
|ALP seats||Division||Proj margin||Coal seats||Coalition Seats||Proj margin|
Out of the 139 seats analysed, we have the ALP winning 75 of them. Then using the national swing to project to the ALP the seats in the states and territories that Newspoll doesn’t use in the quarterly breakdown we get: Tasmanian seats (5), ACT seats (2) and NT seats (2).
The total estimated number of seats using just the data of the type that the Newspoll quarterly provides is 75+5+2+2= 84
Which just so happens to be the actual number of seats that the ALP won.
If we used the national pendulum approach instead, and projected a 5.62% swing – we end up with only 81 seats being projected to fall.
This is why I use this methodology for the Newspoll Quarterly breakdown. I’ll say it again, it’s not about any given projected margin – for it’s simply a set of pendulums, it’s about the total number of seats that those projected margins estimate will fall.
So those that criticised the methodology on the basis of not understanding it in the first instance, refusing to allow it to be explained to them in the second instance, and simply making shit up about it in the third by projecting onto it meaning it does not contain (which tends to happen when one doesn’t understand something and refuses to listen to explanations of it) – well the proof is in the pudding. 84 seats projected to fall using this methodology (and only the type of data that Newspoll provides in its quarterly breakdown) vs. 84 seats actually falling.
Over to you Dr Adam Carr.
Adam gave a reply you can see over here.
45 Responses to “Polling Analysis and Carr-ying on”
Sorry, the comment form is closed at this time.