Possums Pollytics

Politics, elections and piffle plinking

Analysing the Poll Bias: Morgan vs. Newspoll Part II

Posted by Possum Comitatus on July 8, 2007

Continuing on from Part I, where we found that Morgan does show higher primary vote estimates compared to Newspoll for the ALP and lower estimates for the Coalition, let’s move onto the question of “by how much?”

First up, let us compare the aggregate monthly primary vote estimates for the ALP and the Coalition that both Morgan and Newspoll have produced since 1996.

mvnpgovprimary1.jpg

mvnpopprimary1.jpg

As we can see, Morgan tends to produce higher estimates for the ALP and lower ones for the Coalition, however there is an awful lot of variation between the two over the period from April 1996 to June 2007(which is the common sample I used for the two series).Since 2001, Morgan has consistently produced higher ALP primary vote estimations than Newspoll, but only since mid 2004 have Morgan consistently produced primary vote estimates of the Coalition that are lower than Newspoll.

If we take a three month moving average of both Newspoll and Morgan primary vote estimates for the ALP and the Coalition, we can not only remove some of that volatility, but it will even up those split Morgan polls a bit.

3mamvnp1.jpg

I had this theory that a large chunk of the differences between the two polling organisations could be explained in terms of the way each measures the minor party and undecided vote. While that may be the case, I couldn’t find a great deal of quantitative evidence to support it, with most analysis only showing that the minor party and undecided vote interaction could explain between about 15 and 20% of the difference in the two polls across the whole time series.

However, what did become apparent was that since 2005, the larger the size of the minor party and undecided vote as measured by Newspoll could account for about 40% of the size of the difference between the Morgan and Newspoll primary vote estimations. This isn’t that surprising considering the analysis of swinging voter movement over that period.

Now, getting to some numbers. If we subtract the monthly Newspoll primary vote series from the monthly Morgan primary vote series for each party we’ll end up with some idea of the size of the differences in estimation between the two polls for both parties over the period April 1996 through to June 2007.

alphist1.jpg

coalhist1.jpg

From this we can say that over the sample period, Morgan has on average estimated the ALP primary vote to be about 1.5 points higher than Newspoll, and estimated the Coalition primary vote to be about 0.7 points lower than Newspoll.

However, if we take the period from January 2000 through to June 2007 and repeat the process, a different pattern emerges.

alphist20001.jpg

coalhist20001.jpg

From this we can say that since 2000 Morgan has, on average, estimated the ALP primary vote to be about 2.3 points higher than Newspoll and estimated the Coalition vote to be about 1 point lower than Newspoll.

AddThis Social Bookmark Button

About these ads

23 Responses to “Analysing the Poll Bias: Morgan vs. Newspoll Part II”

  1. Alan H said

    I find it interesting that for the only poll in the last 5 years for which there is any ‘real’ figure with which to compare, ie the polls immediately before the 2004 election, Morgan (45.5%) was closer to the actual Coalition Primary (46.7%) than NewsPoll (45%) or Nielsen (49%), and Morgan (38.5%) was also closer to the ALP actual primary (37.6%) than NewsPoll (39%), and only marginally further away than Nielsen (37%). Since we have no idea of how far away the ongoing polls are from ‘reality’ (whatever that means), surely we should just go with what we know, that in the most recent testable case, Morgan was better at forecasting the actual primary vote than NewsPoll. On what possible basis should we decide that the Newspoll or Nielsen primary vote estimate is ‘better’ than Morgan’s.

  2. Richard Jones said

    We need to look at how they each gather their data and what time of day or night and whether face to face or telephone. It goes without saying that not everyone has a phone. Also people with different voting intentions are more or less likely to be in at certain times, Thursday night for instance. Whilst a number of younger people may be contacted by one pollster than another, it does not mean they will actually vote or are even on the roll. Do they ask people if they are on the roll?
    What the charts show is that we have to take all the polls together and average them to come out with a reasonable snapshot of how people are thinking at that time.
    The “soft” vote conjecture based on whether people think the country is heading in the right direction also may not be accurate. Many of the 20% ALP “soft” vote may think we are heading in the right direction and would also do so under Rudd’s leadership for example.

  3. Possum Comitatus said

    Alan – You’re right. We don’t know which of Morgan or Newspoll is biased against the true voter sentiment, just that they are biased against each other.I answered this in more depth over at your question on Pollbludger.

    Richard – Most of the polling organisations tend to do their surveys over a large time window, so rather than just ring up 1000 people between 4 and 6pm on a Sunday arvo, they ring people over, say, Friday, Saturday and Sunday including mornings, afternoons and early evenings. By doing it that way they attempt to reduce the effects of certain demographic groups being under represented in surveys because of their lifestyle or likelihood of not being home a particular time. Its not perfect, but it should in theory minimise the those types of problems.

    I don’t think they do ask whether people are enrolled to vote, well I cant remember being asked that and I’ve been polled by both Galaxy and Newspoll over the last 6 months (which is quite a statistical anomaly in itself!).

    Morgan uses face to face polling sometimes and telephone polling at other times. Morgans face to face polling often produces results which aren’t quite “out of whack” with the other surveys, but they certainly produce primary vote estimates that are fairly different to the other polling groups, with an ALP primary vote estimation being larger than usual.

    You’re right on the averaging as its not an individual poll that’s important, but cumulative incremental changes over time that shows changing voter sentiment. And that works for all polling organisations as they all move together in the same general directions over time.Probably the most accurate “snapshot” measure of voting intention is Bryans graph over here:
    http://www.ozpolitics.info/guide/elections/fed2007/polls2007/

    Combined poll measures are theoretically more accurate and contain less uncertainty than any individual poll.

    As for Morgans “soft vote” measure – no offence to the good people at Roy Morgan Research, but it’s complete rubbish in my opinion.

    People can believe the country is moving in the right direction and still want to change government. People might think the country is moving in the right direction because they believe the ALP is going to win the next election! It’s too ambiguous a question to be able to draw any valuable information from in terms of voter sentiment.

  4. wilful said

    I was a poller for Newspoll a long time ago (1997-8) so I don’t know if they’ve changed their practices subsequently, but my experience was that the poll was always done on saturday and sunday, 9 – 4 (10 am sunday), as part of a larger market research poll, taht would go for 15 minutes or so. We didn’t ask whether they were enrolled to vote, we assumed that if over 18 they would be voting. The Melbourne office covered VIC, SA and TAS.

    The demographic spread was always monitored and we’d receive regular instructions throughout the day to exclude certain types eg women over 50. Generally start with an open book and gradually narrow it down. 18-24 yr old men were always the hardest, and inner city people too. Country folks (particularly Tassie) were easy, we liked calling them, would get a response maybe 1 in 3 from them.

    We’d basically hang up on non-english speakers – too hard to bother with. You would always make personal variations on the question, it would rarely be read exactly as stated, however these would be non-systematic unless the question was a dog (which was not the case for the poll Q).

  5. There’s a couple of quite interesting insights in what you said, wilful. First, I guess non-English speakers who are voters are underrepresented in Newspoll. Is that an anti-Labor bias? (I wonder how Morgan handles this in face-to-face calls?) Second, the variation in the way a question is asked may mean the results don’t mean what they seem?

  6. Stig said

    I wish ABC’s “Insiders” would use your assessment of the relative values of Morgan & Newspoll, rather than the flatulence that Piers Ackerman comes up with. How surprising was it that he rated the News Ltd-commissioned Galaxy and Newspolls as most worthwhile?

  7. Possum Comitatus said

    Wilful,
    That’s interesting what you say about non-English speakers, although (assuming that non-English speakers really do significantly favour one party over the other), the actual difference it would make to the results would probably be hidden in most polls by the margin of error. It would certainly have more of a serious influence however in the marginal seat polling that is done occasionally, as some seats have quite thick densities of non-English speakers.

    Stig,
    This is trying to be a quality site and I will not have the likes of Ackerman mentioned unless it starts something like “two Ackermans walk into a bar” ;-)

    Morgan gets a hard time by people, especially pretend journos that couldn’t find the buttons on their calculator – but their primary figures are no more or less accurate than other polling groups as all of them usually get the primary vote estimations right at election, within the window of uncertainty that exists for them (which is the margin of error plus the people that are still undecided just before the election).

    Their face to face polls still seem to show a larger variance in their results than their telephone polls though – I’m not sure why that is and if anyone from Morgan could tell me, I’d be most appreciative.

  8. Tomasso said

    Hi Poss,

    Nice to see that Crikey’s story number 1 today was basically about this fine site! Someone is listening. And now about 30000 more (aka “Crikey’s Army”) will tune their eyeballs…

    Tomasso.

  9. EconoMan said

    Congrats possum on the increased blogwareness. This topic has been getting mentions at a bunch of blogs (LP, Pollbludger – by your good self), and even on Morgan’s site with release of their latest phone poll.

  10. Pollio said

    I think a reasonable summary of the 2004 election was that they all did ‘pretty well’ on primary vote, but that Newspoll and Morgan’s preferences did not work.

    A better reason for disregarding Morgan is the 2001 election when they measured the Labor landslide that never was. This is the cause of most journalists and politicians ignoring the Morgan numbers (Ackerman, of course, is a goose but was just repeating what everyone in the gallery thinks).

    Not sure what everyone else thinks, but I feel pretty irritated by Crikey allowing Gary Morgan to self promote in their number one news item.

  11. Tomasso said

    Roy Morgans are often given prominence in Crikey. I expect they can’t find a friendly commentator at Newspoll. Likewise, Rod Cameron gets a bit of a run (for more in depth), but Textor and Co, and Kroger and other Lib tech strategists are not seen. Galaxy got a lot of coverage too, but in retrospect they might have been better off without it.

    This was the first time Possum’s site got a mention (although the quote was a comment by Alan H). I’m far happier with what I read here than what I read in speculatorland.

  12. Possum Comitatus said

    Hiya Tom and Eco

    I wonder if this makes me a celebrity?

    I’ve always aspired to be a vacuous famewhore!

    Pollio – Crikey and Roy Morgan Research are a business like any other.The Oz/Newspoll, the News Ltd Tabloids/Galaxy and Fairfax/AC Nielson relationships are pretty much the same.

    I wonder sometimes if we try to hold the comercial online world to a higher standard of ethical purity than we do with the treeware mainstream media, even though they’re all in the same business of running a business?

  13. Tomasso said

    I recall when both The Oz and the SMH had real credibility. Nowadays I don’t know why I even subscribe to any papers. SMH letters can be witty, but the opposite page is a bunch of vagrant commentators (probably excluding Gittins and Hartcher) and tokens (eg, Abbott and Plibersek). All the pages before the letters pages have a majority dubious selection and focus.

    So when it comes to reporting “opinion” polls, it’s about the best story that can be spun from a bunch of numbers that I (professionally) find a challenge to make any conclusions from. AND a punching headline that is usually aligned (IMHO) with the publisher’s bias. And all to boost advertising rates (rates are tied to circulation). The consumer is the goose that lays golden eggs, but a somewhat trainable goose.

    So why complain about this? Why try to make any sense? Why not let Poss join the elite ranks up there with Piers and Miranda, and Mr Ramsey (for “balance”)?

    Because the goose is somewhat trainable, and the media (incl online, but maybe it has less pull, despite more immediacy). Because voting behaviour can be influences by herd behaviour and driven by finely tuned stimuli (message, bribes, dirt, etc). Measuring and tracking that is part of democracy, in the same way as derivatives are part of the shares market. It’s just that the market derivatives work a whole lot better than the commentariate and their followers.

    See, I can rant as well as anyone.

    Tomasso.

  14. EconoMan said

    “I’ve always aspired to be a vacuous famewhore!”

    You know that’s why you do it Possum. It’s certainly not the streams of advertising and subscription revenue :P

    Spot on re Crikey and morgan’s relationship too.

  15. Kevin 007% policy differences said

    Thats not surprising. I worked in commercial market research for almost 5 years and know how disinterested people can be, especially after talking about dog food for 15 minutes. And also not surprising about the unfair treatment of non english speakers. Its the same with TV ratings participants too where because you dont speak english you dont really get a proper say in things. I just can’t believe how even Dr Haneef can give Howard a boost, this is when the AFP are proven to have made mistakes and then the Immigration Min. has just reaffirmed this countrys paranoia about foreigners. Did you know 41 Anti terrorism laws were passed since 2001, thats more than most other countries.

  16. Kevin 007% policy differences said

    BTW, thats a very liberal looking squirrel with the gun and blue tooth savvy look there

  17. Iain said

    So, two Akermans walk into a bar.

    One says “That’s a very Liberal looking squirrel with the gun and blue tooth..”

    The other says “It’s supposed to be a Possum. I’m confused. Let’s ring Textor.”

    Days pass.

    Turns out its a beaver. And it needs a Brazilian.

  18. Peter T said

    10 years or so ago my job required me to read most of the feeds (AAP, Reuters) on overseas news on selected topics, as well as what was published in the mainstream dailies (SMH, Age, Australian). Fairfax was fairly straight – would put together 2-3 articles, editing for space and local interest. The Australian was far more selectively edited – unwelcome qualifications left out, tone altered. EG – “guerillas kill three soldiers in retaliation for village massacre” was likely to come out as “leftist bandits murder three soldiers in savage attack….”. Rupert ran a tight shop.

  19. apostrophe said

    I was polled by morgan once and I found it a very intrusive experience. Most of it is marketing and there are a lot of personal financial questions. I’m sure I gave a few misleading answers.

  20. Melbcity said

    Should you not have also overlaid the actual results/outcome of the election. Also a comparison of the polls with the Senate results would begin to show up the accuracies of the various opinion polls.

    It would be useful if the pollsters provided a state by state breakdown of voters intention by showing each party. This data could then be used as a seed to project the outcome of the Senate helping to determine the upper and lower thresholds for each party to win representation. Based on some of the data I have seen it looks like the Greens will need to secure ALP preferences/surplus first and foremost in order to be in the chase for a seat.

  21. follow the preferences said

    1)Possums comment on preferences struck me as really important. Newspoll always underestimate minor numbers.
    2)When you think of this election, imagine that Rudd had been the leader in 2004, he wins with as minority government supported by independents, (Ala vic 1999) then get the mother of all swings in next election (ala vic 2002) Now add these together and you get somewhere out in the stratosphere of the Morgan poll.
    3)Latham stumbling out of the bushes also brings up the invisible 2-3%. The argument goes that the ALP ‘won’ 2-3% in this election simply by not having Latham as the leader, ipso facto they only needed a swing of say 2.5-3% to win government etc etc.

  22. Pnzwjpqi said

    NsHa2H comment2 ,

  23. Observer said

    I have heard a few complaints on radio (and read in the media) that Newspollsters ascertain political preference and then hang up on Labour voters – i.e. there is strategic bias factored into Newspoll’s results. Given Newspoll was part established by News Limited (which may still have a controlling influence), and is published ‘exclusively for The Australian’, I think it goes without saying that it is highly likely that Newspoll is used as a political tool to drive for support for the coalition and against Labour (or those out of favour with News Limited). The results here bear it out to some extent. I invite former Newspoll employees to comment on their experience of what instructions they were given.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
Follow

Get every new post delivered to your Inbox.

Join 57 other followers

%d bloggers like this: