Possums Pollytics

Politics, elections and piffle plinking

Request for mental assistance on US Election Intrade stuff.

Posted by Possum Comitatus on June 26, 2008

I’m having a few problems with using the Intrade data for US election analysis and I’d love your help or assistance to figure a few things out. This might not be for everyone as it gets a little nerdy – but for those without stats backgrounds, I’ll use some charts and stuff to explain the maths in ways that will hopefully make it a bit more digestible.

I’m more interested in using the State by State betting markets on Intrade, the “Democrat/Republican to win State X” as a focus because I think those markets actually contain superior information than any of the headline markets such as “Democrat/Republican to win the Presidency”.

My thinking on this comes from seeing the US Presidential election not as a single election, but a collective result of some 51 individual electoral contests (the States and DC) that just all happen to occur on the same day.

I reckon that the average amount of information that a participant in a State betting market has about the true political situation on the ground, as a proportion of the total amount of political information available about that State, is on average, higher than the average amount of information that a participant in the headline markets have as a proportion of the total amount of information that is available about all 51 electoral contests in the US.

As a result, I see the State markets as containing, both individually and collectively, far superior information about the true “current state of play” in the US political system.

The problem comes in deciding which way to aggregate that info.

Over at the US election page on the site, I’m using monte carlo simulations to try and aggregate the information contained within those State markets. But I’m not using monte carlo simulations in the usual way that it is done, because I don’t believe that the usual way is actually a valid methodology for political betting markets.

For those of you reading that just went “WTF is a monte carlo simulation”, fear not, it will all become clear in a bit.

The way monte carlo simulations are usually used in these markets is, very basically, that you take a State, lets say Ohio , and look at the probability the Intrade market gives it to be won by a particular party- let’s say the Democrats. At the moment the Intrade probability for the Democrats winning Ohio is 67%, or 0.67.

Then we generate a random number between 0 and 1( to, say, three decimal places) and we compare our random number against the Intrade probability – if the random number is 0.67 or less, then we give the Electoral College Votes for Ohio to the Democrats, if the random number is greater than 0.67 we give those electoral college votes to the Republicans. But we don’t just do this for one State, we do it for all 51 electoral contests at the same time, and after every state has one random number generated for it where it is compared to the Intrade probability and the college votes are distributed accordingly, we add up all of those college votes that each party would win across the country to get a simulation of the election results.

Then we do that exact same thing a million more times to end up with 1 million possible outcomes that look like a bell curve. The mean of that bell curve is the mean number of electoral college votes the Democrats will win after 1 million simulations, and from that we can calculate the current implied probability of a political party not only winning the presidency according to the Intrade State markets, but the probability of them getting any number of electoral college votes in total.

That’s the way it is usually done, but I think that methodology is completely and utterly invalid to use for Intrade political betting markets. Let’s say on the day before the election, the Intrade markets have the Democrats a 5% chance of winning Alabama the next day.

Theoretically, using the orthodox monte carlo approach, if 100 elections were held the next day, the Democrats would win Alabama in 5 of them. Now that is clearly nonsense. You could have one hundred thousand elections the next day and Alabama wouldn’t turn blue in any of them! Ordinarily we would expect those strange results to wash out in the million simulations, and those extreme ones do – but there is also a problem with the less extreme probabilities which we’ll get to in a bit. This is just a simple example of the sorts of problems we face.

So rather than deal with these funny little “not in a million years but regularly on Intrade” results that occur in the simulations, I’m doing it differently.

I create for each State a normal probability distribution with a mean of their current Intrade probability and a standard deviation of, currently, 0.2.  I then generate a random number from that probability distribution and if that random number is greater than 50% I give the State to the Democrats, if it’s less than 50% I give it to the Republicans. I then do that once for every state and add up the electoral college votes, then repeat the process a million times to end up with a bell curve of the electoral college results that gives us our implied State market probability of the Democrats winning the presidency.

For the non stat types, the way to visualise this is if we take an imaginary State that has a current Intrade probability of exactly 50% for the Democrats winning it, then the distribution would look like a bell curve where the highest point on that bell curve is exactly 0.5. By having a standard deviation of 0.2, it means that around 68% of the random numbers I pull out of that distribution will be between about 0.3 and 0.7 (the mean of 0.5 plus or minus the standard deviation of 0.2), and that 13.5% of the random numbers that will be pulled out of that distribution will be between 0.1 and 0.3 and another 13.5% will be between 0.7 and 0.9 (those figures aren’t exact to so many decimal places, they’re approximately correct). This means that the random numbers which are pulled out of that distribution will be more likely to be closer to the mean of 0.5 than further away from it.

However – is that standard deviation right, should the same standard deviation apply to all States and should it stay at a value of 0.2 all the way through to the election?

For the not stat types, the following graphs will be handy. The larger the standard deviation, the wider the shape of the bell curve – below shows how it works. They are two distributions I calculated using 1 million simulations, the first is a normal distribution with a mean of 0.5 and a standard deviation of 0.2, the second one is the same except it has a standard deviation of 0.1. Notice how the smaller the standard deviation, the tighter the range of random numbers that can be generated within it (the random numbers we generate for the US States come from the area under the curve).The smaller the standard deviation, the closer the random number we pull from the distribution will be, on average, to the mean.

On the question of whether the standard deviation should remain the same through to the election, I’m of the mind that it shouldn’t – but I’d love to hear your thoughts about it?

I think that the standard deviation we give to the individual state market distributions here should be a function of uncertainty – as in, how sure are the punters that the market is true, how sure are the punters that the probability for a given state is true?  A lot of that uncertainty is reduced by information about the state of play on the ground in a given state, information like polls for instance.

As we approach the election, the uncertainty of each state market should reduce as more information like polling gets released. To accommodate this we should probably reduce our standard deviations for the State markets over time as well.

But the big question is whether the uncertainty reduces linearly or non-linearly as we approach the election. For instance, if we chart how the reduction of uncertainty would look over time as we approach the election, both as a linear function and a non-linear function we get:

I’m of the mind here that uncertainty will reduce in a non-linear fashion, simply as function of the number of polls and the timing of their release. We can all remember our own election here last year when the number of polls released gradually increased in the lead up to the election, before increasing dramatically over the campaign period. Because such increasingly vast quantities of polls will be released in the US as Election Day looms, I’m thinking that people will become increasingly certain of their bets in the State markets as Election Day approaches.

Any thoughts?

We’ve got our current uncertainty represented by a standard deviation of 0.2 which at the moment is fairly wide, but we also need our final uncertainty to use on election eve in order to generate all of the standard deviations between now and then that we will use.

As for what the uncertainty should be on the day before the election, unfortunately we can’t really model it to get a number because we just don’t have enough data – so we’ll have to use our heads and make an assumption.

For instance, what are your thoughts on having an uncertainty level on the day before the election of a size that is represented by a standard deviation of 0.025, or 2.5%?

That would mean, essentially, that were a hypothetical State on election eve to have a 50% probability of going to the Democrats, than the uncertainty around that final result would be such that there would be roughly a 68% chance of the true probability being between 47.5% and 52.5% of the Democrats winning the State, and an approximate  95% chance of the true probability being between 45% and 55% for the Democrats winning that state.

Does that sound like a reasonable standard deviation to represent election eve uncertainty?

This also gets us back to why I think we should use this type of monte carlo simulation methodology rather than doing things the way they’re usually done.

If we believe that markets actually contain good information, then using a standard monte carlo approach is inconsistent with that belief. On the one hand we’d be saying that markets know best, but on the other treating them as if they don’t by drawing random probabilities to judge them against.

For instance, if Florida was given a 30% chance of falling to the Dems on election eve – does that really mean that if 100 elections were held the next day, the Democrats would be expected to actually win Florida 30 times? Of would that probability be substantially less on the basis that the market has probably got it right in outcome if not probability?

Especially since Intrade predicted every State result correctly last election, but often by small margins of only a few percent. It’s almost improbable that Intrade would have predicted every race were the chances of each party winning a given State truly represented, literally, by the Intrade odds. On Nov 2nd in 2004, Intrade had the Republicans in front by less than 7% probability in Florida, New Mexico, Ohio and Iowa – 59 Electoral College votes all up. Intrade actually predicted every winner, but by margins so small that suggest we should treat the results with more respect for the predicted outcomes than standard probability theory tells us we ought to.

Hence, I think that we should measure uncertainty in terms of drawing random numbers from within a probability distribution for each State and comparing that to the 50% probability mark to distribute Electoral College votes, rather than randomly draw numbers and compare it to the implied probability in each of the States and distribute Electoral College votes accordingly.

If I use this methodology (with standard deviations reducing nin-linearly) with Intrade data for the 2004 presidential election, on election eve the Mode of the simulation (the number of Electoral College votes for the Democrats that get projected most often) is 252, which was exactly the result. The final probability of the Dems on election eve was a 36.9% chance of victory. On June 21st in 2004, the Intrade State markets gave the Dems a 29% chance of victory with a Mode of 255 Electoral College votes. So the methodology plays out pretty well using 2004 Intrade data that I’m slowly gathering.

The other two big questions that I haven’t quite got my head around are:

1. Should each State have the same type of distribution, as in a normal distribution, or would there be other types of distributions that might represent them better on the basis of circumstances happening in each state – and if so, what sort of distributions and on what basis should we select them (I can effectively use any type of probability distribution known to man here)

2. Should the depth of each State market, as in the volume of contracts traded for a State market, have a say on the size of the standard deviations we give to each State’s normal distribution, and if so – anyone got any ideas on how to make that so?

High volumes of contracts traded should theoretically represent greater certainty because more people believe a given probability. So should we include market volume when it comes to determining the standard deviations of the States, and how?

And if I haven’t explained anything adequately here, please let me know or ask me, because I’d really like to construct simulations between now and US election day that try and extract the best information we can get out of those knowledge filled State markets.

All suggestions would be really appreciated.

On something else US election related, here is the Obama campaign strategy that’s been flying around the intertubes very recently (it’s a small pdf version of a powerpoint presentation). Thanks to LL for sending me that. Some of you may not have seen it, and it’s pretty interesting.

On something more local – that slayer of psephological piffle and all round electoral legend Antony Green has a spiffy blog .He tags Newspoll for being gooses in preference distributions when OPV is running.

51 Responses to “Request for mental assistance on US Election Intrade stuff.”

  1. Peter said

    Why not have a variable standard deviation? The closer the odds are to 50%, the larger the deviation. The further apart the odds are (like the 5%/95% in Alabama), the smaller the standard deviation.

    If I understood your explanation correctly, this would result in more certainty for those contests like Alabama, where a 5% chance for the Democrats might be regarded as generous🙂

  2. LuckyDave said

    Big Question 2 : should the depth of state markets have a say? Ideally you’d lean to yes, but smaller states will have smaller betting pools, unless it is a particularly contentious state with higher perceived betting “value”.

    Question 1 : Largely beyond me. However I think the idea that any state with less than an arbitrary 30% probability of a Democratic or Republican win in October being treated as a zero chance for the is legit.

    I work in sales and an approach widely used in business is to multiply all sale probabilities (even the 5% chances) by the total value of the potential sale, then add them to calculate a total probable sales value. So here we’d just take each state probability and multiply it by Electoral Votes, then add them up. Do you hate this?

    My gutfeel is that this is going to be a landslide to Obama. Larry J Sabato’s Crystal ball Blog says “Presidential Politics is always a crapshoot”. Luck goes a long way. Voluntary voting makes hard stats close to impossible. Voter enthusiasm, weather on the day all play in.

    I prefer the latest, largest polls over betting markets, because betting markets hold a book that factors in old bets (data). Whereas opinion polls are fresh. I think recent close elections have made the bookies look better than they deserve.

  3. Possum Comitatus said

    Peter – that sort of variable deviation would be good for the Alabama type states, but with the closer states we run into the problem where the actual intrade probability for each state has been exceptionally accurate in terms of predicting which way a state will go in close contests, so if we widen those standard deviations for close races, we’ll effectively be discounting the capability of Intrade to get it right. It’s a really tricky little problem because nearly every solution I can come up with for one of the problems, completely stuffs things up for elsewhere!

    LuckyDave – that whole “expected value” thing that you mention regarding probable sales shits me to tears! Over on the US page I have that expected value masquerading around as the EV probability sum.I deliberately did not call it expected anything! It will have it’s uses, but not for that.

    With Intrade, and what makes it better than usual betting markets is that there is effectively no book. In order to put a bet on, you have to find someone else willing to take your bet – so the whole thing is more like a futures market than a bookie operation. Especially as the “last price” or last contract is what makes up the odds – so old bets should be washed out of the system as soon as any new bet comes in. In theory anyway. In practice it really all depends on new punters stepping up to the plate. But there seems to be volume in the markets, so new prices at least appear to washing away old ones without too much trouble.

  4. David Gould said

    I think reducing the standard deviation over time would be a good idea. And maybe the SD should be a function of the difference – for example, in an 80 to 20 state, the standard deviation should be smaller than that in a 50 to 50 state.

    I am also of the mind that you should completely exclude states that have no chance of stepping across the line, so to speak.

    Reducing the data crunched to, say, 20 potential swing states might remove a lot of noise in any case. That would effectively take care of the five per cent chance of the Democrats winning Alabama.

    But I am not knowledgable in much of this, so take my ideas with a grain of salt.

  5. David Gould said

    As an aside, personally, I would like to see something done that simply focuses on Virginia, Ohio and Michigan and looks at the likelihood of McCain winning all three, as I do not think that McCain can win without winning all three of these states.

  6. Possum Comitatus said

    That’s a good idea DG, I might look into small collections of must win Republican states and do them seperately and more intensively as we approach the election.

  7. Peter said

    Possum – Fair enough. Well, if Intrade’s been exceptionally accurate in the past, why not work backwards from a baseline success rate?

    Here’s what I mean. Assuming past Intrade data is available, work out an accuracy rate for marginal contests. For example, for those in the 45-55 or closer range, have a look at how many Intrade got right.

    Once you have that baseline figure, work out a standard deviation which would give the result to the favoured party in close contests that percent of the time. To clarify – if Intrade has historically got 90% of the close contests right, and is predicting that the Democrats had, say, a 50.1% chance of winning Ohio, work out the standard deviation that would give 90% of the contests to the Democrats, based on that 50.1% probability.

    And then reduce the standard deviation as the gap gets larger – so if 90% was the baseline, states like Alabama would be much higher than that. You could again base the reduction in standard deviation on historical Intrade data.

    Would that sort of method be feasible?

  8. Possum Comitatus said

    That’s feasible Peter. The only historical data we can really use is the 2004 Intrade data because of the volume issues involved in the 2000 race when they first started (there wasnt a lot of volume). Intrade had 100% accuracy in predicting the winner of each state in the 2004 election, even predicting close races like Ohio which had a republican probability of winning at 51.1% on Novemebr 2nd.

    If we were to go down that path, it would leave us a really small standard deviation of around 0.5% (or 0.005) as the starting point for close races, getting smaller the further away each state probability got from the 50% mark. That would work but after running some numbers, our simulations would actually end up being – for all intents and purposes – identical to just looking at whatever the probability is for each state and assigning electoral college votes accordingly without having to do simulations.

    So we’d be effectively saying that Intrade is 100% accurate all of the time – which is the nasty consequence of doing it that way, because Intrade is pretty accurate… but 100%? That’s a big step. It was 100% accurate last time, but how much chance was involved in that result?

    It’s a real pain in the arse – we solve a problem, and give ourselves another one in the process!:mrgreen:

  9. Possum, I’m just a “bush statistician” so I don’t follow you 100%.

    A while ago I had a go at trying to work on probabilities in elections. Had an extended email conversation with Andrew Leigh (whose contribution would be worthwhile here as he seems to know a bit about stats and that). The big problem I found is that the outcome in each state is not independent; in other words if, say, a Democrat gets up in one state on election day, it is more likely that other Democrats will get up elsewhere too (i.e., the swing across the country). I understand in the 2001 federal election one of the online bookmakers (Centrebet?) just starting up with election betting, offered multis, that is, the linking up of multiple outcomes in one bet. Enough smarties realised that the money to be made was by coupling up a string of Labor victories in one bet, and a string of Coalition in another. They cleaned up because of the (relative) dependency between outcomes.

    Does this make sense and is it relevant to what you are trying to do? I’m also worried in your text where you talk about Intrade probabilities, and then say they can be “nonsense”. If they are nonsense, why use them or rely on them? If Alabama is given a 5% chance of retuning a Democrat then it *does* mean a 100 theoretical elections would return 5 Democrat results, doesn’t it? Isn’t that what probabilities mean?

    Sorry for this. Probably way off. Can you look at individual probabilities of events occurring and aggregate them *if* the outcome of the events are not independent?

  10. Possum Comitatus said

    Peter T,

    While it’s true that the outcome in each State isnt independent, in a market like Intrade (rather than statistically polluted markets run by bookies, or even polling analysis), any “when the swing is on, it’s on” type effects would be accounted for as one of the many pieces of information that are the basis of any bets in these markets – so the independence (or rather absence of it) in these markets will be priced into the contracts, effectively solving the problem.

    I have big problems with trying to apply probabilities to polling – it simply doesnt work in practice for a whole lot of reasons.

    So with Intrade, unlike polling and perhaps unlike some of the bookmaking, this isnt really an issue that is problematic in practice as long as the assumption that the people making the bets are taking into consideration things like the “swing being on” is true. And it’s probably a hard argument to suggest that participants in Intrade arent taking that into account.They’d be some dumb mofo’s if they were ignoring it.

    The problem with Intrade probabilities, or any derived probability for that matter, is they arent actually “true” in the literal meaning of the word – they are simply implied and we make the assumption that they are true in order to work with the data. From history we can be pretty sure that implied probabilities are generally, and approximately true – but nailing down exactly how truthful is almost impossible.

    We know that when Intrade says the probability of a particular Party winning a State is greater than 50%, the most likely outcome is actually that Party winning that particular State. But we cant prove that the difference between an implied probability of 51% and an implied probability of 60% is actually 9% on the Intrade markets – all we can do is either assume that it is, or assume that the difference is approximately 9%, or at least that the State on 60% has a greater likelihood of being won by a particular Party then does another State on 51%.

    The further we get away from the 50% mark (which is the threshold here), the less certainty we can have about the truthfulness of the probabilities except that we know that the State will most likely fall one way or the other depending on whether it’s probability is greater than or less than 50%.

    So even though Alabama may have a 5% probability of a Democrat victory on election eve, that doesnt mean it’s actually, really truly, a real deal 5% probability of victory – what it means in practice is that it’s so far away from 50% as to be irrelevant from contention, and that whatever price and probability it may have is more a product of people playing around the edges of the market than it is of any realistic chance of a Dem victory.

    The big problem comes on where to draw the line between being out of contention and being realistically in contention. I think that line is easier drawn through building probability distributions for each state and letting the cards fall where they may in being either over or under the 50% threshold in the simulation than saying “OK, anything under 29.4% is out of contention”.

  11. David Gould said

    Possum,

    Just on the last paragraph, doesn’t that take us back to the problem that we are discussing here – that the Intrade probabilities are most likely not accurate at the extremes? If we know that Alabama is going to be won by the Republicans, why allow for the possibility that it might be won by the Democrats? The same with California and New York, just the other way round. Aren’t meaningless numbers being crunched to no purpose for those states?

  12. Peter said

    Well, if data volume is an issue, is there any other historical data that could be considered?

    Given that you’re looking at this on a state-by-state basis, does Intrade provide any data on past State elections?

    If the volumes for State elections are too small (or they’re not recorded on Intrade), and you just want to assess the general accuracy of Intrade markets, as opposed to the US federal political markets specifically, you could also look at other markets with higher volumes. Various sports, for example – I’m sure any Super Bowl betting would have high volumes, if such is on Intrade (I had a quick look at the political betting, but that was it).

  13. Possum Comitatus said

    David,
    It’s just easier for me to crunch the numbers on all states than to choose which states to do it for.

    So if we replace the 5% probability for Alabama with a probability distribution (which I’m currently doing) that has a mean of 0.05 and a standard deviation of 0.2 – at the moment about 1.3% of the simulations show Alabama as being a Democrat victory. That is so small that it ends up not affecting the mean probability of all 51 electoral contests in any significant way, especially as there’s plenty of States at the very margins for both the Dems and the Reps that have these 1% chance of victories here and 2% chance of victories there. But as our standard deviation decreases as we approach the election, there will be zero chance of any of Alabama simulations ever showing a Democrat victory. The closer we get to the election, the number of States having no random numbers drawn from their probability distributions that ever cross the 50% mark will increase, so as we get closer the probability simulation results tighten up.

    Peter, the only other place we could use is the Iowa Electronic Market that’s been going since, I think, 1988. But that market has all sorts of betting limits and whatnot. It’s been successful like Intrade, and for a lot longer – but Intrade is a deeper market and I’d rather use that. Combining the two isnt quite apples and oranges, but it’s certainly mandarins and oranges.

    I had thought of looking at other Intrade markets and seeing how the probabilities play out, but the problem I’m having there is not being able to reconcile (mostly with myself) that sporting markets are sufficiently similar to political markets in terms of the knowledge base of the punters that would make such comparisons compatible.

  14. David Gould said

    Interesting. Thanks.🙂

  15. So I’m curious about your basic assumption. What makes you think that the people betting on the headline race aren’t already using the information available on the state odds? The effective overround in Intrade is very small, so I would’ve thought that the efficient markets hypothesis isn’t a bad place to start.

    (FWIW, I’m doing some work with Wolfers & Zitzewitz on longshot bias in election betting markets, which suggests that you may want to pay a little less attention to probabilites that are out in the tails – such as your Alabama example above.)

  16. Possum Comitatus said

    Andrew, I most certainly think that people betting in the headline market are using information about the State odds – but also a whole lot of other things. There’s a pretty good discrepancy between Obama as President and Democrat as President. There’s a discrepancy between those two markets and the markets on the number of electoral college votes each party will win, and another discrepancy between the number of electoral college votes each party will win and the State market probabilities.

    Intrade has so many markets that the information from a given market isnt flowing purely into the next – or far more likely, that the headline market(s) is/are being more influenced by newstraders and other assorted people that like being fleeced then the State by State markets are…. which is to be expected.

    I’ve said on the site a few times that I think that the Intrade markets seem to exhibit varying degrees of semi-strong form efficiency, but with the State markets having the strongest of that semi-strong form – at least as far as the 2004 result was concerned, as well as the 2008 State market being mostly ahead of the polling data and political strategy since February.

    On the long shot bias – paying a little less attention to those fringe dwellers is what I’m effectively doing by not using monte carlo sampling in the usual way, but rather turning implied probabilities of each State into probability distributions, selecting randomly from those distributions and comparing those random numbers to the 50% threshold to allocate Electoral College Votes. I’ve effectively reduced the chances of long shot States by around 70% doing it this way, which will reduce even further over time as the standard deviation I’ll be using for those individual state probability distributions decreases as the election approaches.

    Oddly enough – the entire reason I’m doing the simulations this way is to reduce the effect of what you’re calling longshot bias.

  17. Pollfoolery said

    Let’s see if I can make this make sense:

    If participants in the market are close to rational, they will place a bet (buy a contract) where their belief is that the expected payoff of the bet is above the risk free rate of return. At Intrade, we require a buyer AND a seller and they must have divergent beliefs. The further out from an election we are, the more divergent their beliefs must be to have each think they are beating the risk free rate of return. On election eve, we can imagine two rational participants being basically neutral to either side of a bet. Before that… it doesn’t make sense to think that.

    So what are the chances of market participants with quite divergent beliefs being rational? Maybe lower than the ones with only slightly different beliefs? Unless I’m thinking something horribly wrong, it seems the participants with the information that best predicts the election will have no incentive to trade with each other unless either the market is off by a decent margin or the election date is close. Does this kind of thinking justify the increasing accuracy assumption?

    And another thing… does this undermine the efficiency hypothesis for markets prior to just before election day? Doesn’t that require rational participants at the margin? But here we end up with both participants in a contract being wide of the margin by a good amount… and we are relying on a kind of symmetry of irrationality. Which is conceivable but a much uglier thing to work with than a rational marginal player.

    Are these thoughts even vaguely coherent?

  18. Py said

    What about using historical betting data (perhaps weighting it so that the oldest data gets least weighting) and constructing a standard deviation out of that?

  19. Local Identity said

    I have no f#(king idea what any of you are talking about, but I’ll have a tenna on #4 in the first at Dapto dogs, thanks😉

  20. Catrina said

    Possum

    I’m thinking about all of this and while reading through all of the above has given me a headache, I’m still thinking to myself that there is a correlation between the aggregated state results and the overall election result predictions – and I’m thinking that this difference is the factor that should be driving the standard deviation. I’m assuming here that the aggregated state probability and overall probabilities will converge – and the difference is the reflection of the uncertainty (but from memory you has said that these numbers are typically reversed which blows my thinking out of the water – however, I’ll ignore that fact for the moment).

    I think it is safe to say that as we move closer to the election date, the state level prediction becomes more certain, and the national prediction becomes more certain. I figure we can use the national level prediction to control the standard deviation. But – what I’m not clear on is the impact of certainty on the standard deviation. My intuition is telling me that we are moving from a lower standard deviation towards a higher standard deviation as the potential for significant variables are removed as a function of the time available for major gaffs/incidents reduces (in addition to the quantity of background data from polls).

    Thing is – these to measurements (aggregated state versus national numbers) should be resolvable via variable standard deviation derived as a function of the two measurements, combined with the magnitude of the event horizon (days to the election).

    Hope that makes sense.

  21. Catrina said

    Whoops
    Should have said “these two” – not “these to”.

  22. Jennifer said

    As an actuary, I have to love the first blog I’ve read that uses monte carlo simulation to advance political discourse!

    My view on reading your post was that I would want to use as much evidence as possible to estimate standard deviations – looking at potentially using the variability of the implied probability over time as a way of estimating the underlying standard deviation of the distribution, or else looking at the predictive ability of each market, compared with the size of each market.

    In your comment #8, it sounds as if using the specific Intrade history won’t work, but is there a way in which you can look at the general success of betting markets with different probabilities as a way of estimating the standard deviation? At least that will give you a way of scaling it for different underlying probabilities?

    WIthout having looked at the data, I’m inclined to agree that the state by state markets are more likely to be efficient – they are more likely to have the people betting with full information, whereas the countrywide ones might have too many people betting for emotional reasons.

  23. Topher said

    If you assume that the results close to the election are close, could you use the difference in results between what is seen further out, and the results now to work out how the uncertainty changes with time.

    Is there any correlation between the intrade bets and the final margin? That might be useful as well.

  24. Re #10 and #9, “when the swing is on” arguments may be accounted for by assuming that the market factors them in, but they can nonetheless give an inaccurate result when the results in several states are incorrectly assumed to be independent for probability calculations purposes. Aggregation is a problem for trying to use state-by-state results to get a net probability of a certain result occurring.

    I’ll give an artificial, very simplified example (which will overdramatise how much of a problem this sort of thing is). Suppose that all the following are true:

    * McCain wins the presidency if he wins states A, B and C, and loses if he fails to win all three.
    * These are states McCain is unlikely to win unless Obama goofs in some significant way (the swing).
    * The probability of Obama goofing is 30%.
    * If Obama doesn’t goof, McCain has a 15% chance of winning each state. If he does, McCain’s chance of winning each state jumps to 70%.
    * Apart from the Obama-goof factor, the outcomes in the given states are otherwise independent of each other.

    On this basis, McCain’s actual probability of winning each state is .7*.15+.3*.7 = .315

    If you simulate this by treating all states as totally independent of each other in terms of outcome, then the probability of winning all three states of .315^3=.0312

    But actually, because the Obama-goof factor makes them partly non-independent, you have to look at the probability of McCain winning all three states together under each scenario.

    Probability of McCain winning all three if there is no Obama-goof factor = .15^3 = .0034
    Probability of McCain winning all three if there is an Obama-goof factor = .7^3 = .343
    Overall probability of McCain winning all three = .7*.0034+.3*.343 =.1053

    …which is more than three times higher than if the probabilities for the three states are mistakenly treated as independent.

    This sort of issue only matters greatly when trying to work out the probability of the underdog winning. If you’re just assessing the likely margin, it shouldn’t make a big difference, although if you’re estimating the likely range of outcomes then assuming the results in given states/seats are independent will likely lead to an underestimate of the confidence range of probable outcomes.

    What I would do (if I had the data to model it on, and the programming knowledge to do it, both of which I lack) is run the randomisations as a two-stage process. Each randomisation starts with its own random value for “swing factor”, on the basis of which value the probabilities in every state are modified in a specified way, and then you run the random distribution for each state using the modified probabilities for that distribution. So, for instance, if the swing factor value is close to the maximum possible in McCain’s factor, that might turn the 80+% McCain states into 99%s for that run of the simulation, the 60% McCain states into 90s, the 20%s into 50s and the 2%s into 10s (or something like that). Then the next distribution might be one marginally in Obama’s factor, and it might turn his 50%s into 60s, his 20%s into 25s and have no real impact on his safe or write-off states.

  25. Oops; “then the probability of winning all three states of” should read “then the probability of winning all three states is”.

  26. charles said

    First off look at the graphs over time, if they had any predictive power they would be steady.

    You comments show that you want to bias the market. Like it or not there is someone who doesn’t agree with your views on Alabama, that is there in no chance Alabama will turn blue.

    What your doing with the random method is seeing the outcome with differences disturbances in the system. Its the overall result that will have a normal distribution.

    There is a change that Alabama will turn blue, I don’t know, the Republican is found nude in bed with a nude male jocky. Thats what the random allocation is about.

    My feeling is a different random input for each seat to allow for nude jockies in random seats, but before using it multiply it by a global random variable to allow for someone blowing up ground zero.

    But I would be going a test to see if the second variable made any difference. I suspect it wouldn’t

  27. charles said

    And the real interesting result would be the standard deviation of the simulation over time. I’d love to know if the SD falls closer to the election. I suspect it will. And that is the beauty of the method, the result is a distribution.

  28. caf said

    I’m a skeptic, I think that if you believe the implied probabilities are wrong, then you need to have a concrete model of why and how they’re wrong to base your corrections on. Applying corrections based on “gut feel” is too unscientific an approach for mine.

    Here’s a start for why the 5% implied probability for Alabama is being set too high for the market: Diving into Intrade’s rules, you’ll see this:

    Also, before any trade is accepted by the exchange each party to that transaction must have adequate trading capital in his/her account sufficient to cover the risk that trade represents. This guarantees that all traders will be paid by the party on the other side of his/her winning trade.

    What this means is that I, as a trader, may think that the true probability of the Alabama result really is 1%. However, if I offer to sell contracts at that price, for every $1 I stand to win I’d have to have $100 of my Intrade working capital tied up to cover that contract. That’s working capital I could be more profitably using to buy or cover contracts in other markets on the exchange, so I’m not going to offer that contract (or if I do, I’m only going to do so at a considerable premium on the odds – say, 5%, which only requires a fifth of the working capital to cover). Essentially, the “longshot effect” is down to traders having limited bankrolls. Now, the hard part is to translate this into a mathematical model of how the probabilities are skewed…

    I also agree with Peter Tucker and Kevin Bonham that you cannot run the monte-carlo simulations with the underlying assumption that the State elections are independent events – they’re not. Over many (simulated or real) elections, the outcome in a particular given state will be correlated with the outcome in other states, and you do need to take this into account. I don’t agree that this is “built-in” to the implied probabilities from Intrade, because the markets are based solely on the outcome in a single state.

  29. David Gould said

    I think it would be built in.

    If there is a general perception, say, that blue-collar workers are going to swing to McCain, then this will affect the state market more for those states where the blue-collar vote is crucial.

    Thus, we would see a correlation between the markets for Penn., Michigan and Ohio (for example).

    If this is not the case, then people operating in the Penn. market must be ignoring the blue collar effect. Why would they do this? Only if there was some other effect specific to Penn. that counterbalanced it.

    If you think that market operators would ignore a factor that crosses state boundaries, on what basis do you think that?

  30. David Gould said

    In other words, the reason that states like Penn. and Ohio tend to move together is not because there is any causal connection between Penn. and Ohio. Instead, there is something that causes Penn. and Ohio to move in the same way.

    A does not cause B. Rather, C causes both A and B.

  31. caf said

    I don’t think that the market traders are ignoring factors that cross state boundaries, but that doesn’t fix the problem. The problem is that the monte-carlo simluation still treats the contests as independent events, when they are not.

    As an example, suppose that the probabilities of a D. win for Michigan and Ohio were both 40%, in part factoring in the cross-state issue that you mention. Now treating them as independent events in the simluation says in effect “Over millions of elections like this one where the Democrats win Michigan, they also win Ohio in 40% of them; and over millions of elections like this one where the Democrats lose Michigan, they also win Ohio in 40% of them”. I do not think this is realistic. I think that if in a given election, Obama has won Michigan, it is also more likely that he has won Ohio than the a priori probability would suggest.

    It is in fact mathematically impossible for the markets to have this “built-in”, because it is perfectly possible for the true probability of winning in Michigan and Ohio to both be 40% (that is, the probability independent of the other result) and yet the probability of the result in Michigan and Ohio being the same can range from 0% to 100%. The system has more than 2 degrees of freedom, so 2 values cannot constrain it.

  32. Possum Comitatus said

    OK folks – the excellent help is greatly appreciated, and I’m slowly going through each suggestion and looking for ways to make them all work.

    On treating the states as independent, which has popped up as a big issue – you’ve utterly convinced me, we cant treat them as independent.

    Unfortunately though, we dont seem to be able to fully adjust the overall simulated electoral college vote distribution for the non-independence because we dont have enough Intrade data on the one hand to do it cleanly, and even if we could find other data, it would be extraordinarily complex and time consuming – to the point where it would need to be a regression not too disimilar to what fivethirtyeight.com is using.

    And honestly – I only have 24 hours in a day!

    But if anyone has any ideas on how to “mostly” or even partially adjust the monte carlo sim for non-independence, I’m all ears.

    We could use a ranking system of the type Ray fair used:
    http://econpapers.repec.org/paper/cwlcwldpp/1496.htm

    …which requires a number of not terribly outlandish assumptions, to get a metric for the probability of a Dem victory using the State by State data that does mostly take into account non-independence, however that method would only give us a point estimate at a given time, and wouldn’t allow us to create any real distribution at all worth looking at.

  33. caf said

    Sorry, with P(Mich) = 0.4 and P(Ohio) = 0.4, P(Mich = Ohio) can range from 0.2 to 1.0 (20% to 100%), not 0% to 100% as I said.

    Nonetheless, 3 degrees of freedom.

  34. caf said

    Possum: Well, the extremes are all states a independent (where you generate an independent random value for all each state in the Monte Carlo simulation), and all states are completely dependent (where you generate 1 random value and use it for all states). The latter means that in the 1% case (or whatever) that Alabama goes to the Democrats, the whole country does.

    As a middle ground, you can do this:

    * For each iteration of the Monte Carlo simulation, generate a mean outcome M. This is evenly distributed over {0, 1} as in classic Monte Carlo.

    * For each state election within this iteration, generate a value with mean M and standard deviation Q, call this S.

    * Use S to call the outcome of that state within this iteration of the simulation.

    This lets us plug in values of Q anywhere from infinite (representing all states are independent) to 0 (representing all states move in lockstep).

  35. David Gould said

    Hmmm. That is interesting. I had not thought of it in that way. Thanks.🙂

  36. Possum Comitatus said

    That’s interesting Caf, I’m playing around with what you’ve just suggested and I’m getting probabilities for the Dems getting at least 270 votes of between the high 50’s and mid 60’s depending on Q.

  37. Catrina said

    Poss at 36

    That would bring you overall probabilities in line with your state aggregates. However – I’ll need a good page long explanation of what this means in terms of real world stuff.

    But for now … Vero Possumus.
    🙂

  38. EconoMan said

    Jennifer at #22 and all others. Can I direct your attention to:

    <a href=”www.fivethirtyeight.com” title=”FiveThirtyEight”.

    Agree with Charles @ 26 and Caf @ 28.

    Lack of independence — Caf is all over it.

    Looking forward to Andrew Leigh’s work on longshot bias!

  39. Possum Comitatus said

    Caf – OK, using your suggested method, and running it with the 2nd November 2004 data (ignore my earlier response, I fluffed my spreadsheet), we need a standard deviation “Q” of 0.01 to get around 50.5% probability for the Repubs (keeping in mind that Fairs approximate method gives us 51.1% and the headline probability was 55% at the time)

    If instead of generating a mean outcome M distributed as {0,1} we generate M as {0.1,0.9}, this let’s us knock out the very extreme tails which have problems in terms of the reality of their probability (and we know that those less than 10% probabilities are problematic because we’ve never seen them win in any Intrade political contest when we certainly would have somewhere, sometime, were they correct). Once we run the November 2nd data again, we get a probability of the Republicans getting at least 270 seats of 51.5%.

    That’s lower than the headline market, but a little higher than Fairs approximation which sticks us at least in the vicinity of where we want to be.

    If we then use those same figures, but this time run the with Sundays Intrade data, we get a probability of the Democrats gaining at least 270 seats of 70.8%, which compares well against Fairs 66% and the headline market of 65% this far out from election day.

    Also, by generating a random number between 0.1 and 0.9, it takes care of a lot of what I was trying to do by giving each State a probability distribution rather than a particular probability, especially in terms of those pesky Alabama’s and DC’s.

    Any thoughts?

  40. lilprawn said

    LilPoss, would you like to be introduced to some bookies whom i have shown your site? You may be amazed how they frame the odds & what formulas they use. Obamah to win, 2 horse race $2.34. good odds

  41. Possum Comitatus said

    Lilprawn – it’s for precisely that reason (the odds setting sausage factory) that I dont touch betting market analysis of the bookie type on this site with a barge pole!

  42. caf said

    That all sounds good, and I think it’s a significant (if you’ll pardon the pun) improvement upon Fair’s method (Q=0 should give his result).

    Simply assuming that an intrade price less than 10 or greater than 90 implies a lightning-strike probability sounds like a more defensible approach to me, too. Although this does discount states which might really have a 10% probability of falling from being considered properly – maybe you could adjust the probability to 0% / 100% only for states where the market volume was small? Are there any states that actually fall into this category?

  43. Possum Comitatus said

    On the that 10% a side kill value, I cant actually find an Intrade political result where anyone less than 20% has ever won the race. I haven’t yet looked (or rather found) all of the results ever done yet, but certainly the 04 results, the 06 Senate results and scores of other bits and pieces that I can find have never produced a winner on a 20% probability just prior to the election, or even in the cases of a few weeks and months out where I can find the data.

    It looks like the 10% cut off is actually really conservative.

    There’s a few States that have really low volumes. DC has zero trades, Wyoming only 10 and the Dem contract in Idaho has 6.

  44. lilprawn said

    Possum, i may have phrased that statement incorrect. I agree with you there is too many variables in the bookie market,you cannot transfer it to a political area.I was just stating that you might like to see there programs.

  45. I had a very rough go at using Aus Federal bookie data to produce predicted seat totals before the last federal poll. Indeed it was the main basis of an 84-seat prediction that I was foolish enough to mark up to 87 closer to the day. I don’t remember exactly what system I used but I do recall that I converted everything less close than 90:10 to 100:0 and also stretched the spread for everything less close than 60:40 (I may have halved the underdog’s chances, or something like that). For those inside 60:40 I had different rules for both parties depending on who was in front, based on my observation that the market was moving towards Labor and was probably carrying some drag from older money.

    States that really have a 10% chance of falling are probably trading at closer than 80-20.

  46. Poss #16, can you give me some links to your writings on the state markets being more efficient than the national markets? (Put differently, is there a consistent formula that would combine the state odds to produce a better number than the national betting odds?) I take your point on discrepancies, but that’s not enough to show that there is unused information in the state odds.

    In the stockmarket, we typically assume that you can’t make super-normal profits trading GM stocks by knowing the prices of coal and iron ore. I guess there would be an exception in political markets if something like the insight of Caf (#28) wasn’t generally recognised.

  47. Harmless Cud Chewer said

    Possum, here’s my half baked response:

    Let’s create an S shaped function. The X axis describes individual voters, or at least suitably small equal sized subsets of them. The Y axis is the probability that a given voter will vote republican.

    The point of this is the relatively safe assumption of the baked on voter. And that the more polarized the state is, the more of those you have to start with.

    Create a parameter that you can vary such that the sum of probabilities equals the current polling/betting data. Now find the parameter that best matches each state.

    Observe that in states that are more polarised you get more baked on voters, and that occurs at the expense of the middle.

    Now, try out some plausible mechanisms that will cause the tails to grow over time as an election approaches. Here’s two:

    1: As each day passes a given subset of voters chosen at random is treated as if that voter gets to effectively vote on that day, according to his own probability. Net result is each voter thus chosen now moves to the edge and contributes to the tail. The mean moves towards the center but the deviation also shrinks.

    2: As above but you apply the principle that elections tend to push people back to pre existing prejudice. So instead of a random choice its the least undecided voters that get chosen preferentially. The deviation shrinks. The mean does what?

    You could also treat this as radioactive decay.. Or as a rolling down the hill process. Either way your deviation shrinks with time. Then you could factor in a variable that estimates the extent to which these processes are sped up closer to the election.

    Anyhow, the bottom line is you apply the monte carlo simulation not just on each state but on each suitably small subset within each state. Then sit back and watch how the deviation of the whole country behaves, over time.

    My gut feeling is, its non linear, but not by much.

    -moo

  48. Harmless Cud Chewer said

    oops.. the bottom half of that last post is half edited text that I lost below the margin of the window.. feel free to delete or otherwise ignore it below the moo. Thanks

  49. James L. said

    I think it was going to be some tedious old posting, however it compensated for my time. I’ll post a link to this web page on my web log. I am certain my site visitors will discover that very helpful.

  50. Andru said

    Here’s what I mean. Assuming past Intrade data is available, work out an accuracy rate for marginal contests. For example, for those in the 45-55 or closer range, have a look at how many Intrade got right.

  51. I have read so many articles concerning the blogger lovers
    however this post is truly a fastidious paragraph, keep
    it up.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: