On probabilistic predictions

Betting markets can be used to generate probabilities of victory. We can then use these probabilities to make predictions. The process of estimating probabilities from betting markets is distinct from using them to make a prediction, and this seems to cause some confusion.

One criticism of using betting markets has followed this chain of argument:

a. In a previous election, betting markets were used.

b. The prices implied one party had a higher probability of winning.

c. That party did not win.

d. Ergo, the betting markets and the probabilities are useless.

It is fully possible for the first three statements to be true and yet for the betting markets and probabilities to be right.

One example:

a. You have a fair coin.

b. You know for certain that the probability of heads = probability of tails = 50%.

c. You know this information perfectly but still wont know for sure whether a head or a tail will come up on the next coin toss.

It is possible that the betting markets are perfectly priced, and the implied probabilities are fully accurate. But the result is still uncertain!

Making predictions is a two stage affair. First, we try to model the range of possible outcomes and how likely each outcome is. Even if that modeling is fully accurate, it only tells us the probability of each outcome occurring. Second, we then try making inferences or predictions from this modeling. This stage is more subjective: based on the probabilities, we might feel confident in predicting a seat will go to the Coalition if the probability of Coalition victory is greater than 70%. Nevertheless, we would still expect the Coalition to lose this seat 30% of the time. The first stage may be correct (i.e., the estimated probabilities may be correct), but our predictions may be wrong.

This is why arguments of the form “betting markets performed poorly/well in this limited set of elections” are bad arguments. We expect probabilistic predictions, like those from betting markets, to be wrong some of the time, depending on the probability! For instance, if the probability of Coalition victory is 51% in a particular seat, we expect a prediction of Coalition victory to be wrong 49% of the time. This is not a weakness of betting markets, but a property of probabilistic predictions. This may be perceived as a weakness; we see recognizing uncertainty in a highly uncertain situation, such as an election campaign, as a strength. The only way to test betting markets is to look at how they performed over many elections. Fortunately, some very smart people have had a go at this. There is always more research to be done, but they found betting markets tend to outperform polls and pundits.

Next week, as the election draws near, Kaighin and I will be making our own predictions. The probabilistic nature of these predictions means we will expect some of them to be incorrect! Betting markets have their weaknesses, but even if they were perfect, we would expect this to be the case. Stay tuned for predictions next week!

Happy Beattie vs Sad Beattie

Some recent polling in Forde suggests Peter Beattie has a tough fight on his hands, with an estimated 2PP 60-40 against him. As of August 17, the Sportsbet odds give him a 36% chance of victory. Characteristically of betting markets, the market appears to have noted the polling but hasn’t swung as hard against Labor as might be naively expected for such a bad poll. This is consistent with the fact that polls and betting odds are measuring different things.

The electorate of Forde. Source: Wikipedia.

The electorate of Forde. Source: Wikipedia.

But overall, the news doesn’t look good for Beattie. He only has a 36% chance of winning the seat. But wait, for Peter Beattie, it gets worse. There are worse things in politics than not winning the seat. Let’s look at the full range of possible outcomes for Beattie:

(a) Beattie loses, ALP wins

(b) Beattie loses, Coalition wins

(c) Beattie wins, ALP wins

(d) Beattie wins, Coalition wins

Clearly, (a) and (b) are bad for Beattie. But also, even though Beattie wins in (d), he would then need to serve in opposition for at least three years, since he has pledged to serve a full term regardless of the overall election outcome. For a former Premier used to governing, that would really suck.  Let’s combine (a), (b) and (d) into one scenario: Sad Beattie. Call the other scenario, (c), Happy Beattie.  What are the implied probabilities of Sad Beattie and Happy Beattie?

Sad Beattie vs Happy Beattie

Sad Beattie vs Happy Beattie

As with our previous scenario analyses, we ran lots of simulations for two extreme cases using data from August 17. The implied probability of Happy Beattie is somewhere between 0% and 22%, for the independent and maximum covariance cases, respectively. The true value lies somewhere between these two probabilities.

For the independent seats model, the probability of Happy Beattie is zero. This is because the independent seats model gives Labor an effectively zero chance of winning the election. Under this model, even if Beattie is elected, he will automatically be part of the opposition. For the maximum covariance model, Labor victory is possible: around 22%, the same as the maximum covariance probability of Happy Beattie. So the limiting factor on Peter Beattie’s happiness right now isn’t his performance in his own seat; it appears to be the overall performance of the ALP. Note, therefore, that the probability of Happy Beattie is even less than the probability of Beattie winning his seat. With just under three weeks till election day, Sad Beattie is looking more likely than Happy Beattie.

Three myths about betting markets

Since our latest analysis appeared in the AFR and, for the first time, the SMH, there has been some chatter on Twitter about the legitimacy of using betting odds to make predictions.  We’ve written about the strengths and weaknesses of betting markets before, but wanted to address a few specific criticisms.

Betting markets certainly have their weaknesses. They don’t work if there aren’t enough people betting. They suffer from longshot bias. And some situations are just so uncertain that no prediction method, including prediction markets, can offer much additional insight (a good example of this is the election of the Pope, where only a handful of people in the world have any useful knowledge about the outcome).

But betting markets have also been shown, time and again in the literature, to outperform pundits and polls on average. They’re sometimes misunderstood. Here are a few myths we’ve heard recently:

Opening up a can of whoop-ass on myths about betting odds

Leng and I will never be able to be as hirsute as these dudes.

Myth #1: Betting markets were lousy in the 2010 election.
This is not right. The electorate-level betting data (which we use) predicted a hung parliament in 2010, with Labor expected to win 74 seats. This is a remarkable result, given the 2010 election was probably one of the toughest election outcomes to predict in Australian history.

Myth #2: Betting markets just track polls.
Betting markets aggregate information from many sources, including polls. But betting markets and polls measure different things. Polls track instantaneous voter sentiment; betting markets track the probability of victory over time. So while we’d expect a bump in the polls to be noticed by the betting markets, you’d expect the markets to be cautious about weighting them too heavily. That’s exactly what we’ve seen in this campaign. While the Labor victory probability has usually moved up and down with the polls, at no point has Labor been anywhere near the favourite in the betting odds. That’s a very important qualitative difference between the betting odds and the polls (which have recently suggested the race is neck-and-neck).

Myth #3: Betting markets predicted a landslide in election X, but the result ended up being very close.
A probability of victory implied by betting odds is not a prediction of vote proportion. A 90% probability of Labor victory is not a prediction that Labor will win 90% of the vote; the market may just be very confident Labor will win 76 seats. So, for example, while the odds for the overall election result in 2010 were putting Labor on as much as a 70% chance of forming government, this doesn’t say anything about the expected margin of victory. It simply says the markets were moderately confident Labor would form government (which it did).

Betting markets aren’t perfect. But we think they’re a really promising tool for making predictions in very uncertain situations, such as elections. We’re still a long way from election day, but we expect our predictions to become more and more accurate as it draws closer.

Momentum not with ALP

On the day of the debate (August 11), we ran the numbers again. We’ve adjusted our longshot bias correction threshold from 0.1 to 0.2 because we now have more data for Palmer United Party candidates. It really takes a confident individual to name an entire political party after themselves so hats off to Mr Palmer.

Enough adulation for the wealthy and back to the predictions. Our predictions indicate the ALP seat count went down from 65 to 63. The Coalition is predicted to now gain 85 seats, up from 84. The ‘other’ category rose from 1 to 2 predicted seats.

What does this all mean? From the beginning of our blog prediction, the ALP were a long way behind. But since the Rudd ascension, they did have a bit of forward momentum. Given things were steady from July 30 to August 6 (at 65 seats), and has now fallen to 63 seats, things are looking tough. It is difficult to nominate a defining issue that could help the ALP turn things around. Where they find another 13 seats to hold onto government looks difficult. Our coming posts are going to continue looking at more electoral level analysis that K started with the Eden-Monaro post. More soon!

Labor wipeout in Tasmania?

It looks like Labor faces a tough fight  to retain its seats in Tasmania. Before the switch to Kevin Rudd, there was some speculation that Labor may even be completely wiped out in Tasmania. The switch back to Rudd seemed to ease those fears in the Labor camp. But with the Coalition appearing to have the momentum in the polls, what do the betting markets think about a Labor wipeout in Tasmania?

Tasmania's five federal electorates. Source: Wikipedia.

Tasmania’s five federal electorates. Source: Wikipedia.

Tasmania has five federal electorates: Bass, Braddon, Denison, Franklin and Lyons. Denison is held by the independent Andrew Wilkie. The other four are held by the ALP.

We ran lots of simulations for two extreme cases (as in this post), and found the implied probability of Labor winning zero seats in Tasmania is between 5% and 25%, according to the Sportsbet odds from August 14. What does this mean? As things stand, the betting markets believe Labor probably won’t get wiped out in Tasmania. Then again, this is an extreme scenario for a state which is currently wall-to-wall Labor (excluding Denison). Yet there is still at least a 1-in-20 chance it will happen.  To give you an idea of what this means, it’s about the same probability as getting four heads when you flip a coin four times. It’s unlikely, but not really a position you’d want to find yourself in.

Will Eden-Monaro remain a bellwether seat?

The bellwether seat of Eden-Monaro has been held by the ruling party for more than 40 years. Will it remain that way this time around?

One of the unique strengths of our model is that we can estimate the probability of a scenario like this from the electorate-level betting odds. You can’t do this with the betting odds for the overall election result. And doing it with national poll results involves a lot of poor assumptions about the size of swings.

We ran our model for both the maximum-covariance and independent cases and simulated lots of elections. We then looked at each election and counted up the number of times Eden-Monaro went with the party who formed government.

The electorate of Eden-Monaro. Source: Wikipedia

The electorate of Eden-Monaro. Source: Wikipedia

Converting these to probabilities, we got probabilities of 48% and 74%  (for the independent model and maximum-covariance model, respectively) that Eden-Monaro will remain a bellwether seat at the 2013 election. In other words, if we assume no covariance, there is a 48% probability that Eden-Monaro will be won by the same party that wins the election. If we assume maximum-covariance, there is a 74% chance that Eden-Monaro will be won by the same party that wins the election.

What does this mean? Recall that the maximum-covariance and independent models are two extreme ends of a spectrum, so the true probability lies somewhere in between (although not necessarily right in the middle). So the true probability that Eden-Monaro keeps its bellwether streak going is somewhere between 48% and 74%, according to the betting markets. That means, at this stage, it’s more likely than not to stay a bellwether seat, but it’s very uncertain.

This is an interesting result since the betting odds in Eden-Monaro are currently favouring the Labor candidate Mike Kelly, despite the fact that overall the betting markets are pointing to a Labor defeat. If you made the mistake of treating probabilities inferred from betting odds as vote proportions, you would incorrectly conclude Eden-Monaro was on track to break its streak. Of course, we are still a long way from the election and things can change.

We’ll be looking at all kinds of other scenarios over the course of the campaign. Let us know if there’s one in particular you’d like us to look at.