Ed Morrow had a nice sign off. We dont have one of our own so we just thought we’d steal his. This will be our last blog post until tomorrow when we either crow or eat crow. Either way, we’ve had fun doing the stats and writing stuff. Its 1am now and we’re done. It’s time to eat pizza. Thanks for reading. Good luck to all candidates and people making predictions.
K and I have been using betting markets to make our predictions. We wanted to share the probabilities associated with the Coalition, ALP and independent winning each seat. Here is a link to a pdf which lists the probabilities for each seat.
Since predictions based on individual seat probabilities are always subjective (do you predict a candidate will win when they have > 70% probability of victory? 80%? 90%?) and prone to misinterpretation, we don’t intend to make predictions about who will win in all 150 seats. As we discussed in a previous post, it is easy to jump to wrong conclusions with these odds. In particular, just because a party has the highest probability of victory for a particular seat doesn’t necessarily mean the market is confident they’ll win: for instance, if Labor has a 51% chance of victory in a seat, the market still expects them to lose roughly half the time. This is why counting up the number of seats in which Labor has > 50% probability of victory is a bad way of estimating the number of seats they are likely to win.
One big thing we’ve tried to stress on this blog is the role of uncertainty in making predictions. This post is about the uncertainty around our final seat count predictions. With our Monte Carlo simulations, we simulate 100,000 elections and the predictions are the average number of seats each party will win. But as we’ve shown before, there is a distribution of outcomes. The shape of this distribution depends on the covariance structure between seats. We don’t know what this is, so we run two models representing two extreme cases: an independent seats model and a maximum covariance model. Here is the distribution when we assume independence between seats:
What we’d like to do now is provide the 95% credible interval for the predictions. What the hell is a 95% credible interval? Most of you would have heard of a 95% confidence interval which is based off repeated sampling. We don’t have that here. A credible interval is much simpler to interpret. We had 100,000 simulations – a 95% credible interval tells us the range of outcomes which cover 95% of the simulations. The 95% credible interval for the independence model:
a. ALP: 43-53 seats
b. Coalition: 94-104 seats
So in 95% of our simulations, the ALP will win between 43-53 seats and the Coalition will win between 94-104 seats. Really should have put money on the result a few weeks back!
The distribution under the maximum covariance model is a bit more funky looking. It’s not a classical bell shape, with the extremes having the highest probability. The reason for this was discussed in a previous blog post. Unsurprisingly, the variance of this distribution is greater than under the independence model.
The 95% credible interval for the maximum covariance model
a. ALP: 32-64 seats
b. Coalition: 84-114 seats
With this maximum covariance model, the credible intervals are wider.
Note that there is zero probability of a Labor victory under both models.
Here are our final predictions
1. Coalition wins government gold with 99 seats
2. ALP a polite runners up with 48 seats
3. Independents win bronze with 3 seats
The difference between our predictions and those based on national polls is that national polls have to assume uniform swings. For example, extrapolating from the latest Roy Morgan poll, the Coalition is predicted to win 89 seats. Hopefully having data on every seat and not relying on uniform swings mean our predictions are more accurate. We’ve made a pal over the internet – Kevin Bonham. He’s been a good guy re commenting on our blog etc and he’s also got a nice blog of his own. Below is a summary of the predictions that lots of different parties have made found on his blog.
Poll Bludger: Currently has 90-57-3
Kevin Bonham: 95-52-3
Mark the Ballot: Aggregation has 88-60-2 off a 2PP of 53.1%
Pottinger Model Final Prediction:
95-54-1 UPDATED: 97-51-2
AFR Election Explorer: 89-58-3.
Simon Jackman: 50 Labor seats
ABC 7:30 Report: No idea what their model was or if there was one but they had about 56 for Labor.
Mackerras: Malcolm Mackerras 94-54-2
So while the election result looks like a foregone conclusion, the test of the predictive powers of betting markets is going to rest on predicting the number of seats each party will win. Compared to all others, our predictions for the ALP are the most dire. More to come.
It’s election day! The campaign has been only 33 days. I think K and I have managed to post on about 8 of those days so I dont think anyone’s going to give us jobs as bloggers any time soon. But the good news is that he is visiting me from MIT and we’re going to try make some predictions in the coming hours.
Like K said, it seems pretty apparent from all the polls and other analysis that the Coalition is going to get up. So our first goal of showing the effectiveness of betting markets in predicting the overall election isn’t going to be really tested. One thing we can point to is that the betting markets were definitely a lot more aggressive with a big Coalition win a lot earlier than a lot of the polls. Just two days after Rudd called the election, the betting markets pointed to the Coalition winning 84 seats to the ALP’s 65. At that time, the national level polls were calling it pretty close. The Newspoll 2 party preferred on both Aug 4 and Aug 11 was 52-48 in favor of the Coalition. So we werent complete muppets.
Anyway, we want to make amends re our poor blogging record. In the next few hours, we’re going to post
a. our final seat prediction
b. the probability distribution of the seat predictions
c. 95% confidence interval re the seat prediction
d. predictions for each seat
e. predictions re various seats we think are interesting
f. rantings and ravings as we get tired
So thank you for reading our shitty blog!
Betting markets can be used to generate probabilities of victory. We can then use these probabilities to make predictions. The process of estimating probabilities from betting markets is distinct from using them to make a prediction, and this seems to cause some confusion.
One criticism of using betting markets has followed this chain of argument:
a. In a previous election, betting markets were used.
b. The prices implied one party had a higher probability of winning.
c. That party did not win.
d. Ergo, the betting markets and the probabilities are useless.
It is fully possible for the first three statements to be true and yet for the betting markets and probabilities to be right.
a. You have a fair coin.
b. You know for certain that the probability of heads = probability of tails = 50%.
c. You know this information perfectly but still wont know for sure whether a head or a tail will come up on the next coin toss.
It is possible that the betting markets are perfectly priced, and the implied probabilities are fully accurate. But the result is still uncertain!
Making predictions is a two stage affair. First, we try to model the range of possible outcomes and how likely each outcome is. Even if that modeling is fully accurate, it only tells us the probability of each outcome occurring. Second, we then try making inferences or predictions from this modeling. This stage is more subjective: based on the probabilities, we might feel confident in predicting a seat will go to the Coalition if the probability of Coalition victory is greater than 70%. Nevertheless, we would still expect the Coalition to lose this seat 30% of the time. The first stage may be correct (i.e., the estimated probabilities may be correct), but our predictions may be wrong.
This is why arguments of the form “betting markets performed poorly/well in this limited set of elections” are bad arguments. We expect probabilistic predictions, like those from betting markets, to be wrong some of the time, depending on the probability! For instance, if the probability of Coalition victory is 51% in a particular seat, we expect a prediction of Coalition victory to be wrong 49% of the time. This is not a weakness of betting markets, but a property of probabilistic predictions. This may be perceived as a weakness; we see recognizing uncertainty in a highly uncertain situation, such as an election campaign, as a strength. The only way to test betting markets is to look at how they performed over many elections. Fortunately, some very smart people have had a go at this. There is always more research to be done, but they found betting markets tend to outperform polls and pundits.
Next week, as the election draws near, Kaighin and I will be making our own predictions. The probabilistic nature of these predictions means we will expect some of them to be incorrect! Betting markets have their weaknesses, but even if they were perfect, we would expect this to be the case. Stay tuned for predictions next week!
The latest article in the AFR using Electionlab’s analysis.
On the day of the debate (August 11), we ran the numbers again. We’ve adjusted our longshot bias correction threshold from 0.1 to 0.2 because we now have more data for Palmer United Party candidates. It really takes a confident individual to name an entire political party after themselves so hats off to Mr Palmer.
Enough adulation for the wealthy and back to the predictions. Our predictions indicate the ALP seat count went down from 65 to 63. The Coalition is predicted to now gain 85 seats, up from 84. The ‘other’ category rose from 1 to 2 predicted seats.
What does this all mean? From the beginning of our blog prediction, the ALP were a long way behind. But since the Rudd ascension, they did have a bit of forward momentum. Given things were steady from July 30 to August 6 (at 65 seats), and has now fallen to 63 seats, things are looking tough. It is difficult to nominate a defining issue that could help the ALP turn things around. Where they find another 13 seats to hold onto government looks difficult. Our coming posts are going to continue looking at more electoral level analysis that K started with the Eden-Monaro post. More soon!