On probabilistic predictions

Betting markets can be used to generate probabilities of victory. We can then use these probabilities to make predictions. The process of estimating probabilities from betting markets is distinct from using them to make a prediction, and this seems to cause some confusion.

One criticism of using betting markets has followed this chain of argument:

a. In a previous election, betting markets were used.

b. The prices implied one party had a higher probability of winning.

c. That party did not win.

d. Ergo, the betting markets and the probabilities are useless.

It is fully possible for the first three statements to be true and yet for the betting markets and probabilities to be right.

One example:

a. You have a fair coin.

b. You know for certain that the probability of heads = probability of tails = 50%.

c. You know this information perfectly but still wont know for sure whether a head or a tail will come up on the next coin toss.

It is possible that the betting markets are perfectly priced, and the implied probabilities are fully accurate. But the result is still uncertain!

Making predictions is a two stage affair. First, we try to model the range of possible outcomes and how likely each outcome is. Even if that modeling is fully accurate, it only tells us the probability of each outcome occurring. Second, we then try making inferences or predictions from this modeling. This stage is more subjective: based on the probabilities, we might feel confident in predicting a seat will go to the Coalition if the probability of Coalition victory is greater than 70%. Nevertheless, we would still expect the Coalition to lose this seat 30% of the time. The first stage may be correct (i.e., the estimated probabilities may be correct), but our predictions may be wrong.

This is why arguments of the form “betting markets performed poorly/well in this limited set of elections” are bad arguments. We expect probabilistic predictions, like those from betting markets, to be wrong some of the time, depending on the probability! For instance, if the probability of Coalition victory is 51% in a particular seat, we expect a prediction of Coalition victory to be wrong 49% of the time. This is not a weakness of betting markets, but a property of probabilistic predictions. This may be perceived as a weakness; we see recognizing uncertainty in a highly uncertain situation, such as an election campaign, as a strength. The only way to test betting markets is to look at how they performed over many elections. Fortunately, some very smart people have had a go at this. There is always more research to be done, but they found betting markets tend to outperform polls and pundits.

Next week, as the election draws near, Kaighin and I will be making our own predictions. The probabilistic nature of these predictions means we will expect some of them to be incorrect! Betting markets have their weaknesses, but even if they were perfect, we would expect this to be the case. Stay tuned for predictions next week!

2 thoughts on “On probabilistic predictions

  1. Pingback: election lab | Our final predictions: probabilities for every seat

  2. Pingback: election lab | Our final predictions: how did they go?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s