Massachusetts Forecasting Tournament - Primary Results

We recently launched a forecasting tournament for Massachusetts elections. You can read more about it here or about forecasting tournaments in general.

It has two purposes. First, to give people a way to test their predictive skills. Second, to generate some insights into Massachusetts politics.

On the first front, you can play it yourself through the buttons above - the questions related to the general election will be open until the polls close in November.

On the second front, now that the primaries are finished, we can look at the results and see how good the forecasts were.

Track record - pretty good!

Of the six questions related to primary elections, the candidate with the highest average forecast won in five of them. The only upset was the Democratic Auditor’s Race, where State Sen. Diana DiZoglio was given a 37% chance of winning and it happened.

However, when we think of forecasts probabilistically, this type of thing would be expected. For six questions asked, with multiple candidates in each race, participants made 16 predictions. None of the average forecasts that were predicted to come true between 0% and 25% happened (0%), all of the forecasts with a greater than 75% chance came true (100%), and 66% of the forecasts given a probability between 25% and 75% came true.

In other words, the crowd forecasts were correct about which outcomes were highly unlikely, which were highly likely, and which were in the middle.

Which races had the most accurate predictions?

We use the Brier Score as a method of testing the accuracy of a prediction. It is the distance between the prediction and the outcome. A score of 0 is best (eg, giving 100% on a result that happens) and the higher the score, the worse (eg, someone who said 50% for something that happened was further away, and therefore worse, than someone who said 90%).

A higher average Brier score indicates that the crowd was less accurate on what would happen. We can see that the results fit what most observers of last night would suggest. The Democratic Auditor race was an upset - it had the highest Brier score - and the Democratic Secretary of State race was not a close contest.

What is interesting is that the Second Suffolk Senate primary was the second worst in terms of accuracy. This does not mean that the forecasters were wrong - the said that Rep. Liz Miranda had the best chance and she won - but that in a contested multicandidate primary it was hard to be clear about who would win. We see that Miranda led the forecast, but with only 38%.

Another surprise is that the Republican gubernatorial primary was the most accurate of the crowd’s forecasts. This shows that despite Chris Doughty’s support from Howie Carr, the crowd always thought that Geoff Diehl was the clear favorite.

Want to participate?

Interested in making your own predictions and seeing how you fare? The tournament is open until November. Take a chance to win $250 for the most accurate individual predictions, another $250 for the most accurate team predictions, or $100 in the bonus round if you retweet or like our announcement post.

To the questions

Primary results and forecasts

Question 2: Who will win the Lieutenant Governor Democratic Primary - Kim Driscoll

Kim Driscoll: 51% - Winner
Eric Lesser: 34%
Tami Gouveia: 14%

Question 3: Who will win the Attorney General Democratic Primary

Andrea Campbell: 63% - Winner
Shannon Liss-Riordan: 19%
Quentin Palfrey: 16%

Question 4: Who will win the Secretary of State Democratic Primary

Bill Galvin: 80% - Winner
Tanisha Sullivan: 20%

Question 5: Who will win the Auditor Democratic Primary

Chris Dempsey: 63%
Diana DiZoglio: 37% - Winner

Question 6: Who will win the Governor Republican Primary

Geoff Diehl: 83% - Winner
Chris Doughty: 17%

Question 8: Who will win the Second Suffolk District?

Liz Miranda: 38% - Winner
Nika Elugardo: 24%
Dianne Wilkerson: 16%
Miniard Culpepper: 13%