I like the HFC competition, for my own reasons. However, most forecasters don’t seem to understand the highly punative scoring system imposed upon them. They will cast a question thinking that because they understand the problem domain more than most, they should answer it. That is flat out wrong, worse, the HFC scoring system inadvertantly encourages casters to participate in more questions because the fewer in a question the more points they can earn. Yet, they don’t seem to process that is a double edge sword.
Lets look at the last few questions that were resoved (correctly, I will point out).
Here are the net points the forecasters got as a group
Paraguayan election -187
Syria strike -303
Cuba election -338
Baffin bay St Lawrence ice extent -305
Let me point out for those who think it is a dash, it is not, it is a negative sign. That is *negative* 187 points. As in, the ‘team Prescience’ would have been better off not forecasting it. At all. But there is an algorithm in the back that will heavily weight the forecasts of the high forecasters and ignore the low ones. So I will avoid discussing the entire team avoiding questions, instead lets look at picking the *right* questions.
If you are wanting to get a high score then lets look at some math. For example the Baffin Bay Ice problem. I got the high score…of 0 points. In fact 13 of us did. The low score got -106. So from a game theory perspective, we risked -106 for achieving 0. That is stupidity.
Another question, the Colorado party Paraguyan question. It started out about 80/20. Top score got only 6 points, the worst got -79. What does that tell you? That you need to be about 13x more confident before you enter a question if it is above 80 or below 20. I could make a heuristic here and say you need to take the percentage ratio 80:20=4 and multiply that by 3 (=12x) to get how much more confident you know more about this question than anybody else. That is, if their is very little variance amongst the casts, so if everyone is in the same range, like 80/20, 82/18, 78/22 then there is low variance.
If high uncertainty reigns. And there are 70/30 and 50/50 and 20/80 and other casts all over the place, then you will be able to win more points as the median estimates will be farther off from your estimate. Like with the Material conflicts in Occupied Palestine question. Almost 1/2 the scores were positive and 1/2 negative, so a favorable question to participate in if you have a small advantage with your judgment.
Conclusions – Triage the new questions carefully. Avoid getting into a question for emotional reasons, or because you know more about it than the average dude at a party.
Avoid questions where there seems to be a consensus in the answer. Unless you really, really, know a shitload more than everyone else *and* you can avoid your own hedgehog bias. You have much to lose and little to gain. On the flip side, if there is chaos amongst the casts, then you can enter the fray with a slight advantage.