Seven steps to better forecasting
Seven steps to better forecasting
Business leaders make judgement calls every day. Philip Tetlock says there are ways to improve the chances of getting them right. Below are the seven steps to better forecasting.
- Break seemingly intractable problems into tractable sub-problems.Superforecasters split problems into knowable and unknowable parts. They flush ignorance into the open. “Expose and examine your assumptions,” says Tetlock. “Dare to be wrong by making your best guess. Better to discover errors quickly than to hide them behind vague verbiage.”
- Strike the right balance between inside and outside views.Superforecasters know there is nothing new under the sun. “Uniqueness is a matter of degree,” says Tetlock. Superforecasters conduct creative searches for comparisons even for seemingly unique events, such as the stand-off between a new socialist government in Athens and Greece’s creditors. “Superforecasters are in the habit of posing the outside view question – ‘How often do things of this sort happen in situations of this sort?’.”
- Strive to distinguish as many degrees of doubt as the problem permits – but no more.While few things are either certain or impossible, “maybe” is not an informative response. Tetlock says nuance matters. “The more degrees of uncertainty you can distinguish, the better a forecaster you are likely to be,” he explains. “Translating vague-verbiage hunches into numeric probabilities feels unnatural at first, but it can be done. It just requires patience and practice. The superforecasters have shown what is possible.”
- Strike the right balance between underreacting and overreacting to evidence.Tetlock says updating your beliefs is as important to good forecasting as brushing and flossing is to good dental hygiene. It can be boring, occasionally uncomfortable, but it pays off in the long term. “Skilful updating requires teasing subtle signals from noisy news flows, all the while resisting the lure of wishful thinking.” Tetlock adds that superforecasters tend to update their beliefs in small increments, often moving from probabilities of, say, 0.4 to 0.35. These distinctions are too subtle to capture with vague verbiage such as “might” or “maybe”, but in the long run, Tetlock says these distinctions define the difference between good and great forecasters.
- Look for the errors behind your mistakes but beware of rear-view mirror hindsight biases.Rather than trying to excuse your failures, you should own them and conduct unflinching post-mortems on where you went wrong. “Remember that although the more common error is to learn too little from failure and to overlook flaws in your basic assumptions, it is also possible to learn too much,” says Tetlock. “You may have been basically on the right track but made a minor technical mistake that had big ramifications.” He suggests examining your successes, too. “Not ail successes imply that your reasoning was right,” he says. “You may have lucked out by making offsetting errors. If you keep reasoning along the same lines, you are setting yourself up for a nasty surprise.”
- Bring out the best in others and let others bring out the best in you.Master the fine art of team management. This includes understanding the arguments of the other side, precision questioning so you can help others clarify their arguments, and learning to disagree without being disagreeable. “Wise leaders know how fine the line can be between helpful suggestions and micro-managerial meddling… or between a scatterbrained group and an open-minded one,” says Tetlock.
- Focus on questions where your hard work is likely to pay off.Certain classes of outcomes, such as oil prices and currency markets, are hard to predict. Tetlock says we usually don’t know how unpredictable outcomes are until we’ve spun our wheels trying to gain analytical traction. “There are two basic errors it is possible to make here,” he says. “We could fail to try to predict the potentially predictable or we could waste our time trying to predict the unpredictable. Which error would be worse in your situation?”
‘Seven steps to better forecasting’ In The Black (March 2016): 47. Print