I was very intrigued by a recent MIT Sloan School of Management Whitepaper, “Why Forecasts Fail. What do to Instead.” Authored by Anil Gaba, Robin Hogarth and Spyros Makridakis.
http://sloanreview.mit.edu/the-magazine/2010-winter/51214/why-forecasts-fail-what-to-do-instead/
As you probably know, MIT’s Sloan School is among the top quantitative business schools in the country. It’s the bastion for better business through data analysis. So I was surprised that they would publish a whitepaper with such a title, and I simply had to read it. I found the authors made a compelling case for why forecasts fail. They make a less compelling case for what to do about it, but more on that later.
The authors do a great job of looking back at history and highlighting forecast follies, some recent, some in the distant past. They discuss the fact that noted experts didn’t see the sub-prime mortgage collapse and the ensuing economic recession in 2008; just like experts back in the 17th century failed to predict the collapse of the Tulip market that just about brought down the entire economy of Europe. It’s a really a fascinating account of failure after failure of predictive analysis. They draw some interesting conclusions about the root cause of these failures, among them:
- The future is often a bit like the past, but never exactly the same. That means that extrapolating patterns and relationships from the past to the future can’t provide accurate predictions
- There are plenty of statistically sophisticated models that can fit – and thus “explain” – past data almost perfectly. However, these complex models don’t necessarily predict the future quite as well
- Human beings are often extremely surprised by the extent of their forecasting mistakes.
What to do about it? The authors devote little space for that, but do say “Exactly how you deal with uncertainty is for you and your team to decide. Perhaps you’ll use a hedging strategy or develop a plan B for evolving your business model.”
Really?
My own conclusion is a little different. I’ve never seen a company or organization “predict” its way to success. It’s only when management sets clear objectives and the people on the front lines commit –really commit — to delivering those results do expected outcomes become a reality. In other words, it’s not about models, it’s about people. And as long as the focus remains instead on inputs and outputs, variables and drivers, scenarios and models, then the organization itself will continue to be “surprised” by reality.
Or, as a former CEO of Pepsi use to say, “Stop predicting the future — drive it.”
