Skip to main content

Forecasts aren't facts

I regularly write about the fun to be found in probability and statistics - but also the difficulties we face in understanding them, because our brains don't seem to be wired in a way that adequately deals with this kind of mathematics. However, there's one misunderstanding that stands out heads and shoulders above the rest - and this is that we find it very difficult to distinguish between forecasts and facts.

Data about past events can be (though isn't necessarily) accurate. Data about the future is often only accurate if you are lucky. I'm not saying all forecasting is worthless - it's the best of a set of bad options (certainly better than relying on Mystic Meg) - but we always need to be aware of its limitations.

Let's take a simple personal example. My mortgage will soon be coming off a fixed rate, and the bank has kindly offered me a series of options for new fixed-rate details. To be able to decide between them, whether consciously or not, I have to do some forecasting. The choice came down to a two-year deal, fixed at a rate slightly better than I have at the moment, or a five-year deal fixed at a rate that is slightly worse than my current one. With my forecasting head on, I suspect that interest rates will rise significantly over a five-year period - and so I opted for the five-year deal. However, let's be clear about what I just did: I took a bet on what interest rates will do over the next five years. This is gambling just as much as taking a punt on a horse running in the 3pm at Doncaster. The probabilities may be different - but I don't know what's going to happen. All I can do is guess, based on the data I have, and place my bet.

This educated guessing applies just as much to the majority of decisions I may make about the future (financial or otherwise) - or, for that matter, to leaked Cabinet Office forecasts of the impact of Brexit or to a company's budget. The forecast isn't a fact, and when we make a decision based on it, we are taking a (hopefully calculated) gamble.

Let's finish of with that business favourite, comparing forecasts with actuals. I have witnessed many times anguished post-mortems where companies attempt to explain why what actually happened has differed from the budget or sales forecast, or whatever the prediction happens to be. They look for changes in the market, or the environment or their staffing... but what they hardly ever do is say 'Yep, the forecast was wrong, wasn't it?' Admittedly, incorrect forecasts aren't always the reason for a deviation. It can be due to those external factors. But I suspect poor forecasting is to blame in many of the cases where an organisation spends many person-hours trying to unpick what went wrong operationally.

I'm not suggesting you should shoot the messenger, or, in this case, the forecaster. Because we are dealing in probabilities and, yes, guesses, they will go wrong. Equally the forecaster may have biasses themselves - because there are usually a huge amount of assumptions in making a forecast. Rather, when we hear a forecast, it's essential to get a better idea of the probabilities involved (if known), the accuracy of the data and, crucially, the assumptions that are being made. Only then can we make sensible use of a statistical crystal ball. If that information isn't available, don't give the forecast the time of day.

Comments

Popular posts from this blog

Is 5x3 the same as 3x5?

The Internet has gone mildly bonkers over a child in America who was marked down in a test because when asked to work out 5x3 by repeated addition he/she used 5+5+5 instead of 3+3+3+3+3. Those who support the teacher say that 5x3 means 'five lots of 3' where the complainants say that 'times' is commutative (reversible) so the distinction is meaningless as 5x3 and 3x5 are indistinguishable. It's certainly true that not all mathematical operations are commutative. I think we are all comfortable that 5-3 is not the same as 3-5.  However. This not true of multiplication (of numbers). And so if there is to be any distinction, it has to be in the use of English to interpret the 'x' sign. Unfortunately, even here there is no logical way of coming up with a definitive answer. I suspect most primary school teachers would expands 'times' as 'lots of' as mentioned above. So we get 5 x 3 as '5 lots of 3'. Unfortunately that only wor

Why I hate opera

If I'm honest, the title of this post is an exaggeration to make a point. I don't really hate opera. There are a couple of operas - notably Monteverdi's Incoranazione di Poppea and Purcell's Dido & Aeneas - that I quite like. But what I do find truly sickening is the reverence with which opera is treated, as if it were some particularly great art form. Nowhere was this more obvious than in ITV's recent gut-wrenchingly awful series Pop Star to Opera Star , where the likes of Alan Tichmarsh treated the real opera singers as if they were fragile pieces on Antiques Roadshow, and the music as if it were a gift of the gods. In my opinion - and I know not everyone agrees - opera is: Mediocre music Melodramatic plots Amateurishly hammy acting A forced and unpleasant singing style Ridiculously over-supported by public funds I won't even bother to go into any detail on the plots and the acting - this is just self-evident. But the other aspects need some ex

Which idiot came up with percentage-based gradient signs

Rant warning: the contents of this post could sound like something produced by UKIP. I wish to make it clear that I do not in any way support or endorse that political party. In fact it gives me the creeps. Once upon a time, the signs for a steep hill on British roads displayed the gradient in a simple, easy-to-understand form. If the hill went up, say, one yard for every three yards forward it said '1 in 3'. Then some bureaucrat came along and decided that it would be a good idea to state the slope as a percentage. So now the sign for (say) a 1 in 10 slope says 10% (I think). That 'I think' is because the percentage-based slope is so unnatural. There are two ways we conventionally measure slopes. Either on X/Y coordiates (as in 1 in 4) or using degrees - say at a 15° angle. We don't measure them in percentages. It's easy to visualize a 1 in 3 slope, or a 30 degree angle. Much less obvious what a 33.333 recurring percent slope is. And what's a 100% slope