Big number statistics and press release journalism

Photo by Martijn Baudoin on Unsplash

On Friday I was in the car from 7 to 9, so listened to two hours of the Today programme. I was getting increasingly irritated as the BBC's economics editor, Faisal Islam  was repeatedly played reporting on the news that the British Retail Consortium (BRC) had stated that a no-deal Brexit would result in £3.1 billion a year in tariffs on EU food. There was no analysis, just the big number and a couple of other shocking figures like a 57% tariff on the import (sic) of cheddar cheese.

One of the main rules of good reporting of statistics is that you don't just give a big number, but that you put that number into context - and when dealing with a forecast, it's essential to explain the assumptions that lie behind that forecast.

I looked on the BRC website and found the press release that contained all the information in the BBC's report. It also gave no context and no details of the assumptions. Worse, it gave no way to dig into the data behind the press release.

One useful way to give context to that £3.1 billion figure, which is pretty much meaningless as a comprehensible number, is to get an average per household. The UK has around 28 million households. So £3.1 billion a year is, on average, £110 a household per year. Perhaps more meaningfully to the way we food shop, that's around £2.10 a week. Obviously something no one wants to pay, but perhaps not as scary sounding as £3.1 billion.

The reason I'm concerned about assumptions is that we have no idea, for example, what assumptions were made about changes of purchasing patterns. So, for example, if EU cheddar cheese became really expensive, would those who buy EU cheddar cheese (boggle) switch to a cheaper British cheese? The cheddar example is a dramatic one, but almost anything we buy from the EU may have cheaper alternatives. In fact, they may not even be the same type of product. 

One of the best illustrations I've heard of the importance of not only considering direct substitutions was a discussion with an executive at a major car company - I think it was Mercedes - who said that it was missing the point to think that competitors for the cash of buyers of their £100,000+ vehicles were necessarily other car manufacturers. It was just as likely to be, say, a yacht manufacturer or a swimming pool company. Of course, we're not dealing with these here, but if, for example, beef mince had a 48% tariff imposed (another of the examples from the press release), while some would switch to British mince, others might switch to, say, Quorn.

UPDATE 26/9/20 12:21 - Many thanks to the BRC for a very rapid response to a query (at the weekend too!) They responded 'Calculations were taken from supermarkets together accounting for almost half the UK grocery market, and were then upscaled to account for the rest of the market. We did not try and calculate substitution effects (which might lower total tariff cost) or how domestic suppliers would change prices (which would likely increase), simply because it would be almost impossible to calculate.'

I take the point it would be difficult to forecast substitution effects, but they surely should have made clear that there was no attempt to do so, meaning the £3.1 billion figure was pretty much fictional.

Please note that this piece is not about Brexit and its impact. Whether or not you are happy to pay the price that Brexit will indubitably put on us, the point is about how numbers are presented in a way that best informs the public. It's about time it was done better.