Tuesday, 1 September 2015


There's nothing like politics to bring out lies, damned lies and statistics, and we have to be particularly careful when throwing around percentage figures when commenting on political figures, their supporters and their views. Recently published YouGov data compares the views of the supporters of different contenders in the Labour leadership election, and it is ripe with possibilities for statistical misrepresentation, if those reporting it aren't careful and have trouble with numbers.

I was brought to this observation by the blaring headline above from that intellectual powerhouse, Shortlist magazine. (Sorry, the snide factor just slipped up accidentally there.) The reason the headline made me want to dig deeper into the numbers were that, of itself, this headline doesn't tell us anything, because there's nothing to compare with. Is that a lot? Perhaps 50 per cent of the British people believe the world is run by a secret elite and the Corbynites are unusually rational. As it happens, that's not true, but the comparative figures make more interesting reading.

To give Shortlist their due, they did link to their source at YouGov, which meant those figures were available. Note, by the way, that these were the percentages who selected 'Strongly agree', of which more later:

Strongly agree to 'The world is controlled by a secretive elite'
Corbyn Supporters: 28%
Burnham Supporters: 19%
Cooper Supporters: 16%
Kendall Supporters: 7%
All GB: 13%

So the Corbynites are somewhat more paranoid and liable to believe in controlling conspiracies, which isn't particularly surprising - in fact for me it was rather more surprising that Andy Burnham and Yvette Cooper's supporters were also above average. Note, also, by the way that Shortlist's headline subtly changed what the elite were, from secretive to secret, which isn't quite the same thing.

However, for me, the more interesting statistic was another one that Shortlist picked out. They highlighted that Corbyn supporters were much more in favour of nationalisation of the railways than the country 'with 86 per cent wanting this reform, compared to 31 per cent of the public.' (Actually they conflate railways and the health service, where the percentages aren't identical, though similar, while the figures they quote aren't quite either of these.) This confused me as the Corbyn supporters who seem to dominate parts of my Facebook and Twitter feeds are always telling me the public is in favour of nationalisation of the railways.

So I took a look at the YouGov page and what it says makes it clear that Shortlist's analysis is not necessarily true. Here's the YouGov result:

STRONGLY support nationalisation of the railways:
Corbyn Supporters: 86%
Burnham Supporters: 68%
Cooper Supporters: 72%
Kendall Supporters: 46%
All GB: 34%

Leaving aside the 34/31 difference, doesn't this support Shortlist's claim? Not necessarily. Let's imagine the question and the 'all GB results' were something like this:

What is your attitude to nationalisation of the railways?

Strongly support: 34
Slightly support: 25
Neither/don't know: 21
Oppose: 10
Strongly oppose: 10

Which, as it happens, I can tell you they were, as YouGov very kindly responded within minutes to a request for more data. Given these numbers it would be madness to say that only 34 (or 31) per cent of the public wanted the reform. And it is pretty clear that there is a stronger weight in favour than against.

Without that information, Shortlist could have been correct. It could even have been the case that the question had been:

Do you STRONGLY support nationalisation of the railways:
Yes: 34
No: 60
Don't know/no particular feeling: 6

But that's the only type of situation where the conclusions being drawn are valid. It could also, of course, have been:

Do you STRONGLY support nationalisation of the railways:
Yes: 34
No: 25
Don't know/no particular feeling: 41

... which feels very different from the previous result.

This is why we need more breakdown of information when such surveys* are presented. The headlines can be totally misleading. And it would help if the responsible pollsters, which I believe YouGov to be, gave rather more information, like question and range of responses, to make sure that we know what the statistics really indicate.

* Thanks to Freddie Sayers of YouGov for also emphasising that it wasn't actually a survey, but rather data extracted from the profile information of YouGov members.

No comments:

Post a Comment