I had assumed collider bias would be something to do with the kind of huge statistical analysis necessary to interpret what's going on in a piece of equipment like the Large Hadron Collider at CERN - but in reality the 'collision' in question is simply down to the way a pair of arrows point to the same location on a kind of flow diagram. What this statistical anomaly can produce, though, is the kind of result we love to hear (and scientists love to find) - results that make you go 'Huh? That's surprising.'
Examples given included that of those hospitalised with COVID-19, smokers were more likely to survive than non-smokers; amongst cardiovascular disease sufferers, obese patients live longer; success at basketball is not linked to a player's height; and PhD candidates who have high scores in the tests often used to decide if someone should start on a PhD are not more likely to succeed than those who score badly.
To understand this, Tom and Stuart ask us to imagine a study of Hollywood stars. They suggest you get to be a Hollywood star because you are either beautiful or a talented actor (or both). Assuming more actors major on one attribute than both, then, for the population sample that is 'Hollywood stars', you will find that beauty is negatively correlated with acting talent. Actors with talent, it would seem, tend not to be beautiful, and vice versa. This would be statistically true, but there is no causal link. The real danger is then to apply the same reasoning to the population at large and think that to be a great actor, a person should be ugly. But it's an artefact of the way Hollywood stars are chosen, not a true causal relationship.
In the surprising examples mentioned above, where this has occurred in real studies (often resulting in convoluted arguments as to why, say, being obese gives better survival from cardiovascular disease), it's because in each case we are looking at a sub-population - for example professional basketball players or PhD candidates, not considering people at large. So, for example, successful PhD candidates tend to be either highly intelligent or very hard workers (or both). But by only looking at successful PhD candidates, those two groups will dominate, where looking at the population at large, highly intelligent (and hence high scoring) people will be more likely to gain a PhD.
In their podcast (to be honest, one of their more meandering episodes, as this is a really difficult effect to describe), Tom and Stuart point out that this is relatively easy to spot when the result is so counter-intuitive, a reasonable flag to check if there is something wrong with the analysis. But the error can be missed if it's less stand-out.
Arguably a starting point should be that if you are studying a group that isn't typical of the population at large, then you need to be aware of this danger. This should be of particular interest, for example, to psychologists, who often do studies using university students as participants, because they are cheap and readily available. Unfortunately, though, they may well beg a collider bias population just waiting to happen. Take a listen to the podcast to find out more.
These articles will always be free - but if you'd like to support my online work, consider buying a virtual coffee:
See all Brian's online articles or subscribe to a weekly email free here
Another salient example of collider bias, shamelessly exploited by Kovid Kranks, is that the majority of people who die of Covid have been vaccinated. This is true but massively disingenuous (at best). It's exactly the result we would expect if large numbers of people have been given a good, but imperfect, treatment, versus those who did not get the treatment and were exposed to the original (highly dangerous) problem. It's equally true that the majority of people who die in car crashes were wearing their seat belt, but nobody would stop wearing seat belts on that basis — although similar arguments were actually used to argue against seat belts in the 1980s by the "but muh freedom" crowd.
ReplyDelete