Why psi research and ghost hunting will always struggle as science

I've always been interested in the possibilities of psychic phenomena and ghosts - as a teenager, the books about the Borley Rectory hauntings were amongst my favourite reading material, and it was part of the reason I wrote Extra Sensory. There are, without doubt, those amongst the community investigating these phenomena who take a genuinely scientific approach. But it has struck me recently, while reading a book by a scientist on the effects of the moon on living creatures, that in this kind of field it will always be an uphill struggle to take a scientific view.

Here's why. Let's take the example of physics researchers attempting to detect gravitational waves. These ripples in spacetime are predicted by the general theory of relativity, but have never been directly detected. A couple of experiments have recently failed to detect these waves, in one case (BICEP2) rather dramatically, after first claiming that they had been found. But here's the thing. In physics, the null hypothesis is usually just as interesting as finding what theory predicted. If you can show that it's highly likely that gravitational waves don't exist, it's arguably even better than finding them. Yes, finding them would support the current best theory - but if you can show they don't exist you have found a hole in a major theory and have cause for celebration.

However, most reseachers in psi and ghost hunting (and for that matter in medicine, though that's a different story) have a particular outcome they want to support. They want, for instance, ghosts to exist. And finding nothing is not an exciting alternative. It's an unwanted outcome. So although some investigators will be truly scientific and publish a null result, many will put the negative outcome to one side and will only ever report an apparent positive finding. Cherry picking of results becomes endemic - because the null hypothesis 'there are no ghosts' or whatever, means that what you've been doing is pretty much a waste of time, apart from highlighting some oddities of human psychology.

So I think we should have some sympathy with anyone trying to be sceptical and scientific in these kinds of fields. They face a challenge that scientists working in more core disciplines rarely have to encounter.