Tuesday, 1 April 2014

End of the peer show

I am currently reading for review the latest book by James Lovelock, and a strange and sometimes wonderful thing it is too. I was fascinated to read fairly early on an impassioned attack on something most of us take for granted as part of the mechanism of modern science - peer review.

The idea of peer review is that before a paper is published or an experiment etc. is funded a group of other scientists (the peers in question) with appropriate knowledge assess the value of the paper/experiment and act as gatekeepers, only allowing through work they feel is worthwhile. But Lovelock points out that this process tends to support the status quo, rather than radical new thinking, and is heavily biassed in the way it is operated towards the 'throw large teams at it' approach that emerged largely in the Second World War and is strongly weighted against individual scientists working on their own, which, he suggests, is a problem.

Lovelock points out, correctly, that only individuals can come up with an idea. You can't have an idea by committee. (It's interesting, we use 'team' when we want to make the concept of throwing a group of people at a problem to sound good, and 'committee' when we want it  to sound bad.) It's not that he's against teams, but he sees them as largely responsible for the grunt work to support the ideas from the individuals. And this is fine and good, but unfortunately the peer review process has come to assume a certain way of working, and will tend to reject without consideration input from individuals who don't fall within the classic academic institution model.

One of the examples used to support this is an experience Lovelock had in the early days of the CFC/ozone hole debacle. Lovelock had applied for a small grant from the Natural Environment Research Council as part of a plan to travel on a ship to Antarctica and back, using a device he had invented that enabled him to measure the levels of CFCs in the atmosphere down to parts per trillion. The peer review included this wonderful piece of text:
Every schoolboy knows that the CFCs are among the most inert of chemicals, it would be difficult to measure their abundance in the air, or in sea water, as low as a part per million; the proposer claims to be able to measure their abundance at parts per trillion. The claim is bogus and the time of our committee should not be wasted by frivolous applications of this kind.
Lovelock made the journey and took the readings unfunded, providing the primary evidence that would result in the eventual banning of CFCs. The more common form of peer review, deciding on whether or not a paper should be published in journals, he suggests, is equally biassed against anything new/outside current received wisdom/from individual scientists working alone - it's just rare that the negative comments get seen.

There is no doubt that Lovelock is right, but unfortunately what he doesn't do (unless it comes later in the book - I am only part way through) is come up with a solution to the problem. Because the fact is that there are far more people who really will be making bogus claims and coming up with silly papers than happen to be effective individual scientists like him. But at the very least, the processes should allow for an individual who, like Lovelock, has the appropriate qualifications and experience to be published or funded just as much as those who are part of academic institutions. (Admittedly there aren't many such people, but there are still a few.) At the moment this just doesn't happen.

What's more, I do think Lovelock has a point when he suggests that the vast majority of new thinking and new inventions come from such individuals, rather than from big teams. The way we go about science now is very conservative (with a small c). Which is fine where things can continue in 'more of the same' mode, but when we need a radical idea to overthrow current thinking - or for a really impressive new invention (Lovelock stresses most of the great scientists have also been inventors, something big team science doesn't seem capable of matching) - then we are likely to be moribund if we don't find a way to support these maverick individuals. Science will, frankly, be a poor cousin of what it could be otherwise.

No comments:

Post a Comment