One of the onerous parts of being a research scientist is the pressure to publish lots of papers, which ranks second only to getting grants as a prime inducer of stress. To obtain tenure or promotion, young scientists now must publish prolifically. I remember that Mel Green, one of my mentors as a postdoc at The University of California at Davis, told me that in his day (he’s now in his early nineties, and still pushing flies at the bench), a mere one or two quality publications per year was considered a good output.
No longer. Today almost nobody stands a chance of getting tenure at a major research institution without at least four or five papers per year, good or not. Numbers are important. When we were discussing how the administration at the University of Chicago regarded publications at tenure time, my friend Russ Lande told me, “They may count ‘em, and they may weigh ‘em, but they won’t read ‘em!” Young students are pushed to submit their papers to Science or Nature, a sure recipe for disappointment since those journals are so selective. The pressure also leads to what I see as a destructive level of competition and ambition in young scientists.
My own output has been modest: I have 119 peer-reviewed papers since I started graduate school, which works out to about 3.1 per year. On the other hand, I do have two books, and pride myself on not gratuitously slapping my name on my students’ papers—one reason why some scientists with big labs can have more than 600 publications in their lifetime!
In Monday’s Guardian, David Colquhoun, a well known pharmacology professor at University College London, a fellow of the Royal Society, science critic, and author of the website Improbable Science, decries the publication frenzy that characterizes modern science. His essay, “Publish-or-perish: peer review and the corruption of science,” points out how the the publication culture has spawned a number of low-quality “peer reviewed” journals, some of such absymal standards that they’ll publish anything.
Peer review is the process that decides whether your work gets published in an academic journal. It doesn’t work very well any more, mainly as a result of the enormous number of papers that are being published (an estimated 1.3 million papers in 23,750 journals in 2006). There simply aren’t enough competent people to do the job. The overwhelming effect of the huge (and unpaid) effort that is put into reviewing papers is to maintain a status hierarchy of journals. Any paper, however bad, can now get published in a journal that claims to be peer-reviewed.
Colquhoun gives one example of an infamous paper by a group at Exeter University that studied the medical efficacy of acupuncture. Despite the finding of at best a tiny effect of the procedure (barely a placebo effect!), the authors—and the journal (The British Journal of General Practice)—trumpeted that acupuncture “resulted in improved health status and wellbeing.” Have a look at the figures Colquhoun presents to see how “improved” the status really was! (note that Colquhoun’s column is followed by a response from the study’s authors as well as the journal’s editor, standing by the paper). But clearly, there are substantial problems with the research, and the results are so unimpressive that they’re hardly an endorsement of acupuncture.
Colquhoun’s solution is self publication:
So what can be done about scientific publishing? The only service the publishers provide is to arrange for reviews and to print the journals. And for this they charge an exorbitant fee, a racket George Monbiot rightly calls “pure rentier capitalism”.
There is an alternative: publish your paper yourself on the web and open the comments. This sort of post-publication review would reduce costs enormously, and the results would be open for anyone to read without paying. It would also destroy the hegemony of half a dozen high-status journals. Everyone wants to publish in Nature, because it’s seen as a passport to promotion and funding. The Nature Publishing Group has cashed in by starting dozens of other journals with Nature in the title.
Interestingly, Colquhoun suggests Mel Green’s formula of “an average of two original papers per year” (and adds that scientists should hold only one research grant at a time, a presciption that I agree with).
An obvious problem is that junior people would be afraid, in the comments section, to criticize more established ones. Colquhoun suggests that comments can be anonymous: after all, reviewers’ comments in regular journals are not attached to names. As for worries of fraud because of the lack of peer review, Colquhoun responds:
Deer [Brian Deer, a journalist who exposed Andrew Wakefield's vaccine/autism fraut] has recently backed a proposal from the House of Commons Science and Technology select committee that an official regulator should be appointed to police science. I don’t think this could work. Is the regulator going to repeat experiments, or even check original data, to make sure all is well? In all probability, a regulator would soon degenerate into yet another box-ticking quango, and end up, like the Quality Assurance Agency, doing more harm than good. The way to improve honesty is to remove official incentives to dishonesty.
By and large, the problem does not arise from outright fraud, which is rare. It arises from official pressure to publish when you have nothing to say.
The last sentence is absolutely on the mark.
Colquhoun’s proposal has substantial merit. What I see as the biggest problem is that it forces tenure and promotion committees, as well as granting agencies. to actually read and evaluate a candidate’s papers, rather than relying on secondary indicators of “quality” like journal “impact factors,” grant dollars, or numbers of publications. The main impediment to this system is the laziness of university administrations.