The usual way a scientific paper gets published is this. First, it’s submitted to a journal by one author (usually the “senior author”), and the journal’s editor then sends it to an “associate (or corresponding) editor”. That editor then chooses two or three reviewers, preferably ones who are prompt, thorough, critical in a good way, and also have no strong association with or animus against the author(s). The reviews are anonymous to the author, so that the reviewer can feel free to express his or her opinion without fear of retribution. Based on the reviews, the associate editor decides whether to accept the paper, reject it, or send it back to the authors for revision (revision is common before a paper is accepted). Eventually a final decision is made.
Crucial in this step is the anonymity of the reviewers and the ability of the associate editor to make the best choice of reviewers to evaluate the paper honestly, objectively, and critically. For that ensures the needed criticality of any published science. Some journals allow authors to suggest the names of reviewers as a favor, but that’s something I don’t like, for authors invariably will suggest people who will be friendly to the paper: people they know who will look kindly on the results and are usually colleagues or friends of the authors. When I was an associate editor for two journals (Evolution and The American Naturalist), I would routinely ignore the authors’ suggestions for reviewers, knowing that those reviews were less likely to be objective.
Now, however, according to an article in the Chronicle of Higher Education by Paul Basken, “Letting researchers choose their peer reviewers gets another shot,” one journal, the open-access microbiology journal mSphere, is changing the review process. Instead of an editor choosing reviewers, the author him/herself will choose the reviewers, get the reviews, and submit both of them to the editor. The editor will then make a decision within five business days; there will be no opportunity for revision.
The intention of the journal, which is owned by the American Society of Microbiology, is to speed up publication and also promote transparency because the names of the reviewers will be published at the end of each article.
I’m quoted at fair length in the Chronicle piece, explaining why I oppose this idea. The reasons include the following:
- Although there seem to be safeguards in place to prevent too much nepotism (e.g., postdocs of the authors can’t be reviewers), there’s still plenty of opportunity for corruption. Who, after all, would chose reviewers known to be extra critical or petulant to review one’s paper? You’re still going to send it to people who you think will accept the paper.
- Publishing the names of the reviewers at the end of the paper doesn’t really enforce much transparency. After all, will people recognize those names, and know the relationship of the author to the reviewers? I can think of several close colleagues/friends of mine whom I’ve never published with but who, I think, would be friendly to my work. The readers of the article wouldn’t know that when seeing their names.
- Now that there is early online publication, the big reduction in time to publication has already occurred, which previously was getting the article set in print and sent out in a paper journal. Reducing it by, say, two or three weeks further by waiting for editor-chosen reviewers won’t make that much difference. After all, how much science is so pressing that it has to be published within two weeks rather than five?
- If the reviewers are known to the authors from the outset, they won’t want to be super-critical, for—especially if authors are famous or powerful‚—you don’t want to anger them. This reduces your prospects for professional advancement. That’s why reviews have always been anonymous (though you can sign your review if you want to be known to the authors).
- The lack of opportunity for revision means that correcting some errors in the paper, or adding analyses that would improve it, simply won’t be done.
Here’s one of several quotes I gave in the Chronicle piece; the important bit is in bold:
“This is a really, really bad idea,” said Jerry A. Coyne, a professor of ecology and evolution at the University of Chicago who led condemnations of PNAS in 2009 after it published an article claiming caterpillars and butterflies evolved separately.
The 2009 article, by a retired academic at the University of Liverpool, came to PNAS through a channel that let members of the National Academy of Sciences choose reviewers for papers submitted by colleagues. PNAS eliminated that option shortly afterward, though it continues to permit academy members to choose reviewers for their own articles.
Such a right “undercuts one of the whole purposes of scientific review, which is objective critical scrutiny,” Mr. Coyne said. He also questioned mSphere’s anticipation of any meaningful gain in publication speed, given that online formats have largely removed the portion of the process — printing and distribution — that has historically been the major source of delays in scientific publishing.
I think Basken got the PNAS issue slightly wrong: the paper by Donald Williamson, published in PNAS, went through Academy member Lynn Margulis as the associate editor. The paper was execrable, claiming that the origin of complex life cycles in Lepidoptera originated when already-evolved butterflies that didn’t have a larval stage were mistakenly fertilized by onychophorans (velvet worms). That, claimed Williamson, produced an animal that first went through a caterpillar stage before becoming a moth or a butterfly.
The paper was uniformly ridiculed, and my big reason was that you could easily detect such hybridization genetically, by showing that butterflies had substantial genes acquired fairly recently from onycophorans, while retaining a set of unrelated Lepidopteran genes. I called it “the worst paper of the year” in 2009.
How was this travesty published? As I recall—and I may be mistaken—Margulis submitted the paper to several reviewers, but since she wanted the paper to be published because its thesis appealed to her (she often had weird ideas), she simply did not submit the bad reviews to the journal editor. The editor then got the impression that the paper was uniformly approved. (And that editor must not have known enough biology to see how fatuous Williamson’s thesis was.)
This exemplifies the dangers of choosing (or culling) reviews based on their likelihood of liking your paper. And it’s why mSphere’s idea is so bad. It removes the criticality that is the backbone of science, producing a bunch of spineless reviews that will flop amiably into the journal.