A really, really bad idea about reviewing scientific papers for journals

The usual way a scientific paper gets published is this. First, it’s submitted to a journal by one author (usually the “senior author”), and the journal’s editor then sends it to an “associate (or corresponding) editor”. That editor then chooses two or three reviewers, preferably ones who are prompt, thorough, critical in a good way, and also have no strong association with or animus against the author(s). The reviews are anonymous to the author, so that the reviewer can feel free to express his or her opinion without fear of retribution. Based on the reviews, the associate editor decides whether to accept the paper, reject it, or send it back to the authors for revision (revision is common before a paper is accepted). Eventually a final decision is made.

Crucial in this step is the anonymity of the reviewers and the ability of the associate editor to make the best choice of reviewers to evaluate the paper honestly, objectively, and critically. For that ensures the needed criticality of any published science.  Some journals allow authors to suggest the names of reviewers as a favor, but that’s something I don’t like, for authors invariably will suggest people who will be friendly to the paper: people they know who will look kindly on the results and are usually colleagues or friends of the authors. When I was an associate editor for two journals (Evolution and The American Naturalist), I would routinely ignore the authors’ suggestions for reviewers, knowing that those reviews were less likely to be objective.

Now, however, according to an article in the Chronicle of Higher Education by Paul Basken, “Letting researchers choose their peer reviewers gets another shot,” one journal, the open-access microbiology journal mSphere, is changing the review process. Instead of an editor choosing reviewers, the author him/herself will choose the reviewers, get the reviews, and submit both of them to the editor. The editor will then make a decision within five business days; there will be no opportunity for revision.

The intention of the journal, which is owned by the American Society of Microbiology, is to speed up publication and also promote transparency because the names of the reviewers will be published at the end of each article.

I’m quoted at fair length in the Chronicle piece, explaining why I oppose this idea. The reasons include the following:

  • Although there seem to be safeguards in place to prevent too much nepotism (e.g., postdocs of the authors can’t be reviewers), there’s still plenty of opportunity for corruption. Who, after all, would chose reviewers known to be extra critical or petulant to review one’s paper? You’re still going to send it to people who you think will accept the paper.
  • Publishing the names of the reviewers at the end of the paper doesn’t really enforce much transparency. After all, will people recognize those names, and know the relationship of the author to the reviewers? I can think of several close colleagues/friends of mine whom I’ve never published with but who, I think, would be friendly to my work. The readers of the article wouldn’t know that when seeing their names.
  • Now that there is early online publication, the big reduction in time to publication has already occurred, which previously was getting the article set in print and sent out in a paper journal. Reducing it by, say, two or three weeks further by waiting for editor-chosen reviewers won’t make that much difference. After all, how much science is so pressing that it has to be published within two weeks rather than five?
  • If the reviewers are known to the authors from the outset, they won’t want to be super-critical, for—especially if authors are famous or powerful‚—you don’t want to anger them. This reduces your prospects for professional advancement. That’s why reviews have always been anonymous (though you can sign your review if you want to be known to the authors).
  • The lack of opportunity for revision means that correcting some errors in the paper, or adding analyses that would improve it, simply won’t be done.

Here’s one of several quotes I gave in the Chronicle piece; the important bit is in bold:

“This is a really, really bad idea,” said Jerry A. Coyne, a professor of ecology and evolution at the University of Chicago who led condemnations of PNAS in 2009 after it published an article claiming caterpillars and butterflies evolved separately.

The 2009 article, by a retired academic at the University of Liverpool, came to PNAS through a channel that let members of the National Academy of Sciences choose reviewers for papers submitted by colleagues. PNAS eliminated that option shortly afterward, though it continues to permit academy members to choose reviewers for their own articles.

Such a right “undercuts one of the whole purposes of scientific review, which is objective critical scrutiny,” Mr. Coyne said. He also questioned mSphere’s anticipation of any meaningful gain in publication speed, given that online formats have largely removed the portion of the process — printing and distribution — that has historically been the major source of delays in scientific publishing.

I think Basken got the PNAS issue slightly wrong: the paper by Donald Williamson, published in PNAS, went through Academy member Lynn Margulis as the associate editor. The paper was execrable, claiming that the origin of complex life cycles in Lepidoptera originated when already-evolved butterflies that didn’t have a larval stage were mistakenly fertilized by onychophorans (velvet worms). That, claimed Williamson, produced an animal that first went through a caterpillar stage before becoming a moth or a butterfly.

The paper was uniformly ridiculed, and my big reason was that you could easily detect such hybridization genetically, by showing that butterflies had substantial genes acquired fairly recently from onycophorans, while retaining a set of unrelated Lepidopteran genes. I called it “the worst paper of the year” in 2009.

How was this travesty published? As I recall—and I may be mistaken—Margulis submitted the paper to several reviewers, but since she wanted the paper to be published because its thesis appealed to her (she often had weird ideas), she simply did not submit the bad reviews to the journal editor. The editor then got the impression that the paper was uniformly approved. (And that editor must not have known enough biology to see how fatuous Williamson’s thesis was.)

This exemplifies the dangers of choosing (or culling) reviews based on their likelihood of liking your paper. And it’s why mSphere’s idea is so bad. It removes the criticality that is the backbone of science, producing a bunch of spineless reviews that will flop amiably into the journal.

43 Comments

  1. Posted December 13, 2016 at 1:17 pm | Permalink

    Perhaps because I’m a masochist (or just an idiot), I once intentionally suggested a reviewer that I knew would be hostile to my work. The paper eventually got published, but only after major revision.

    • Diane G.
      Posted December 13, 2016 at 4:40 pm | Permalink

      Or you were just onto Jerry’s MO. 😉

  2. eric
    Posted December 13, 2016 at 1:31 pm | Permalink

    It strikes me that the real problem here is editors knowing relevant reviewers, when a journal may have a fairly wide range of subject matter (beyond the field of the editor). Its hard to do.

    Well a few potential solutions spring to mind. The least radical is something JAC has previously mentioned: pay the editors, and stipulate in the job description that they’re expected to do an hour or few of work finding reviewers per paper. Then they’ll be more likely and more amenable to doing it.

    My second thought is that when the journal gets started, and maybe once a year after that, you develop a big list of potential reviewers by area of expertise, based on past review work, editor suggestions, etc. Again, pay the experts you bring in to advice on this big list, and you’ll probably get a higher quality of participation.

    My third is a bit more radical, which is that journals allow scientists to submit their CVs to the journals as potential reviewers. Have people who want to be reviewers send in their CVs in some specific (hint: easy to database and search) format. Then when the editors need to look for reviewers, they can do a keyword search on this database, pull up 5-10 CVs, and pick the ones they think are most appropriate. This would probably also provide younger scientists with a more transparent and less “old boys network” method of getting into the business of being journal reviewers and editors.

    • Posted December 13, 2016 at 1:57 pm | Permalink

      The first problem is solved by having an editorial board, an appropriate number of editors with a sufficient range of expertise.

      The second idea, I’m pretty sure is already implemented by major journals.

      On the third one, well at least in my field, the journals already seem to get their clutches on to potential reviewers when they are relatively junior (e.g. junior postdocs). They just go by who is publishing in the area, and after all the journals are always in need of reviewers.

      As for payment, well, it would push up the price of publishing, of course. I’m not convinced it would produce better reviews,

  3. Teresa Carson
    Posted December 13, 2016 at 1:35 pm | Permalink

    That’s horrifying! Can you imagine George Williams or Bill Salt letting the author select the reviewers? I have to say that many people have provided the names of individuals that they did not want to review their papers because of previous private or public skirmishes. Not allowing for revisions is also astonishing to me. I know that folks in the “Nobel Prize business,” as someone once called it, are anxious to publish first and have their papers processed quickly, but this seems to defeat the whole concept. Why have the papers reviewed at all?

    • Posted December 13, 2016 at 1:43 pm | Permalink

      Yes, and if authors didn’t want someone to review a paper, and had good reasons, I would respect their wishes. You don’t want any personal antagonisms in the process.

      • Posted December 13, 2016 at 1:51 pm | Permalink

        This comment partially answered my question but not totally. Referring to associate editors you said “also have no strong association with or animus against the author(s)”. Would the editor be wholly dependent on you listing these folks? Are you expected to list John Doe hates me and Jim Bob is my post doc? Sounds as if you can list people who you would not want to review but what are the disclosure requirements?

      • Posted December 13, 2016 at 1:59 pm | Permalink

        Possible typo: “Crucial in this step is the anonymity of the authors …” Reviewers?

        • Posted December 13, 2016 at 2:29 pm | Permalink

          Yes, thank you. Fixed.

        • eric
          Posted December 13, 2016 at 2:29 pm | Permalink

          IIRC journals often strive for both: have the authors be anonymous to the reviewers, and have the reviewers be anonymous to the authors. Though in my very limited and now somewhat out of date experience, neither is achieved with great success.

          The problem is that in many sub-fields, the major researchers are familiar with each others’ work. You know which labs are working on which problems. You talk to each other at conferences. You read each others’ papers. You might even have been past collaborators. So when a reviewer gets a paper, even without a name or any other signifier, they can often say something like “that’s Bob’s work, his post-doc presented the early results of this work last year at the APS meeting.” And the author, in turn, can often smell out specific reviewers. “That critique must mean this reviewer is Bob, he always complains about statistical methods in his reviews, that’s his pet peeve.” Language is often a clue too – its pretty easy for a native English speaker to spot a paper written by a nonfluent, non-native English speaker; they make different types of grammar and spelling mistakes than the native speakers.

          This problem also sometimes leads to a bit of a kabuki dance when you next meet the person. You know they negatively reviewed your paper. They know you know. You know they know you know. 🙂 But neither of you may want to admit it, because that would break ‘science’s anonymity.’ Those conversations were always somewhat amusing.

          Still, anonymity is a good goal to have. Its the standard we should aspire to meet. If its not always achieved, well, its still good to try.

          • Simon Hayward
            Posted December 13, 2016 at 3:11 pm | Permalink

            Agree, the first time I reviewed something blind to who the authors were, it started off feeling a little strange, but I thought I’d cottoned onto the group within the first three sentences of the abstract – confirmed later because, of course, they reference their own work in the text.

            The lack of revision seems very short sighted, many of my own papers have been improved in responding to reviewer’s comments, and I hope, I’ve had a positive effect on the work of others. It often helps to have someone ask the awkward question!

          • barn owl
            Posted December 13, 2016 at 8:26 pm | Permalink

            I’ve reviewed grant proposals for which the investigators were anonymous. The proposals were for an award mechanism that encouraged innovative, high risk projects, within a larger research program that also funded more traditional, non-anonymous R01-type proposals. In some cases you could figure out who the investigators were, but in many cases they were likely new to the field, or perhaps were established investigators trying innovative and out-of-lab-character ideas.

          • gravelinspector-Aidan
            Posted December 15, 2016 at 7:54 pm | Permalink

            its pretty easy for a native English speaker to spot a paper written by a nonfluent, non-native English speaker; they make different types of grammar and spelling mistakes than the native speakers.

            Not only can you spot the bits written by a non-native speaker, you can also have a pretty good idea of what the native language (or language group) of the non-native speaker is. French people make different types of error in English to Dutch or German speakers.

  4. Craw
    Posted December 13, 2016 at 2:48 pm | Permalink

    This is like letting an accountant have his wife do the audit.

    • Randall Schenck
      Posted December 13, 2016 at 3:04 pm | Permalink

      Yes, like all those internal investigations that agencies do on themselves. Surprise, they never find anything. Maybe like a president-elect who never releases his income tax or, when he ever does put his business in a blind trust, it is run by his kids. Blind like a hawk.

  5. Kevin
    Posted December 13, 2016 at 3:58 pm | Permalink

    Publishing reviewing process should be anonymous.

    Proposal review committees should be formed with people who bloody know something about what you are proposing. Too many times people without the will or knowledge to understand a proposal condemn it.

  6. Posted December 13, 2016 at 9:19 pm | Permalink

    Wow, what an unbelievably bad idea. Jerry is absolutely right in his criticisms.

    However, it may interest some to note that peer review doesn’t quite work the same way in pure mathematics. In theory, the process is the same, but in practice, the reviewers are almost always known to the author. This is due to the fact that most papers are on topics so specialized that there are literally only a handful of people on the planet that are qualified to perform the review. Because of this, recommending reviewers is almost always required (there would be no way for editors to know every mathematicians sub-specialities and pet problems).

    Unsurprisingly, pure mathematics produces some of the worst academic papers out there, both in terms of communicative quality and actual content. Nepotism is rife. As is stonewalling.

    I’ll relate a personal story from my grad school days on this point. A major part of my dissertation (about 1/3) consisted of fixing a major result published several years prior. The theorem as stated was correct of course, but the mathematics claiming to prove it were woefully incorrect in several critical places. This was not just a matter of “missing a minus sign”, this was a complete failure to recognize major structural pieces of the objects of study. The original paper was published in a top journal. It is clear to me that no one actually reviewed the work; they simply saw that the author was a student of an extremely well-known mathematician and figured, “If it’s good enough for Dr. X, then it’s good enough for me.” In mathematics, it is verboten to issue corrections or retractions, and it is also considered career suicide to call out mistakes in published work. Not surprisingly, the general literature is awful, and positively riddled with ridiculous twists and holes as this one.

    Part 2: another 1/3 of my dissertation was composed of generalizing this previous result to its n-dimensional setting. This work – which I am quite proud of – has never appeared in print (outside of my dissertation) due to stonewalling by competing mathematicians. There are probably 6 or 7 practicing mathematicians that could reasonably peer-review the work. 3 of these were the most recent ones to work on the problem before me and my supervisor. They seemed to develop a sense of ownership over the problem (not uncommon is most fields I think). As one of these mathematicians is quite senior in the field, he was able to effectively block all attempts at publication by chatting up the remaining 3 or 4 unattached reviewers. My supervisor and I have tried to publish the work repeatedly, and the reviews always come back the same: “This is excellent work, but you do not address the question in Dr. Y’s paper, therefore we reject the paper categorically.” For context, the “question of Dr. Y” referred to here is not the focus of the research; it is a related question that would constitute a separate paper in its own right. Yet the politics are clear.

    While I still dabble in pure mathematics, I am more than happy to have transitioned permanently to statistics and ecological work.

    • Posted December 13, 2016 at 9:27 pm | Permalink

      A point of clarification: this is not just paranoia talking. After the start of this whole fiasco, we of course started to request that Dr. Y and his colleagues not act as reviewers for the paper due to potential conflicts. Whenever we would resubmit the paper however, Dr. Y would contact us demanding that we scrap the work or add his name to it. Keep in mind that there should be no way for Dr. Y to know that we had submitted the work unless the editors and/or reviewers contacted Dr. Y about said work. This was invariably done (more than once).

      • Diane G.
        Posted December 13, 2016 at 9:42 pm | Permalink

        Wow. That’s sort of the last thing I’d have expected from mathematicians! Thinking, everything must be so cut-and-dried…Guess not!

        PS: just had a look at your Glaucous-winged gallery–wow, again! 🙂

        • Posted December 13, 2016 at 10:40 pm | Permalink

          Lol, you would be surprised (and appalled)! Many mathematicians are only happy if you satisfy their egos, and the traditions of the field are such that you must kowtow to the party line if you want to make it academically.

          I’m glad that someone appreciates my GWGull photos! 🙂 Do you mean the ones Jerry has posted on here, or all of the ones on Flickr or Instagram?

          • Diane G.
            Posted December 14, 2016 at 11:43 pm | Permalink

            Well, nice to know mathematicians are just regular academics, then. 😀

            All (at least most!) of the ones on Flickr! Whew! Really interesting portfolio. And now I remember joking about “Olympic” Gulls with you previously. 😉

            • Posted December 15, 2016 at 1:16 am | Permalink

              You know, if you search Flickr for the “Glaucous-winged Gull” tag, more than a 1/3 of the 11,000+ images are mine. Someday, I hope to make it more than 2/3! =D

              • Diane G.
                Posted December 16, 2016 at 12:30 am | Permalink

                Congrats! 🙂

      • Dominic
        Posted December 14, 2016 at 7:57 am | Permalink

        A fascinating and salutary tale. Why not just publish on a website?

        • C Ruffington
          Posted December 14, 2016 at 3:46 pm | Permalink

          Hi biologists. Please don’t read Ed’s comments and start forming bad opinions of mathematicians prematurely. I am not an analyst, so I cannot speak to the correctness of Ed’s paper and preprint, but I do know that:

          1) these Kakeya blah blah problems are notoriously difficult, and have frustrated people smarter than the lot of us combined (e.g. Fefferman)

          2) when a graduate student solves or makes significant progress on problems like this they have more job offers than they know what to do with.

          To be perfectly honest, and I mean this in the nicest possibly way, Ed is probably out of his depth.

          • Posted December 14, 2016 at 8:17 pm | Permalink

            Out of my depth? You can’t just couch a statement with “no offence, but” and expect whatever follows to be free of ridicule.

            I offered an anecdote and a personal opinion on the world of mathematics. I am presuming that you are a mathematician as well, although I cannot find a C. Ruffington through a quick Google search, and if you are, I have to say that your comment does an excellent job of proving my point about egos in mathematics.

            You clearly doubt my work (implying I would have job offers after solving such a problem), but you also clearly know nothing of the subject matter. There are many Kakeya-type (blah blah) problems out there. I have never worked on the true Kakeya problem; I worked on a related problem dealing with directional maximal operators and the existence of certain sparse sets. Plenty of progress has been made over the years on such problems (see Alfonseca, Vargas, Soria, Bateman, to name a few, all non-Fefferman types).

            It’s quite an accusation to cast doubt on my work, especially when you do not take the time to understand it at all. I am happy to enlighten you if you like.

            Also, remember that doubting the veracity of my work directly implies that my supervisor didn’t know what she was doing. If you knew anything about my graduate supervisor’s work, or about the type of mathematician she is, you would quickly realize the foolishness of your uninformed implications.

        • Posted December 14, 2016 at 8:18 pm | Permalink

          We may eventually publish in a monograph form, although there are several other options we will likely try first. The problem with simply publishing on a website is that it doesn’t really constitute as being “part of the literature”.

      • Posted December 15, 2016 at 4:16 pm | Permalink

        I am sorry for the fate of your manuscript, and I hope that its time will eventually come.

    • gravelinspector-Aidan
      Posted December 15, 2016 at 8:02 pm | Permalink

      In mathematics, it is verboten to issue corrections or retractions, and it is also considered career suicide to call out mistakes in published work

      Forgive me if I mis-remember, but didn’t the Wiles proof on Fermat’s Last get precisely that treatment after a couple of yeas, with a hole being found, published, chewed for about 2 years, hen fixed. Or was that the result of the unusual degree of media attention on that particular result?

      • Posted December 16, 2016 at 2:09 pm | Permalink

        Yes, you are remembering correctly. That result was so once-in-a-lifetime, it was always going to be given special treatment. The names behind it were so senior as well, that they can always play by their own rules (not that they haven’t perhaps earned that right – these are absolutely brilliant mathematicians we’re talking about).

        If you examine the regular literature though, the number of published mistakes is disturbing. And since it is taboo to point them out, they just get further entrenched in the literature as other people cite the mistaken work (even when those people see the mistakes and know how to fix them). There is a long tradition of “if you’re not smart enough to fix the mistakes for yourself, then you shouldn’t be working in that area.” But since no one really considers themselves experts in the most famous, intractable problems (like Fermat’s Last Theorem, or the Riemann Hypothesis), this culture of conceit doesn’t apply there.

        • gravelinspector-Aidan
          Posted December 17, 2016 at 1:31 pm | Permalink

          Hmmm, sounds like mathematics needs a visit from the …. someone. Spanish Inquisition? Cochrane Consortium? The ghost of Nicolas Bourbaki?

  7. nicky
    Posted December 14, 2016 at 3:24 am | Permalink

    Caterpillars as symbiosed-in velvet worms? Brilliant! Without any genetic research? That must be worthy of at least an IgNobel prize. Btw, was it published in late March/early April?
    Makes one think of Geoffrey de Saint Hilaire’s vertebrates as inverted and inside-out arthropods (or vice versa), although there appears to be something in the latter after all, IIRC.

    Yes Jerry, I agree 100% it is an extremely bad idea to choose your own reviewers. It has a whiff of incest too.

  8. Mathieu Siol
    Posted December 14, 2016 at 4:39 am | Permalink

    I also think it’s a very stupid idea. But how about the flip side of the coin? Why do reviewers should have access to the names of the authors they are reviewing? It always struck me as unfair and likely to lead to a biased reviewing process as well. Imagine you receive a paper by a renowned big gun in your field of expertise. If you disagree, wouldn’t your first reaction be that you probably misunderstood?
    Approaching the “double blind” gold standard of science in peer-reviewing seems like a good idea to me, yet it is rarely discussed.
    Any thoughts anyone?

    • Dominic
      Posted December 14, 2016 at 7:52 am | Permalink

      I suppose the size of your field is significant. It may be that you know all the potential reviewers.

      • Mathieu Siol
        Posted December 14, 2016 at 9:58 am | Permalink

        It may be that your field is small enough, that you know all the guys in it, but I still don’t see how it would hurt to mask the names of the authors being reviewed. Worst case scenario, you would likely guess who you are reviewing which would basically amount to the current situation where you actually know it. Still, in field large enough, I guess it would remove some possible bias.

  9. jaxkayaker
    Posted December 14, 2016 at 7:47 am | Permalink

    Re: Authors choosing reviewers: isn’t there enough garbage in the primary literature? Recommending reviewers that the editor can choose or ignore is fine, but authors getting to choose their own is madness.

    Double blind peer review of articles would be an improvement, even if not a panacea.

    Jerry’s heuristic would not have served him well in every case. For publishing my master’s, I suggested top names in the specialty, none of whom were people I knew personally. Come to think of it, that didn’t work out well for me either.

  10. Dominic
    Posted December 14, 2016 at 7:51 am | Permalink

    A friend who used to edit a journal said that one frustrated reviewer wrote ‘What are these guys on?!”… they removed that remark of course! But she said that most reviewers were helpful.

  11. Posted December 14, 2016 at 8:34 am | Permalink

    Following is a blog post from Discover magazine on this topic:

    http://blogs.discovermagazine.com/neuroskeptic/2016/12/13/why-pick-your-own-peer-review-is-a-bad-idea/#more-8405

    On another, but maybe somewhat similar topic (I know that Elsevier has been discussed here before), the January/February 2017 issue of Discover magazine has “100 Top Stories of 2016”. On page 34, there is an article titled “Battle for Access”. The first paragraph is: “If governments fund scientific research, should for-profit publishers be able to copyright the findings? In 2015, Elsevier, a major publisher of academic journals, filed a lawsuit against Sci-Hub, a website started in 2011 that now houses roughly 60 million pirated articles for free download – a violation of copyright law.”

    My memory of previous discussions here had to do with what Elsevier charges for its’ publications. I don’t remember the copyright issue if it was discussed. But, what do you think? Why are we paying for copyrighted Elsevier publications of publicly (government) financed research? Opinions were given by Fred Fenter (open-access publisher), Peter Suber (director of Harvard Open Access Project), Alicia Wise (Elsevier)and Alexandra Elbakyan (Sci-Hub’s founder).

    • gravelinspector-Aidan
      Posted December 15, 2016 at 8:08 pm | Permalink

      In 2015, Elsevier, a major publisher of academic journals, filed a lawsuit against Sci-Hub, a website started in 2011 that now houses roughly 60 million pirated articles for free download – a violation of copyright law.”

      As I understand it, that’s precisely what SciHub don’t do, and that’s why they don’t do it. They have a large database of login credentials (obtained how? I’m not sure) and they probably have some good proxying and shuffling of the credentials to make blocking harder.

  12. Posted December 14, 2016 at 11:53 am | Permalink

    I agree this is a bad idea.

    What I find interesting is that different fields seem to have very very different standards as to how *long* the whole process takes. A 2nd or 3rd tier philosophy journal like Dialogue seems to sometimes publish papers 2 or so years after initial receipt, and only in quarterly fashion. Yet, high profile natural science journals like Science publish almost weekly with turn around the same. Is this purely numbers in the fields? For what it is worth, there seem to be usually about 1/3 as many philosophers as the smaller natural science departments, but …

    • gravelinspector-Aidan
      Posted December 15, 2016 at 8:17 pm | Permalink

      If you look at the publishing details in Arxiv, astrophysics journals take a couple of months between first submission (and almost simultaneous upload to Arxiv) and publication.
      I RTFDSM (Daily Submissions eMail) and there are on the order of 3 novel papers to each “Prof X replaced paper Y with version Z” notice, with the final notice being the published version.
      Two interesting papers on my desktop – considering for submitting for news articles (trying to get into science journalism before the redundancy runs out) : “Bayesian evidence for the prevalence of waterworlds” and “Rings beyond the giant planets” ; but there’ll be another crop to look at in the morning.

  13. Ron DeBry
    Posted December 14, 2016 at 2:29 pm | Permalink

    Basing an entire review system around having the authors procure the reviews is certainly a bad idea.

    That said, in my time editing a journal (in the general field of evolutionary biology) there were many times that I sent papers to a reviewer who was suggested by the authors. It takes a lot of work to find reviewers and a little nudge in the right direction is appreciated.

    There were also quite a number of times when I sent manuscripts to people where the authors specifically requested we *not* send them (I would never do so if the authors’ stated reason was that they were worried about getting scooped or otherwise having their work prematurely revealed to a close competitor). I always kept the authors’ suggestion that a reviewer might be hostile in mind when evaluating the review. I found that the supposedly hostile reviewers nearly always returned fair-minded and balanced reviews. It is only anecdata, because I never kept count, but I was impressed by how often the supposedly hostile reviewers returned positive reviews, often more positive than another reviewer that I had chosen on my own for the same manuscript.

    A related issue worth keeping an eye on is that a number of people over the years have tried to monetize the review process – setting up a service where authors submit a manuscript to the reviewing service rather than a journal, and then submit the manuscript plus reviews to a journal and possibly even to additional journals if the first rejected the manuscript. In various iterations, either the author or the journal paid for the process. My main worry there is that the journal has given up control over vetting the expertise of the reviewers.

  14. Posted December 15, 2016 at 4:30 pm | Permalink

    There is a journal in my country with rules similar to the former rules of PNAS. Basically, you have to submit your manuscript to the Editorial Office of the journal together with a favorable review by a renowned scientist, whom you must solicit yourself. I hate this review process. Approaching a senior colleague with a request to review my manuscript for “the Proceedings” is, to me, among the top 10 unwanted missions.

    To cap it all, I once had my manuscript rejected by the reviewer. So, when people say that no article has ever been rejected by “the Proceedings”, I object that there has been at least one.

    I have also submitted to some international journals who want from the author to suggest reviewers, and I have never succeeded there. Maybe a coincidence.


%d bloggers like this: