tag:blogger.com,1999:blog-8524070301101240472.post5787840442343735585..comments2023-09-28T08:13:11.489-07:00Comments on Only In It For The Gold: Is Peer Review Double Blind?Michael Tobishttp://www.blogger.com/profile/08229460438349093944noreply@blogger.comBlogger9125tag:blogger.com,1999:blog-8524070301101240472.post-31973112859918419742011-06-26T11:12:57.481-07:002011-06-26T11:12:57.481-07:00I believe a lot depends on the field. Some controv...I believe a lot depends on the field. Some controversies have arisen, or been manufactured, because the particular science involved was so specialized that the reviewer pool was small. You can't always talk "science" in a one-size-fits-all set of rules.Marion Delgadohttps://www.blogger.com/profile/09493068399042656060noreply@blogger.comtag:blogger.com,1999:blog-8524070301101240472.post-62044274942365313552011-06-20T14:53:02.136-07:002011-06-20T14:53:02.136-07:00A relevant post from Inside Higher Ed:
Jonathan ...A <a href="http://www.insidehighered.com/news/2011/05/31/american_economic_association_abandons_double_blind_journal_reviewing" rel="nofollow">relevant post</a> from Inside Higher Ed:<br /><br /><br />Jonathan N. Katz, co-editor-in-chief of Political Analysis and chair of humanities and social sciences at the California Institute of Technology, said in an interview that the journal would change with its next volume. He said he sees values in double blind but that "in the age of Google, double blind has become a fiction." The journal did an experiment typing in the titles of 20 recently submitted papers and was able to correctly link almost all of them to authors, who post working papers, talks given at meetings or information about their research on various websites.Jonathan Gilliganhttps://www.blogger.com/profile/09065480842704814847noreply@blogger.comtag:blogger.com,1999:blog-8524070301101240472.post-91765818308224333752011-06-20T09:36:25.058-07:002011-06-20T09:36:25.058-07:00In computer science, I've submitted papers and...In computer science, I've submitted papers and reviewed for venues that use double blind reviewing (SIGCHI, as Ted notes above, and CSCW), as well as a much larger set of venues that don't.<br /><br />In my opinion, double blind reviewing is pointless. Anyone who is sufficiently expert in a field to be valuable as a reviewer will know's who's work it is. I've been able to identify the authors of double blind papers I'm reviewing in all but one case. <br /><br />The main problem is that most research is cumulative - new papers build on your previous work, and the peer review process has to take this into account as context - you can't usefully review a paper in isolation from the work that has gone before it. The only case where I wasn't able to identify the authors was where the authors had started a new line of research that was very different from what they'd done before.Steve Easterbrookhttps://www.blogger.com/profile/11770382994758220846noreply@blogger.comtag:blogger.com,1999:blog-8524070301101240472.post-4951268718800207512011-06-20T08:26:41.511-07:002011-06-20T08:26:41.511-07:00A small P.S. to the last reference to women being ...A small P.S. to the last reference to women being disadvantaged in single-blind review: Budden's data, used by Nature as the one example, was rather questionable:<br />http://web.bgu.ac.il/NR/rdonlyres/D5BB74A3-0AB5-4237-AA1D-E0039B0547EF/66706/Buddenreplyonwhittakerfemaleauthorship3.pdf<br />The response by Budden et al is far from convincing...Marcohttps://www.blogger.com/profile/07262670367947223521noreply@blogger.comtag:blogger.com,1999:blog-8524070301101240472.post-25298224868981579652011-06-19T16:02:10.112-07:002011-06-19T16:02:10.112-07:00Peer review is not double-blind in astronomy, neit...Peer review is not double-blind in astronomy, neither for papers or observing proposals. In practice it's often relatively easy for authors or proposers to guess who might have reviewed their work. And the reviewer is at liberty to waive their anonymity, and it's not uncommon for them to do so.RWhttps://www.blogger.com/profile/07858934058244484435noreply@blogger.comtag:blogger.com,1999:blog-8524070301101240472.post-44155862180559745832011-06-18T18:41:48.120-07:002011-06-18T18:41:48.120-07:00In practice, well-published scientists frequently ...In practice, well-published scientists frequently quote their own work disproportionally, either out of necessity but perhaps sometimes out of self-promotion.<br /><br />Also, in practice, experienced reviewers will know which scientists are working on which major projects, so they will be able to guess the authors in most cases. <br /><br />This means that double-blind peer review would be ineffective in stopping the (alleged) "pal review" that McI and co. decry. <br /><br />But, perhaps, authors should be given the opportunity to submit their manuscripts anonymously.That would at least stop people (especially unaffiliated amateurs) complaining that they are being censored for who they are rather than what they write.Andy Shttps://www.blogger.com/profile/16313161977123410684noreply@blogger.comtag:blogger.com,1999:blog-8524070301101240472.post-91403254780186655402011-06-18T16:05:11.001-07:002011-06-18T16:05:11.001-07:00Some areas of computer science use double-blind re...Some areas of computer science use double-blind review. For example, the main computer-human interaction conference, SIGCHI, states that <a href="http://chi2011.org/authors/chi-anonymization-policy.html" rel="nofollow">"authors are expected to remove author and institutional identities from the title and header areas of the paper"</a>. They also note that genuine anonymization is hard to achieve and they allow authors to be as loose or vigilant about it as they wish.<br /><br />On the other hand, the main journals of the American Economic Association just announced they are dropping double-blind review. Crooked Timber had a good discussion of the <a href="http://crookedtimber.org/2011/06/05/should-the-american-economic-review-drop-double-anonymous-review/" rel="nofollow">pros and cons</a>. Some people have set up a petition contesting the change.<br /><br />As with so many other aspects of reviewing, there seem to be a wide range of conventions and assumptions in use across disciplines.<br /><br />The one generalization I'd make about reviewing is that Michaels' claim is characteristically mendacious. He begins with the assumption that climate scientists lack integrity and then finds "evidence" everywhere he looks.<br /><br />Whatever methods are used by the various climate science journals, I'm confident they work just fine, because the authors and reviewers care deeply about the quality of their work.Ted Kirkpatrickhttps://www.blogger.com/profile/04632492853825498302noreply@blogger.comtag:blogger.com,1999:blog-8524070301101240472.post-80907002887281193402011-06-18T14:08:41.668-07:002011-06-18T14:08:41.668-07:00Well, "Pat." Say no more!Well, "Pat." Say no more!Steve Bloomhttps://www.blogger.com/profile/12943109973917998380noreply@blogger.comtag:blogger.com,1999:blog-8524070301101240472.post-78567819329905610002011-06-18T13:48:08.802-07:002011-06-18T13:48:08.802-07:00To the best of my knowledge, double-blind review i...To the best of my knowledge, double-blind review is very rare in any of the natural sciences. I myself have reviewed for journals that span different areas of the natural sciences, and have never ever encountered double-blind reviews.<br /><br />In quite a few cases it would also have been hilarious. Sentences containing things such as "In our previous work", and "We have previously shown" kinda defeat the whole idea...<br /><br />To me it just sounds like the typical whining of certain people who are upset they can't get their bad papers published in good journals. In this case a whine that contains misinformation. Michaels excels in that (Hansen being a beloved target a few times).Marcohttps://www.blogger.com/profile/07262670367947223521noreply@blogger.com