Category: Peer Review

Crowdsourcing Peer Review and a Response

October 25th, 2010 in Peer Review, Scholarly Communication 0 comments

Mike O’Malley’s provocative essay, “Googling Peer Review,” raises interesting questions about the shift from an era of scarcity of information to an era of abundance of information that affect both scholars and  information professionals.

Ear­lier I argued that the era of scarcity in evi­dence was com­ing to a close, because so much pre­vi­ously hard to get mate­r­ial now exists online. Maybe it’s time for the era of scarcity in peer review to end as well. We ought to be able to rethink peer review in ways that make it more effec­tive and less “clubby.” (O’Malley)

He is not the first to suggest that peer review might be re-conceived in a digital context. Kathleen Fitzpatrick, who has been a leader in exploring “open peer review,” has posted a thoughtful response to O’Malley’s essay:

It’s gratifying to see other scholars getting interested in these wacky ideas about reinventing scholarly publishing that I’ve been pushing for over the last several years. In particular, the entry of scholars who are relatively new to the digital into these discussions confirms my sense that we’re at a tipping point of sorts, in which these new modes, while still experimental, are beginning to produce enough curiosity in mainstream academic circles that they’re no longer automatically dismissed out of hand.

All that said, I do feel the need to introduce a few words of caution into these discussions, because the business of open peer review isn’t quite as straightforward as simply throwing open the gates and letting Google do its thing…. (Fitzpatrick)

read the full response

UPDATE (10/26):

Mike O’Malley continues the conversation:

Peer review has not only served us badly: it’s cost aca­d­e­mics more and more cul­tural author­ity. The gen­eral pub­lic, hav­ing more sources avail­able online, is less will­ing to trust experts, and  sees peer review as akin to the mon­keys in Kipling’s Jun­gle Book: “We all say so, and so it must be true.”

Kath­leen Fitz­patrick made an excel­lent post on peer review. I highly rec­om­mend it as a deeper and more nuanced take than my ear­lier polem­i­cal version.

Fitz­patrick explains more about what Google does,  and how unclear Google is about how it ranks pages: since peer review is cen­tral to pro­mo­tion and tenure and career eval­u­a­tion, it’s dou­bly prob­lem­atic that Google hides its meth­ods. She adds that talk­ing about open sourc­ing peer review  it won’t pro­duce peo­ple will­ing to do the hard work. She’s right on all points.

But I still think a case can be made for ignor­ing a specif­i­cally aca­d­e­mic audi­ence for peer review, just ignor­ing it, and enter­ing aca­d­e­mic work in the gen­eral inter­net fray. Here’s why.

read more

Peer review in the age of Google?

October 19th, 2010 in Peer Review, Scholarly Communication 0 comments

Michael O’Malley’s recent essay, “Googling Peer Review,” raises some really interesting questions about the peer review as we move from an age of information scarcity to information abundance…

Who is not ambiva­lent about peer review?  On the one hand, it estab­lishes a basic, reli­able level of qual­ity in argu­ment and in evi­dence. On the other, it grinds every­thing down to a bland same­ness. Peer review assures pro­fes­sional stan­dards are met, and also enforces ortho­doxy. Anony­mous peer review pre­vents intim­i­da­tion: anony­mous peer review allows  irre­spon­si­ble, spite­ful crit­i­cism. Peer reviews can be extremely help­ful: peer reviews can crush the spirit. They take for­ever, and they also present a sig­nif­i­cant bur­den of uncom­pen­sated labor.

read more

Shakespeare Quarterly and Media Commons unveil “open review” experiement

October 14th, 2010 in Peer Review, Scholarly Communication 0 comments

Shakespeare Quarterly is one of the most recent scholarly journals to begin experimenting with new models for peer review…

For Shakespeare critics and scholars, among the most significant consequences of media change will be transformations in how we communicate with each other about our work and publish new research. In keeping with the topic of its special issue, 61:4, “Shakespeare and New Media,” Shakespeare Quarterly conducted an experiment in open peer review, for this issue, which ran from 1o March to 5 May 2010…..

The Process: After the initial editorial evaluation, authors were invited to opt into the open review process. The essays of those who opted in were posted here for public commentary and feedback by the journal’s readers. Authors have been invited to respond to this feedback in revision, before submitting their revised essays for final selection. The publication decision was based on the revised essays. (Declining the open review and opting for a traditional review would not negatively affect the selection process — it simply established a different review path.) For further details see “About” and FAQs.

Read more