On Fri, 26 Jan 2001, Simon Buckingham Shum wrote:
> I disagree with your terminology. Ask any academic what
> "classical peer review" is, and they will tell you about the
> "anonymous reviewer, no right of reply for author" process.
The one to ask is not "any academic" but journal editors. You will find that
most journals offer referee anonymity as an option (which can be waived by the
referee, and often is). And most competent editors will mediate, even for
several rounds, if there is a substantive basis for interaction between authors
and referees.
It is only to some (disappointed) authors that the system sometimes
looks rigid, conspiratorial and biassed: In reality, editors would like
to publish the best papers, and will do whatever it takes to make sure
they do, including following up author/referee disagreements where it
looks as if it is warranted.
If you remove the option of referee anonymity you invite more bias than
you remedy: How is one to do an honest negative review (if that is what
is called for) of someone more senior than oneself, or someone who may
be refereeing one's next grant proposal? The short answer to why
referees need the anonymous option is: for exactly the same reason we
vote anonymously in a democracy: to make sure we can exercise our
choice without fear or favor. Sure, this can be abused in the other
direction too, but that's what a competent editor is there for: the
reviewer is never anonymous to the editor.
> Our conversational model places author-reviewer interaction at the
> centre right from the start, makes this 'intellectual trace'
> accessible to wider peers, and preserves an edited, open version of
> it with the publication.
I misunderstood about referee anonymity then; I thought anonymity was
possible within the "closed" phase of author/referee interaction,
mediated by the editor, in your system: If referee anonymity is not
possible, then yours is not classical peer review. If the interactions
are, in addition, visible not only to the author/editors/referees but
to the wider community as well, then this your system is still further
removed from classical peer review, and needs empirical testing to
determine the resultant quality it delivers before it can be declared
ready for adoption.
Note that the archiving of the successive embryological phases of
preprint/revision/postprint provides the "intellectual trace" you
desire, even with classical peer review (plus self-archiving of all
those embryological stages, if the author wishes), but it leaves that,
rightly, voluntary. The same is true with the archiving of referee
reports, comments, responses, and postpublication revisions: These too
are possible in the classical system, but voluntary.
As soon as you make variants that could have substantive effects on quality
(e.g., no anonymity option, public review) mandatory, you have to test and
demonstrate what effects that has on quality; till then, everyone is best
advised to stick with tried-and-tested classical peer review.
> BBS has implemented commentary and
> author response as best it could in the paper medium, but as Stevan
> himself has argued, the net enables a more rapid tempo for scholarly
> discourse. This is simply not practical in paper.
Correct, and that is why Psycoloquy was founded in 1989.
> CogPrints restricts this benefit to "pre-journal" status
> contributions.
CogPrints is not a journal but an archive. Any embryological phase can
be self-archived therein, be it pre-refereeing preprint, revised draft,
refereed final draft, postpublication update, referee report,
commentary, or response -- as long as the author of each type of item
chooses to self-archive it.
> JIME gives it
> first class status as a mode of discourse suitable for journal
> submissions, particularly in a new interdisciplinary context. The
> context of use (disciplinary factors and user expertise may be
> variables that determine success of more radical models).
But we will need a few years to find out exactly what the quantity and
quality of the papers undergoing any new, untested system like this
will actually turn out to be. Till then, no conclusions can be drawn,
and certainly no new system should be adopted.
> We are certainly agreed that there are no proven substitutes for
> organised peer review. (Future technologies may appear to provide
> automatic analyses of quality, but you can't beat humans.) To make
> review happen, you need to engage reviewers in a social contract. To
> make this happen requires some form of coordination, up until now the
> role of the journal.
>
> However, it is not inconceivable that the net will enable
> decentralised coordination of some sort, which is able to create the
> social contract to review (or provide some other kind of incentive?),
> without any notion of a Journal or other Learned Body to establish
> the contract. Sort of Napster peer review model.
Not inconceivable, but until demonstrated, this is merely untested
speculation.
> But what's the
> payback for the reviewer? Short of paying reviewers, I don't know of
> anyone who's successfully devised other motivations yet.
Paying reviewers is yet another untested and potentially corruptive
variant on classical peer review.
--------------------------------------------------------------------
Stevan Harnad harnad_at_cogsci.soton.ac.uk
Professor of Cognitive Science harnad_at_princeton.edu
Department of Electronics and phone: +44 23-80 592-582
Computer Science fax: +44 23-80 592-865
University of Southampton
http://www.ecs.soton.ac.uk/~harnad/
Highfield, Southampton
http://www.princeton.edu/~harnad/
SO17 1BJ UNITED KINGDOM
NOTE: A complete archive of the ongoing discussion of providing free
access to the refereed journal literature online is available at the
American Scientist September Forum (98 & 99 & 00 & 01):
http://amsci-forum.amsci.org/archives/American-Scientist-Open-Access-Forum.html
You may join the list at the site above.
Discussion can be posted to:
american-scientist-open-access-forum_at_amsci.org
Received on Wed Jan 03 2001 - 19:17:43 GMT