Pertinent Prior Amsci Topic Thread:
"Self-Selected Vetting vs. Peer Review: Supplement or
Substitute?" (began November 2002)
http://www.ecs.soton.ac.uk/~harnad/Hypermail/Amsci/subject.html#2341
Although clinical medical research is not a representative model for
research in general (and has led to the wrong-headed idea that the
only research that needs to be Open Access (OA) is what concerns our
tax-payers' health) it is an instructive model for giving us a gut sense
of the importance, even the urgency, of OA.
It is in this light that it is a good idea to ask ourselves, when weighing the
adequacy or even the sense of yet another "reform" proposal, to try it out first
on articles that concern our health:
We have many times heard the hypothesis that post-hoc vetting by self-selected
commentators on the web can serve as a substitute for the pre-evaluation and
certification of specialized work by qualified specialists that we call
"peer-review." The question to ask yourself is whether, if you need to have a
loved one treated today, you would like the treatment to be on the basis of
(1) unrefereed preprints posted on the web, and possibly/eventually evaluated by
possibly-qualified experts -- or you would rather have them treated on the basis of
(2) refereed articles that have already been evaluated by qualified experts, and
certified by the quality-standards and track-record of the journal that is
answerable for having published them?
I take it that when it comes to loved ones who need treatment today,
there is no contest in the mind of anyone who reflects seriously on this
question that (2) is the right answer, with (1) at most only a welcome
supplement to, but certainly no substitute for (1).
That was the rather shrill family-health version of the question, but it does not
require much imagination to see that the answer is the same if we ask it from the
viewpoint of a researcher: If you need to decide what finding to invest your limited
earthly research time and resources into trying to build upon, is it (1) unrefereed
findings posted on the web, possibly/eventually evaluated by possibly-qualified
experts, or is it (2) refereed findings already evaluated by qualified experts and
certified by the known quality standards of an established journal?
Once again, (1) seems welcome as a supplement to (2), but certainly not
as a substitute for it.
I leave it as a lemma for the reader to repeat this exercise, but this time with
respect to what papers the overloaded scholar or scientist can afford to spend his
finite time reading-time reading, (1) or (2).
Well if it is transparent that anarchic post-hoc self-selected online
commentary (1) is not and never will be a substitute for systematic
and answerable peer review (2), but only a welcome supplement to it, it
should not take much more reflection to realize that the most minimal and
uninformative aspect of self-selected vetting, namely citation, is even
*less* suited to take on the a-priori quality-assurance role of peer review.
Again, citation-counts (and other measures of research usage and impact)
are welcome post-hoc *supplements* to research evaluation, but they are
certainly no substitute for peer-review itself and its all-important
filtering function, certifying in advance what is "safe" for reading,
using, applying, consuming. Not being infallible, peer review can use all
the extra help it can get from pre- and post-refereeing commentary and
citations, but there is no way to bootstrap any of those into performing
peer-review's essential and indispensable function.
(Please note that the possibility of posting unrefereed preprints has
already made "gate-keeping" an obsolete misnomer for journals: The "gates"
they guard are those to their established quality-certification tags,
not to access to the texts themselves.)
Peer review should never even have come up in the OA context -- except
tautologically, in that it is the 2.5 million peer-reviewed articles
published in the planet's 24,000 peer-reviewed journals to which OA is
meant maximize access. But somehow, ideas about OA have managed to get
entangled with (untested) speculations about peer-review reforms and
substitutes. The result is that misconceptions about peer review have
been among the panoply of misconceptions that have already delayed OA
(and hence research impact and progress) by at least a decade more than
necessary, since the time the online medium has put 100% OA fully
within reach.
To show that these misconceptions are alive and well in 2005, I quote from an
article that appeared today (July 25) in "The Age":
To publish - or to e-publish?
By Leslie Cannold
http://www.theage.com.au/news/education-news/to-publish--or-to-epublish/2005/07/22/1121539155820.html
The argument seems to start off well, correctly pointing out that:
"The truth is that academics and universities hold most of the cards
in the scholarly publishing game. This is not just because they do
the research, write the papers and do the unpaid work required to
provide quality assurance by reviewing the work of their peers. It
is also because their primary objective is not to profit from the
distribution of their work, but to have it read and cited by others."
We expect that the article will now go on to recommend that, with the
advent of the electronic age, reading and citation can now be maximised,
by self-archiving the text online. But instead we read:
"In the new electronic age... both the organising of and participation
in peer reviews may soon become a thing of the past. Instead of
relying on the publisher's reputation and peer review as quality
indicators, future scholars may depend on electronic citation
counts.
So instead of OA to the peer-reviewed articles, we have OA to unrefereed articles
and their citation counts. And now comes the coup de grace: self-publishing,
instead of the self-archiving of peer-reviewed publications; and somehow, out of
this vanity press plus citations, something like peer-review is somehow meant to
emerge anew:
"[E]very university should have its own serial e-press that
would be the first place of publication for first-rate work. Perhaps
this press would develop into a number of disciplinary-specific,
peer-refereed electronic journals."
We have alas already heard all these ungrounded, untested speculations
aired before, and they do not improve with repetition. In essence
we are asked to assume -- for no earthly reason -- that the path to
OA is to renounce journals, self-publish unrefereed papers in our own
institutional vanity presses, count citations, and wait for peer-review
to somehow re-evolve out of all this.
That would already be empty, evidence-free (and almost certainly incoherent)
speculation even if there were not a coherent, evidence-based and well-tested
alternative staring at us as plainly as the noses on our faces: "You wanted OA
to peer-reviewed journal articles? Self-archive them! No need to tamper with
either peer review or publication. And self-archiving is not self-publishing;
it is just access-provision -- to one's own published articles."
Stevan Harnad
Harnad, S. (1998) The invisible hand of peer review. Nature (on-line) and Exploit
Interactive.
http://eprints.ecs.soton.ac.uk/2622/
Harnad, S. (2005) Fast-Forward on the Green Road to Open Access: The Case Against
Mixing Up Green and Gold. Ariadne 43.
http://eprints.ecs.soton.ac.uk/10675/
Harnad, S., Brody, T., Vallieres, F., Carr, L., Hitchcock, S., Yves, G., Charles,
O., Stamerjohanns, H. and Hilf, E. (2004) The Access/Impact Problem and the Green
and Gold Roads to Open Access. Serials Review 30 (4)
http://eprints.ecs.soton.ac.uk/10209/
Received on Mon Jul 25 2005 - 14:39:50 BST