But if you want to get a better idea of the effect of OA on impact, don't just compare the 2% of ISI journals that are OA journals with the 98% that are not, to find that they are equal in impact (for this may well be comparing apples with oranges even if you equate the subject matter). Compare the much higher percentage of articles from the 98% non-OA journals that have been made OA by their authors -- by self-archiving them -- with articles (from the very same journals and volumes) that have not been made OA by their authors: You will find that there is indeed a discernible difference in terms of frequency with which the article is cited, and that that difference is from 250%-550% in favor of the articles that their authors have made OA! That is what an ongoing series of comparisons based on a 10-year sample of the same ISI database across all disciplines is revealing (in computer science and physics so far: Lawrence 2001; Brody et al. 2004; Kurtz 2004).
On Wed, 14 Apr 2004, Eugene Garfield wrote (in the American Scientist Open Access Forum):
Gene Garfield, the father of citation analysis as well as ISI, is quite right that the Lawrence (2001) study on the impact enhancing effects of open access in computer science needed to be replicated in other fields to check whether it was merely an artifact of the fact that computer science is conference- rather than journal-based.
But, thanks to the ISI database licensed to the Observatoire des Sciences et des Technologies (OST) and a special contract generously provided by ISI to conduct the study, we are in the process of testing the Lawrence effect across all disciplines in a 10-year ISI sample of 14 million articles. The physics analyses up to 2001 are already done (Brody et al. 2004), and they reveal even larger effects than those reported by Lawrence, with OA/non-OA citation ratios of 2.5 - 5.8. All indications are that the 2002 data will raise them even further, as the biggest effects occur within the first 3 years of publication in scientific disciplines (and both OA and the awareness and visibility of OA articles are also increasing yearly).
The ratio of "reads" to "cites" will no doubt vary by field. Kurtz
(2004) and co-workers report it as 17:1 and even 12:1 in
(2002) reports similar experiences in mathematics.
Tim Brody's remarkable download/citation correlator/predictor gives
the size of the correlation by field, and can be used to predict
citation 6-24 months later from downloads today (with an adjustable
Not necessarily, but very probably! And also actually, in the fields tested so far. After all, access is a necessary if not a sufficient precondition for citation. And since Open Access (OA) dramatically increases the number of would-be users who would otherwise have been denied access to the article (maximises it, in fact, for all who have access to the web) it stands to reason that OA can only increase both usage and impact.
The way to test this, however, is not just to compare apples and oranges (i.e., OA and non-OA journals). The right way is to compare OA and non-OA articles in the same journals (and years). That is what our study with the ISI data is doing.
(As Gene himself has often stressed, it is the article [and author] citation counts that should be weighed, and not just the average citation counts of the journals in which the article appears!)
It is certainly not irrelevant to have shown "that there was no discernible difference in terms of citation impact or frequency" between the 191 OA journals and the 8509 non-OA journals indexed by ISI, equating for comparable journals as closely as possible: http://www.isinet.com/oaj But obviously there is a certain risk of circularity in this! It does show that the skeptics are wrong (for these OA journals): OA journals are indexed by ISI, and they do have comparable citation impacts.
But the real test of the effect of OA on citation (and download)
impact is on an article basis, where the journals can be equated
exactly (by being the same
journal and year!). And there the
impact-maximising effects of OA are proving to be very dramatic