Future UK RAEs to be Metrics-Based

From: Stevan Harnad <harnad_at_ecs.soton.ac.uk>
Date: Thu, 23 Mar 2006 14:47:59 +0000

As predicted, and long urged, the UK's wasteful, time-consuming
Research Assessment Exercise (RAE) is to be replaced by metrics:

    "Research exercise to be scrapped"
    Donald MacLeod, Guardian Wednesday March 22, 2006

    Science and innovation investment framework 2004-2014: next steps
    supporting UK Budget 2006

        "The Government is strongly committed to the dual support system,
        and to rewarding research excellence, but recognises some of
        the burdens imposed by the existing Research Assessment Exercise
        (RAE). The Government's firm presumption is that after the 2008
        RAE the system for assessing research quality and allocating
        'quality-related' (QR) funding will be mainly metrics-based... The
        Government will launch a consultation on its preferred option
        for a metrics-based system for assessing research quality and
        allocating QR funding, publishing results in time for the 2006
        Pre-Budget Report."

        "Over recent years a number of studies have considered options
        for a radically different allocation system for QR in order to
        avoid or reduce the need for a peer review process. The focus
        in most cases has been on identifying one or more metrics that
        could be used to assess research quality and allocate funding,
        for example research income, citations, publications, research
        student numbers etc. The Government has considered the evidence
        to date and favours identifying a simpler system that may not
        precisely replicate the level of detailed analysis of the RAE
        but would enable an appropriate distribution of QR funding at
        the institutional level."

        "[M]etrics collected as part of the next assessment will be
        used to undertake an exercise shadowing the 2008 RAE itself, to
        provide a benchmark on the information value of the metrics as
        compared to the outcomes of the full peer review process. The
        aim of any changes following this exercise will be to reduce
        the administrative burden of peer review, wherever possible,
        consistent with the overriding aim of assessing excellence"

RAE outcome is most closely correlated (r = 0.98) with the metric of
prior RCUK research funding (Figure 4.1) (this is no doubt in part a
"Matthew Effect"), but research citation impact is another metric highly
correlated with the RAE outcome, even though it is not explicitly
counted. Now it can be explicitly counted (along with other powerful new
performance metrics) and all the rest of the ritualistic time-wasting
can be abandoned, without further ceremony.

This represents a great boost for institutional self-archiving in Open
Access Institutional Repositories, not only because that is the obvious,
optimal means of submission to the new metric RAE, but because it is
also a powerful means of maximising research impact, i.e., maximising
those metrics: (I hope Research Councils UK (RCUK) is listening!).

    "Research exercise to be scrapped"
     Donald MacLeod, Guardian Wednesday March 22, 2006

    Harnad, S. (2001) Why I think that research access, impact and assessment
    are linked. Times Higher Education Supplement 1487: p. 16.

    Harnad, S. (2003) Why I believe that all UK research output
    should be online.
    Times Higher Education Supplement. Friday, June 6 2003.

    Harnad, S., Carr, L., Brody, T. & Oppenheim, C. (2003) Mandated
    online RAE CVs Linked to University Eprint Archives: Improving
    the UK Research Assessment Exercise whilst making it cheaper and
    easier. Ariadne 35.

    Beans and Bean Counters (2005)

Stevan Harnad
Received on Thu Mar 23 2006 - 14:57:17 GMT

This archive was generated by hypermail 2.3.0 : Fri Dec 10 2010 - 19:48:16 GMT