Adminstrative info for SIGMETRICS (for example unsubscribe):
http://web.utk.edu/~gwhitney/sigmetrics.html
On 22-Oct-08, at 10:10 AM, David E. Wojick wrote:
> Speaking metaphysically, I find the following problematic -- "The point is
> that metrics need to be plural, diverse, and validated and weighted by
> field and
> subfield."
>
> I would argue that the concepts of field and subfield are too vague and
> unscientific to be the basis for objective metrics. Science is an ever
> changing body of inquiry, with personal, methodological and subject matter
> clusters that change with scale and over time, sometimes very rapidly. No
> two papers are about exactly the same thing. Each paper is related to its
> neighbors in multiple ways. Each object of study can be looked at in
> multiple ways.
>
> Thus field and subfield are always artificial divisions of convenience
> imposed on an unbroken web of belief and inquiry. Moreover, these
> divisions can often be usefully made in multiple ways. If the science of
> science depends on these artificial divisions as a basis for measurement
> then we are in serious trouble.
Yes, field and subfield are artificial divisions of convenience. So are
institutions, laboratories, research funders, research projects, research
assessment exercises, etc. But we do make these artificial divisions, so we
need metrics to provide objective ways to navigate within them. If the
artificial divisions (such as disciplines) prove inconvenient, or have
untoward consequences, we can artificially redivide them.
The point is that we should not demand more of our metrics than we demand of
the data to which we apply them. And that multiple metrics are much more
promising than sticking to just one.
The kind of radical relativism and subjectivism that is implied by:
"No two papers are about exactly the same thing. Each paper is related to
its neighbors in multiple ways. Each object of study can be looked at in
multiple ways."
would rule out not only metrics, but any sort of evaluation or comparison.
By that token, we may as well toss a coin (or conduct an opinion poll) in
deciding whom or what to fund, credit, or otherwise reward.
'the man who is ready to prove that metaphysics is wholly impossible... is a
brother metaphysician with a rival theory.'
Stevan Harnad
> David Wojick
> DOE OSTI
>
> >
> > On 22-Oct-08, at 7:33 AM, Loet Leydesdorff wrote:
> >
> > > It seems to me that the expectation of the citation frequency is
> > > among other
> > > things a function of the local density of the citation network. A
> > > problem,
> > > however, remains how to define the locale: a journal, a theme, a
> > > patent
> > > class? "Quick and dirty" skips these problems, in my opinion. I agree
> > > that
> > > it may be pragmatical and shows that a solution is possible in
> > > principle.
> >
> > With open access, it is no longer univariate (i.e., not just citation
> > counts) and
> > it is definitely no longer journal-centric (author and article metrics,
> > not journal
> > JIFs, though journal JIFs can be among the metrics used). The point is
> > that
> > metrics need to be plural, diverse, and validated and weighted by field
> > and
> > subfield.
> >
> > To repeat: The "quick and dirty" example I gave was not meant to be
> > used,
> > but to show that solutions (many solutions) are possible in principle,
> > and
> > that their main features are that they are (1) multivariate, (2) field
> > or even
> > subfield-based, (3) require prior (joint) validation, field by field,
> > against an
> > already validated or face-valid criterion (such as peer evaluation),
> > and, most
> > important, they are (4) conditional on the provision of a full Open
> > Access
> > database on which to base them -- a condition that does not yet exist,
> > but
> > one for which we are now fighting (using the potential of multiple Open
> > Access
> > metrics as an incentive).
> >
> > > The problem seems to me in the inference from aggregated citing
> > > behavior to
> > > an expectation of being cited. The analyst transposed the
> > > citation-transaction matrix (Wouters, 1999).
> >
> > The matrix I have in mind is not a citation matrix, but a matrix
> > consisting of a
> > rich and diverse set of metrics, including downloads, chronometrics
> > (growth/decay of citations, downloads), co-citation metrics, co-
> > authorship
> > metrics, funding metrics, student metrics. patent metrics, link metrics,
> > hub/authority metrics, endogamy/exogamy metrics, years of publication,
> > total publications, tag metrics, comment metrics, semiometrics, and
> > more --
> > all these harvested from the Open Access Research Web, once all articles
> > are OA, citation-linked, and download-metered.
> >
> > The difference between this plurimetric world and the world of
> > univariate
> > citations will be like the difference between night and day. But we are
> > still in
> > the night...
> >
> > Stevan Harnad
>
> --
>
> "David E. Wojick, PhD" <WojickD_at_osti.gov>
> Senior Consultant for Innovation
> Office of Scientific and Technical Information
> US Department of Energy
> http://www.osti.gov/innovation/
> 391 Flickertail Lane, Star Tannery, VA 22654 USA
> 540-858-3136
>
> http://www.bydesign.com/powervision/resume.html provides my bio and past
> client list.
> http://www.bydesign.com/powervision/Mathematics_Philosophy_Science/
> presents some of my own research on information structure and dynamics.
Received on Wed Oct 22 2008 - 15:49:07 BST