Tuesday 12 August 2014

Introduction

The purpose of this blog is to share ideas about uses of alternative metrics for evaluation. It is in part a response to David Colquhoun's blogging against the use of altmetrics. I believe that alternative metrics can be useful in some research evaluation contexts and think it is useful to have a blog covering these contexts.

As part of the Statistical Cybermetrics Research Group, I have been using alternative metrics for research evaluations since 2007 and this seems like a good time to make recommendations for specific applications. Previous evaluations have been for a large UK organisation promoting innovation (Nesta), the EU, the UNDP and individual university departments. All the evaluations so far have had the common factor that the organisations evaluated produce knowledge, but not primarily traditional academic knowledge in the form of journal articles, and need evidence about the wider impact of their articles. For these, we have used a range of web-based metrics to give evidence of general impact. We always include extensive discussions of the limitations of the metrics used and and also recommend the use of content analysis in parallel with the metrics so that the numerical values can be interpreted more accurately.

2 comments:

  1. I'm sorry that you didn't respond at all to the criticisms of altmetrics that I and Andrew Plested raised at http://www.dcscience.net/?p=6369 and in the BMJ.

    You say " I believe that alternative metrics can be useful in some research evaluation contexts", but you adduce no evidence that altmetrics measure quality. The examples that we gave show that, on the contrary, altmetrics are more likely to measure triviality.

    Blogs are great fun. Mine has had 3.7 million views and I hope it has helped to explain some scientific ideas. But it has nothing whatsoever to do with research. I couldn't have afforded to spend time on it when I was doing full time research. It us a fun game for the semi-retired and should not count one iota to my scientific reputation. That depends on papers like http://www.onemol.org.uk/Colquhoun%20&%20Hawkes-1982-ocr.pdf
    I don't think that one would get far on social media.

    ReplyDelete
  2. Please see the reply to this comment in the "Alternative metrics in the future UK..." post within this blog.

    ReplyDelete