Do Rankings Reflect Research Quality?

Journal of Applied Economics. Vol. 13, No. 1, pp. 1-38, May 2010

Posted: 12 Jun 2010

See all articles by Bruno S. Frey

Bruno S. Frey

CREMA; University of Basel

Katja Rost

University of Zurich - Institute for Organization and Administrative Science

Date Written: May 1, 2010

Abstract

Publication and citation rankings have become major indicators of the scientific worth of universities and determine to a large extent the career of individual scholars. Such rankings do not effectively measure research quality, which should be the essence of any evaluation. These quantity rankings are not objective; two citation rankings, based on different samples, produce entirely different results. For that reason, an alternative ranking is developed as a quality indicator, based on membership on academic editorial boards of professional journals. It turns out that the ranking of individual scholars based on that measure is far from objective. Furthermore, the results differ markedly, depending on whether research quantity or quality is considered. Thus, career decisions based on rankings are dominated by chance and do not reflect research quality. We suggest that evaluations should rely on multiple criteria. Public management should return to approved methods such as engaging independent experts who in turn provide measurements of research quality for their research communities.

Keywords: rankings, evaluations, universities, scholars, research quality

JEL Classification: H43, L15, O38

Suggested Citation

Frey, Bruno S. and Rost, Katja, Do Rankings Reflect Research Quality? (May 1, 2010). Journal of Applied Economics. Vol. 13, No. 1, pp. 1-38, May 2010, Available at SSRN: https://ssrn.com/abstract=1623412

Bruno S. Frey

CREMA ( email )

Südstrasse 11
Zurich, CH 8008
Switzerland
+41 44 380 00 78 (Phone)

University of Basel ( email )

Peter Merian-Weg 6
Basel, 4002
Switzerland

Katja Rost (Contact Author)

University of Zurich - Institute for Organization and Administrative Science ( email )

Plattenstrasse 14
CH-8032 Zurich
Switzerland

Do you have negative results from your research you’d like to share?

Paper statistics

Abstract Views
646
PlumX Metrics