Bias Against Novelty in Science: A Cautionary Tale for Users of Bibliometric Indicators

30 Pages Posted: 5 Jan 2016

See all articles by Jian Wang

Jian Wang

KU Leuven - Department of Managerial Economics, Strategy, and Innovation

Reinhilde Veugelers

Catholic University of Leuven (KUL) - Department of Applied Economics; Centre for Economic Policy Research (CEPR)

Paula E. Stephan

Georgia State University - Department of Economics; National Bureau of Economic Research (NBER)

Multiple version iconThere are 4 versions of this paper

Date Written: December 2015

Abstract

Research which explores unchartered waters has a high potential for major impact but also carries a high uncertainty of having minimal impact. Such explorative research is often described as taking a novel approach. This study examines the complex relationship between pursuing a novel approach and impact. We measure novelty by examining the extent to which a published paper makes first time ever combinations of referenced journals, taking into account the difficulty of making such combinations. We apply this newly developed measure of novelty to a set of one million research articles across all scientific disciplines. We find that highly novel papers, defined to be those that make more (distinct) new combinations, have more than a triple probability of being a top 1% highly cited paper when using a sufficiently long citation time window to assess impact. Moreover, follow-on papers that cite highly novel research are themselves more likely to be highly cited. However, novel research is also risky as it has a higher variance in the citation performance. These findings are consistent with the “high risk/high gain” characteristic of novel research. We also find that novel papers are typically published in journals with a lower than expected Impact Factor and are less cited when using a short time window. Our findings suggest that science policy, in particular funding decisions which are over reliant on traditional bibliometric indicators based on short-term direct citation counts and Journal Impact Factors, may be biased against novelty.

Suggested Citation

Wang, Jian and Veugelers, Reinhilde and Stephan, Paula E., Bias Against Novelty in Science: A Cautionary Tale for Users of Bibliometric Indicators (December 2015). Available at SSRN: https://ssrn.com/abstract=2710572 or http://dx.doi.org/10.2139/ssrn.2710572

Jian Wang (Contact Author)

KU Leuven - Department of Managerial Economics, Strategy, and Innovation ( email )

Naamsestraat 69 bus 3500
Leuven, 3000
Belgium

Reinhilde Veugelers

Catholic University of Leuven (KUL) - Department of Applied Economics ( email )

Leuven, B-3000
Belgium
+32 16 32 6908 (Phone)
+32 16 32 6732 (Fax)

Centre for Economic Policy Research (CEPR)

London
United Kingdom

Paula E. Stephan

Georgia State University - Department of Economics ( email )

P.O. Box 3992
Atlanta, GA 30302-3992
United States

National Bureau of Economic Research (NBER) ( email )

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
186
Abstract Views
1,804
Rank
159,344
PlumX Metrics