Is Novel Research Worth Doing? Evidence from Journal Peer Review

Abstract

Scientific institutions like journals and funding agencies often express desire for novel ideas, but there are long-standing concerns that in practice they favor conservative ones. Here, we examine the association between novelty and acceptance among 21,406 manuscripts submitted between 2013-2018 to one field-leading and one middle-tier life sciences journal. Measuring the novelty of submissions as atypical combinations of journals in their reference lists, we find no evidence of conservatism. Instead, at the top journal, submissions in the top quintile of novelty are 18.5 percentage points more likely than bottom quintile ones to get accepted, while the middle-tier journal shows no systematic favor for or against novel submissions. Separating plausible mechanisms into supply-side (characteristics of the submission pool) and demand-side (judgements by editors and reviewers) shows the importance of the latter, and particularly editor discretion. On the supply-side, novelty is not strongly associated with submissions’ risk or reward, measured with citations. On the demand-side, at the top journal peer reviewers do not select for or against novelty, while editors select strongly for novelty, even conditional on quality. In further support of editor discretion, different editors show different novelty responses to the manuscripts that happened to be reviewed by both journals. Overall, the findings show that journal peer review, particularly at the top, incentivizes novel work.

Keywords: novelty, evaluation, peer review