No System is Perfect: Understanding How Registration-Based Editorial Processes Affect Reproducibility and Investment in Research Quality

61 Pages Posted: 13 Feb 2018

See all articles by Robert J. Bloomfield

Robert J. Bloomfield

Cornell University - Samuel Curtis Johnson Graduate School of Management

Kristina M. Rennekamp

SC Johnson Graduate School of Management; Cornell SC Johnson College of Business

Blake A. Steenhoven

Queen's University - Smith School of Business

Multiple version iconThere are 2 versions of this paper

Date Written: February 5, 2018

Abstract

The papers in this volume were published through a registration-based Editorial Process (REP). Authors submitted proposals to gather and analyze data; successful proposals were guaranteed publication as long as the authors lived up to their commitments, regardless of whether results supported their predictions. To understand how REP differs from the Traditional Editorial Process (TEP), we analyze the papers themselves; conference comments; a survey of conference authors, reviewers, and attendees; and a survey of authors who have successfully published under TEP. We find that REP increases up-front investment in planning, data gathering, and analysis, but reduces follow-up investment after results are known. This shift in investment makes individual results more reproducible, but leaves articles less thorough and refined. REP could be improved by encouraging selected forms of follow-up investment that survey respondents believe are usually used under TEP to make papers more informative, focused, and accurate at little risk of overstatement.

Keywords: Registered Reports, Reproducibility, Editorial Processes, Research Discretion, Peer Review

JEL Classification: C18, I23, B00, M40, M41

Suggested Citation

Bloomfield, Robert J. and Rennekamp, Kristina M. and Steenhoven, Blake, No System is Perfect: Understanding How Registration-Based Editorial Processes Affect Reproducibility and Investment in Research Quality (February 5, 2018). Available at SSRN: https://ssrn.com/abstract=3118687 or http://dx.doi.org/10.2139/ssrn.3118687

Robert J. Bloomfield (Contact Author)

Cornell University - Samuel Curtis Johnson Graduate School of Management ( email )

450 Sage Hall
Ithaca, NY 14853
United States
607-255-9407 (Phone)
607-254-4590 (Fax)

Kristina M. Rennekamp

SC Johnson Graduate School of Management ( email )

Ithaca, NY 14853
United States
607-255-0500 (Phone)

Cornell SC Johnson College of Business ( email )

Ithaca, NY 14850
United States

Blake Steenhoven

Queen's University - Smith School of Business ( email )

Smith School of Business - Queen's University
143 Union Street
Kingston, Ontario K7L 3N6
Canada

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
526
Abstract Views
3,142
Rank
97,714
PlumX Metrics