Peer Review of Social Science Research in Global Health: A View Through Correspondence Letters to 'The Lancet'

14 Pages Posted: 25 Jun 2014

See all articles by Victoria Fan

Victoria Fan

Center for Global Development

Rachel Silverman

Center for Global Development

David Roodman

Open Philanthropy

William D. Savedoff

Center for Global Development; Social Insight

Date Written: June 24, 2014

Abstract

In recent years, the interdisciplinary nature of global health has blurred the lines between medicine and social science. As medical journals publish non-experimental research articles on social policies or macro-level interventions, controversies have arisen when social scientists have criticized the rigor and quality of medical journal articles, raising general questions about the frequency and characteristics of methodological problems and the prevalence and severity of research bias and error. Published correspondence letters can be used to identify common areas of dispute within interdisciplinary global health research and seek strategies to address them. To some extent, these letters can be seen as a "crowd-sourced" (but editor-gated) approach to public peer review of published articles, from which some characteristics of bias and error can be gleaned. In December 2012, we used the online version of The Lancet to systematically identify relevant correspondence in each issue published between 2008 and 2012. We summarize and categorize common areas of dispute raised in these letters. The five concerns most frequently cited in correspondence letters are as follows: measurement error (51% of papers); omitted variables and confounding (45%); implausibility and lack of external validity (43%); missing or low-quality data (32%); and lack of transparency of methods (30%). We recommend better documentation of areas of potential bias with checklists and guidelines to facilitate more rigorous peer review, drawing on experts with econometric expertise as reviewers, and explicitly and thoroughly linking all correspondence letters to the original articles in The Lancet. Most importantly, we recommend The Lancet adopts the replication standard, whereby the data and the coding used to produce the estimates are provided at least to the journal, for reviewers to analyze and replicate the estimates reported by the authors.

Keywords: global health, The Lancet, medical journals, economic journals, research, social science, error

Suggested Citation

Fan, Victoria and Silverman, Rachel and Roodman, David and Savedoff, William D., Peer Review of Social Science Research in Global Health: A View Through Correspondence Letters to 'The Lancet' (June 24, 2014). Available at SSRN: https://ssrn.com/abstract=2458524 or http://dx.doi.org/10.2139/ssrn.2458524

Victoria Fan (Contact Author)

Center for Global Development ( email )

2055 L St. NW
5th floor
Washington, DC 20036
United States

Rachel Silverman

Center for Global Development ( email )

2055 L St. NW
5th floor
Washington, DC 20036
United States

David Roodman

Open Philanthropy ( email )

182 Howard Street #225
San Francisco, CA 94105
United States

HOME PAGE: http://openphilanthropy.org

William D. Savedoff

Center for Global Development ( email )

2055 L St. NW
5th floor
Washington, DC 20036
United States

Social Insight ( email )

15 Centre Street, Suite 4
Bath, ME 04530
United States
207-443-2331 (Phone)

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
42
Abstract Views
852
PlumX Metrics