Fixed- Versus Random-Effects Models in Meta-Analysis: Model Properties and an Empirical Comparison of Differences in Results

British Journal of Mathematical and Statistical Psychology (2009), 62, 97-128

32 Pages Posted: 27 Feb 2014

See all articles by In‐Sue Oh

In‐Sue Oh

Temple University - Department of Human Resource Management

Frank L. Schmidt

University of Iowa - Henry B. Tippie College of Business

Theodore L. Hayes

The Gallup Organization

Date Written: 2009

Abstract

Today most conclusions about cumulative knowledge in psychology are based on meta-analysis. We first present an examination of the important statistical differences between fixed-effects (FE) and random-effects (RE) models in meta-analysis and between two different RE procedures, due to Hedges and Vevea, and to Hunter and Schmidt. The implications of these differences for the appropriate interpretation of published meta-analyses are explored by applying the two RE procedures to 68 meta-analyses from five large meta-analytic studies previously published in Psychological Bulletin. Under the assumption that the goal of research is generalizable knowledge, results indicated that the published FE confidence intervals (CIs) around mean effect sizes were on average 52% narrower than their actual width, with similar results being produced by the two RE procedures. These nominal 95% FE CIs were found to be on average 56% CIs. Because most meta-analyses in the literature use FE models, these findings suggest that the precision of meta-analysis findings in the literature has often been substantially overstated, with important consequences for research and practice.

Suggested Citation

Oh, In-Sue and Schmidt, Frank L. and Hayes, Theodore L., Fixed- Versus Random-Effects Models in Meta-Analysis: Model Properties and an Empirical Comparison of Differences in Results (2009). British Journal of Mathematical and Statistical Psychology (2009), 62, 97-128, Available at SSRN: https://ssrn.com/abstract=2401182

In-Sue Oh (Contact Author)

Temple University - Department of Human Resource Management ( email )

1801 Liacouras Walk
Alter Hall 343
Philadelphia, PA 19122
United States

HOME PAGE: http://www.fox.temple.edu/mcm_people/in-sue-oh/

Frank L. Schmidt

University of Iowa - Henry B. Tippie College of Business ( email )

Acquisitions
5020 Main Library
Iowa City, IA 52242-1000
United States

Theodore L. Hayes

The Gallup Organization

Washington, DC 20004
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
61
Abstract Views
1,215
Rank
638,193
PlumX Metrics