Constructing 'Proper' ROCs from Ordinal Response Data Using Weighted Power Functions

Medical Decision Making, Forthcoming

31 Pages Posted: 2 Dec 2012 Last revised: 18 Aug 2013

See all articles by Douglas Mossman

Douglas Mossman

University of Cincinnati College of Medicine

Hongying Peng

University of Cincinnati - College of Medicine

Date Written: April 14, 2013

Abstract

Background: Receiver operating characteristic (ROC) analysis is the standard method for describing the accuracy of diagnostic systems where the decision task involves distinguishing between two mutually exclusive possibilities. The popular binormal curve-fitting model usually produces ROCs that are improper in that they do not have the ever-decreasing slope required by signal detection theory. Not infrequently, binormal ROCs have visible hooks that falsely imply worse-than-chance diagnostic differentiation where the curve lies below the no-information diagonal.

Objective: To present and evaluate a two-parameter, weighted power function (WPF) model that always results in a proper ROC curve with a positive, monotonically decreasing slope.

Design/Method: Computer simulation study comparing results from binormal and WPF models.

Results: The WPF model produces ROC curves that are less biased and closer to the true values than curves obtained using the binormal model. The better performance of the WPF model follows from its design constraint as a necessarily proper ROC.

Conclusions: The WPF model fits a broader variety of data sets than previously published power function models while maintaining straightforward relationships among the original decision variable, specific operating points, ROC curve contours, and model parameters. Compared to other proper ROC models, the WPF model is distinctive in its simplicity, and it avoids the flaws of the conventional binormal ROC model.

Keywords: receiver operating characteristic, ROC, binormal ROC, proper ROC

JEL Classification: C13, C15

Suggested Citation

Mossman, Douglas and Peng, Hongying, Constructing 'Proper' ROCs from Ordinal Response Data Using Weighted Power Functions (April 14, 2013). Medical Decision Making, Forthcoming, Available at SSRN: https://ssrn.com/abstract=2184051 or http://dx.doi.org/10.2139/ssrn.2184051

Douglas Mossman (Contact Author)

University of Cincinnati College of Medicine ( email )

260 Stetson Street, Suite 3200
P. O. Box 670559
Cincinnati, OH 45219
United States
513-558-4423 (Phone)

Hongying Peng

University of Cincinnati - College of Medicine

260 Stetson Street, Suite 3200
P. O. Box 670559
Cincinnati, OH 45219
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
83
Abstract Views
591
Rank
539,187
PlumX Metrics