When to Consult Precision-Recall Curves

16 Pages Posted: 3 Apr 2019 Last revised: 13 Jul 2019

See all articles by Jonathan A Cook

Jonathan A Cook

U.S. Securities and Exchange Commission; affiliation not provided to SSRN

Vikram Ramadas

Public Company Accounting Oversight Board

Date Written: June 1, 2019

Abstract

ROC curves are commonly used to evaluate predictions of binary outcomes. When there are a small percentage of items of interest (as would be the case with fraud detection, for example), ROC curves can provide an inflated view of performance. This can cause challenges in trying to determine which set of predictions is better. This article discusses the conditions under which precision-recall curves may be preferable to ROC curves. As an illustrative example, we compare two commonly used fraud predictors (Beneish's (1999) M-score and Dechow et al.'s (2011) F-score) using both ROC and precision-recall curves.

To aid the reader with using precision-recall curves, this article also introduces a Stata module to plot them. This module is now available on the Boston College Statistical Software Components (SSC) archive and can be installed by typing ssc install prtab in Stata.

Keywords: Precision-recall curves, Classifier evaluation, ROC curves, Fraud detection

Suggested Citation

Cook, Jonathan A and Ramadas, Vikram, When to Consult Precision-Recall Curves (June 1, 2019). Available at SSRN: https://ssrn.com/abstract=3350582 or http://dx.doi.org/10.2139/ssrn.3350582

Jonathan A Cook (Contact Author)

U.S. Securities and Exchange Commission ( email )

affiliation not provided to SSRN

Vikram Ramadas

Public Company Accounting Oversight Board ( email )

1666 K Street, NW
Washington, DC 20006-2
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
293
Abstract Views
2,069
Rank
189,557
PlumX Metrics