Measuring Relative Accuracy: A Better Alternative to Mean Absolute Percentage Error

Hertfordshire Business School Working Paper (2013)

24 Pages Posted: 7 Nov 2013

See all articles by Chris Tofallis

Chris Tofallis

University of Hertfordshire Business School

Date Written: October 12, 2013

Abstract

Surveys show that the mean absolute percentage error (MAPE) is the most widely used measure of forecast accuracy in businesses and organizations. It is also used to compare accuracy across multiple data sets, e.g. when choosing a forecasting method. Yet this metric systematically favours methods which under-forecast. Thus when MAPE is used for model selection it will be biased. We explain why this happens.

We investigate an alternative relative error measure based on the forecast to actual ratio, and demonstrate that it overcomes this problem for strictly positive data e.g. costs, sales volume, project times, financial asset prices etc. We also illustrate its use in estimating the prediction model using real data. We demonstrate that the associated regression model involves a multiplicative error rather than the usual additive one. It estimates the geometric mean rather than the arithmetic mean (and so is less affected by outliers), and possesses a form of unbiasedness which is appropriate for relative accuracy. This measure therefore seems preferable to MAPE for use in practice.

Keywords: forecasting, accuracy, error measure, loss function, scoring function, regression

JEL Classification: C13, C22, C52, C53

Suggested Citation

Tofallis, Chris, Measuring Relative Accuracy: A Better Alternative to Mean Absolute Percentage Error (October 12, 2013). Hertfordshire Business School Working Paper (2013), Available at SSRN: https://ssrn.com/abstract=2350688 or http://dx.doi.org/10.2139/ssrn.2350688

Chris Tofallis (Contact Author)

University of Hertfordshire Business School ( email )

College Lane
Hatfield, Hertfordshire AL10 9AB
United Kingdom

HOME PAGE: http://tinyurl.com/ChrisTofallis

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
1,343
Abstract Views
5,312
Rank
27,477
PlumX Metrics