Measuring Relative Accuracy: A Better Alternative to Mean Absolute Percentage Error
Hertfordshire Business School Working Paper (2013)
24 Pages Posted: 7 Nov 2013
Date Written: October 12, 2013
Abstract
Surveys show that the mean absolute percentage error (MAPE) is the most widely used measure of forecast accuracy in businesses and organizations. It is also used to compare accuracy across multiple data sets, e.g. when choosing a forecasting method. Yet this metric systematically favours methods which under-forecast. Thus when MAPE is used for model selection it will be biased. We explain why this happens.
We investigate an alternative relative error measure based on the forecast to actual ratio, and demonstrate that it overcomes this problem for strictly positive data e.g. costs, sales volume, project times, financial asset prices etc. We also illustrate its use in estimating the prediction model using real data. We demonstrate that the associated regression model involves a multiplicative error rather than the usual additive one. It estimates the geometric mean rather than the arithmetic mean (and so is less affected by outliers), and possesses a form of unbiasedness which is appropriate for relative accuracy. This measure therefore seems preferable to MAPE for use in practice.
Keywords: forecasting, accuracy, error measure, loss function, scoring function, regression
JEL Classification: C13, C22, C52, C53
Suggested Citation: Suggested Citation