Evaluating Performance of Regression Machine Learning Models Using Multiple Error Metrics in Azure Machine Learning Studio

16 Pages Posted: 24 May 2018

Date Written: May 12, 2018

Abstract

Data driven companies effectively use regression machine learning methods for making predictions in many sectors. Cloud-based Azure Machine Learning Studio (MLS) has a potential of expediting machine learning experiments by offering a convenient and powerful integrated development environment. The process of evaluating machine learning models in Azure MLS has certain limitations, e.g. small number of performance metrics and lack of functionality to evaluate custom built regression models with R language. This paper reports the results of an effort to build an Enhanced Evaluate Model (EEM) module which facilitates and accelerates Azure experiments development and evaluation. The EEM combines multiple performance metrics allowing for multisided evaluation of the regression models. EEM offers 4 times more metrics than built-in Azure Evaluate Model module. EEM metrics include: CoD, GMRAE, MAE, MAPE, MASE, MdAE, MdAPE, MdRAE, ME, MPE, MRAE, MSE, NRMSE_mm, NRMSE_sd, RAE, RMdSPE, RMSE, RMSPE, RSE, sMAPE, SMdAPE, SSE. Also, EEM supports evaluation of the R language based regression models. The operational Enhanced Evaluate Model module has been published to the web and openly available for experiments and extensions.

Keywords: machine learning, regression, performance metrics, accuracy, error, error measure, multiple types, models, prediction, evaluation, forecast, Azure Machine Learning Studio, R, R package

JEL Classification: C1, C4, C52, C53, C6

Suggested Citation

Botchkarev, Alexei, Evaluating Performance of Regression Machine Learning Models Using Multiple Error Metrics in Azure Machine Learning Studio (May 12, 2018). Available at SSRN: https://ssrn.com/abstract=3177507 or http://dx.doi.org/10.2139/ssrn.3177507

Alexei Botchkarev (Contact Author)

Financial University ( email )

49 Leningradsky Prospekt
Moscow, 125993
Russia

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
2,117
Abstract Views
5,452
Rank
13,678
PlumX Metrics