Multicollinearity and Maximum Entropy Leuven Estimator

9 Pages Posted: 17 Jun 2004

Abstract

Deleterious effects of a high degree of multicollinearity on estimation of regression coefficients (beta) of a linear model are well known. As a remedial measure, Hoerl & Kennard (1970) introduced Ridge Regression, but as Theobald (1974) pointed out, its optimality depends on unknown parameters and replacing the unknown population parameters by their sample estimates does not ensure its advantage over the OLS. Golan et al. (1996) introduced the Generalized Maximum Entropy (GME) estimator to resolve the multicollinearity problem. The GME requires a number of support values supplied subjectively by the researcher, which in a real life situation are hard to obtain. Consequently, GME estimator loses its practical significance.

Paris (2001) introduced the Maximum Entropy Leuven (MEL) estimator. It exploits the information available in the sample data more efficiently than the OLS does, and unlike GME estimator, it does not require any additional information to be supplied by the researcher. Paris concluded: under any level of multicollinearity, MEL estimator uniformly dominates the OLS estimator according to the mean squared error criterion. It rivals also the GME estimator without requiring any subjective additional information.

In this paper we look into the problem of multicollinerarity more closely and shed more light on the MEL estimator through discussion and simulation based on Monte Carlo experiments. We also propose a Modular MEL (call it MMEL) estimator.

In measuring the entropy, the MEL estimator obtains prob(beta) through normalizing the square of beta by the Euclidean norm of beta vector. Instead, the MMEL estimator obtains prob(beta) through normalizing the absolute value of beta by the absolute norm of beta vector. We observe that overall, the performance of the MMEL estimator (in terms of mean estimated coefficients as well as the RMS values in 50 trials) is much superior to that of the MEL estimator.

Keywords: Multicollinearity, maximum entropy, MEL estimator, Leuven estimator, condition number

JEL Classification: C13, C15

Suggested Citation

Mishra, Sudhanshu K., Multicollinearity and Maximum Entropy Leuven Estimator. Available at SSRN: https://ssrn.com/abstract=556564 or http://dx.doi.org/10.2139/ssrn.556564

Sudhanshu K. Mishra (Contact Author)

North-Eastern Hill University (NEHU) ( email )

NEHU Campus
Shillong, 793022
India
03642550102 (Phone)

HOME PAGE: http://www.nehu-economics.info

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
296
Abstract Views
1,964
Rank
187,508
PlumX Metrics