Multicollinearity and Maximum Entropy Leuven Estimator
9 Pages Posted: 17 Jun 2004
Abstract
Deleterious effects of a high degree of multicollinearity on estimation of regression coefficients (beta) of a linear model are well known. As a remedial measure, Hoerl & Kennard (1970) introduced Ridge Regression, but as Theobald (1974) pointed out, its optimality depends on unknown parameters and replacing the unknown population parameters by their sample estimates does not ensure its advantage over the OLS. Golan et al. (1996) introduced the Generalized Maximum Entropy (GME) estimator to resolve the multicollinearity problem. The GME requires a number of support values supplied subjectively by the researcher, which in a real life situation are hard to obtain. Consequently, GME estimator loses its practical significance.
Paris (2001) introduced the Maximum Entropy Leuven (MEL) estimator. It exploits the information available in the sample data more efficiently than the OLS does, and unlike GME estimator, it does not require any additional information to be supplied by the researcher. Paris concluded: under any level of multicollinearity, MEL estimator uniformly dominates the OLS estimator according to the mean squared error criterion. It rivals also the GME estimator without requiring any subjective additional information.
In this paper we look into the problem of multicollinerarity more closely and shed more light on the MEL estimator through discussion and simulation based on Monte Carlo experiments. We also propose a Modular MEL (call it MMEL) estimator.
In measuring the entropy, the MEL estimator obtains prob(beta) through normalizing the square of beta by the Euclidean norm of beta vector. Instead, the MMEL estimator obtains prob(beta) through normalizing the absolute value of beta by the absolute norm of beta vector. We observe that overall, the performance of the MMEL estimator (in terms of mean estimated coefficients as well as the RMS values in 50 trials) is much superior to that of the MEL estimator.
Keywords: Multicollinearity, maximum entropy, MEL estimator, Leuven estimator, condition number
JEL Classification: C13, C15
Suggested Citation: Suggested Citation