PERFORMANCE OF SOME REGULARIZATION METHODS: THE LASSO, BRIDGE AND ELASTIC-NET REGRESSION METHODS
A. BUBA
Deparment of Mathematics, Federal University Birnin-Kebbi, Kebbi, Nigeria.
U. USMAN *
Department of Mathematics, Usmanu Danfodiyo University, Sokoto, Nigeria.
*Author to whom correspondence should be addressed.
Abstract
Collinearity of predictor variables is a severe problem in the least square regression analysis. It contributes to the instability of regression coefficients and leads to a wrong prediction accuracy. This study examines the performance of LASSO, BRIDGE and E-Net methods and traditional method (OLS method) under different levels of multicollinearity. A result from simulation analysis indicates that when the sample size is small and it contains very high multicollinearity, the estimates of the shrinkage methods (LASSO, BRIDGE and E-Net) are efficient than those of OLS. The performance of the BRIDGE and OLS are almost similar at large sample sizes. If the number of predictive variables are much more than that in this setting, the PE (predictive error) given by shrinkage methods will be less than that given by OLS The result also shows that regardless of the level of multicollinearity, OLS remains the least biased and is also most efficient in terms of prediction.
Keywords: Ordinary Least Squares (OLS), mean square error (MSE), prediction error (PE), Least Absolute Shrinkage Operation (LASSO).