COMPARISIONS OF STIMULTANEOUS SHRINKAGE AND VARIABLE SELECTION METHODS: THE LASSO, AND BRIDGE REGRESSION METHOD
U. USMAN *
Department of Mathematics, Usmanu Danfodiyo University, Sokoto. Nigeria.
Federal University Birnin-Kebbi, Kebbi, Nigeria.
*Author to whom correspondence should be addressed.
Multicolinaerity is a serious problem that has attracted the attention of many researchers in the field of regression analysis due to the fact that the use of Ordinary Least Squares (OLS) regression in the presence of multicollinearity leads to high variability in the estimates of the regression coefficients. To solve this problem, several shrinkage methods such as LASSO, E-Net, Bridge, ridge, etc. have been proposed. This study evaluates the performance of the OLS and some shrinkage methods namely the LASSO and BRIDGE for remedying multicollinearity. The study compares these methods under varying collinearity levels and sample sizes based on criteria such as bias, MSE, efficiency and prediction. The results of the simulation show that the estimates of the LASSO and BRIDGE are more efficient than those of OLS only when the sample size is small in the presence of very high multicollinearity. The study further reveals that the performance of the BRIDGE and OLS are almost equivalent at large sample sizes and regardless of the level of multicollinearity, OLS remains the least biased and is also most efficient in terms of prediction.
Keywords: Regression, multicollinearity, The least absolute shrinkage selection Operator (LASSO), BRIDGE Regression and Ordinary Least Squares (OLS).