Selecting the Optimal Value of Penalty Parameter K in Ridge Regression Estimators

Abstract

Ridge regression is one of the popular parameter estimations techniques used to address the multicollinearity problem frequently arising in multiple linear regression. The ridge estimator is based on controlling the magnitude of regression coefficients. The Ridge regression constrains the sum of the absolute values of the regression coefficients to be less than some constant C which is called the penalty.  The Ridge regression shrinks the ordinary least squares estimation vector of regression coefficients towards the origin, allowing a bias but providing a smaller variance. However, the choice of the optimal value of penalty parameter k in Ridge Regression estimators is critical. A Simulation study is conducted to uncover the optimal value of the penalty parameter k under different settings. This simulation study is novel in the field of Ridge Regression Estimators, and it increases the effective capabilities of using the Ridge Regression. Applications on three different real data sets are also considered to support the theoretical findings presented in the simulation study.