Pre-tuned ridge regression and its extension to generalized linear models
Abstract
Ridge regression is regularization or shrinkage method and a common approach in dealing with multicollinearity in conventional regression analysis. Ridge regression is widely used by statistical analyst since it is one of the best compared to other regularization methods. Also, the introduction of high dimension and ultra-high dimensional data has become an issue of concern and ridge regression is one way of dealing with such data. One of the key issues associated with ridge regression is the determination of the tuning or ridge parameter. The common practice is to fit ridge regression for a different number of values of tuning parameter before selecting the best tuning parameter. Our question is - instead of fitting ridge regression for a number of different values of the tuning parameter before selecting the best tuning parameter via minimum GCV or other selection criterion, can the optimal tuning parameter be determined directly using without computing the estimated regression coefficients for each tuning parameter? The goal of this study is to put forward a shortcut method for conducting ridge regression. The main idea is to first estimate the best tuning parameter in ridge regression directly and compute their respective fitted values accordingly. For this reason, we call the proposed method as 'pre-tuned ridge regression'. The objective function naturally becomes an estimating equation of tuning parameter which allows us to formulate selection of the tuning parameter as an estimation problem. In our method, the best choice of tuning parameter can be found beforehand and this is a one-dimensional smooth optimization problem. These modifications would shorten the computational time and improve the fitting performance as we will demonstrate. We will also extend our proposed method to generalized linear models.
Subject Area
Statistics
Recommended Citation
Wonkye, Yaa Tawiah, "Pre-tuned ridge regression and its extension to generalized linear models" (2015). ETD Collection for University of Texas, El Paso. AAI1600357.
https://scholarworks.utep.edu/dissertations/AAI1600357