Why Geometric Progression in Selecting the LASSO Parameter: A Theoretical Explanation

Authors

William Kubin, The University of Texas at El PasoFollow
Yi Xie, The University of Texas at El PasoFollow
Laxman Bokati, The University of Texas at El PasoFollow
Vladik Kreinovich, The University of Texas at El PasoFollow
Kittawit Autchariyapanitkul, Maijo UniversityFollow

Publication Date

4-2020

Comments

Technical Report: UTEP-CS-20-35

Abstract

In situations when we know which inputs are relevant, the least squares method is often the best way to solve linear regression problems. However, in many practical situations, we do not know beforehand which inputs are relevant and which are not. In such situations, a 1-parameter modification of the least squares method known as LASSO leads to more adequate results. To use LASSO, we need to determine the value of the LASSO parameter that best fits the given data. In practice, this parameter is determined by trying all the values from some discrete set. It has been empirically shown that this selection works the best if we try values from a geometric progression. In this paper, we provide a theoretical explanation for this empirical fact.