Why Rectified Linear Unit is Efficient in Machine Learning: One More Explanation

Authors

Barnabas Bede, DigiPen Institute of TechnologyFollow
Vladik Kreinovich, The University of Texas at El PasoFollow
Uyen Pham, University of Economics and LawFollow

Publication Date

4-1-2022

Comments

Technical Report: UTEP-CS-22-49

Abstract

In many applications, in particular, in econometric application, deep learning techniques are very effective. In this paper, we provide a new explanation for why rectified linear units -- the main units of deep learning -- are so effective. This explanation is similar to the usual explanation of why Gaussian (normal) distributions are ubiquitous -- namely, it is based on an appropriate limit theorem.