Smaller Standard Deviation for Initial Weights Improves Performance of Classifying Neural Networks: A Theoretical Explanation of Unexpected Simulation Results

Authors

Diego Aguirre, The University of Texas at El PasoFollow
Philip Hassoun, The University of Texas at El PasoFollow
Rafael Lopez, The University of Texas at El PasoFollow
Crystal Serrano, The University of Texas at El PasoFollow
Marcoantonio R. Soto, The University of Texas at El PasoFollow
Andrea Torres, The University of Texas at El PasoFollow
Vladik Kreinovich, The University of Texas at El PasoFollow

Publication Date

8-2019

Comments

Technical Report: UTEP-CS-19-93

Abstract

Numerical experiments show that for classifying neural networks, it is beneficial to select a smaller deviation for initial weights that what is usually recommended. In this paper, we provide a theoretical explanation for these unexpected simulation results.