Which Activation Function Works Best for Training Artificial Pancreas: Empirical Fact and Its Theoretical Explanation

Authors

Lehel Dénes-Fazakas, Óbuda UniversityFollow
Lásló Szilágyi, Sapientia UniversityFollow
György Eigner, Óbuda UniversityFollow
Olga Kosheleva, The University of Texas at El PasoFollow
Martine Ceberio, The University of Texas at El PasoFollow
Vladik Kreinovich, The University of Texas at El PasoFollow

Publication Date

6-1-2023

Comments

Technical Report: UTEP-CS-23-25

Abstract

One of the most effective ways to help patients at the dangerous levels of diabetes is an artificial pancreas, a device that constantly monitors the patient's blood sugar level and injects insulin based on this level. Patient's reaction to insulin is highly individualized, so the artificial pancreas needs to be trained on each patient. It turns out that the best training results are attained when instead of the usual ReLU neurons, we use their minor modification known as Exponential Linear Units (ELU). In this paper, we provide a theoretical explanation for the empirically observed effectiveness of ELUs.