Why Linear and Sigmoid Last Layers Work Better in Classification

Authors

Lehel Dénes-Fazakas, Óbuda UniversityFollow
Lásló Szilágyi, Sapientia UniversityFollow
Vladik Kreinovich, The University of Texas at El PasoFollow

Publication Date

3-1-2024

Comments

Technical Report: UTEP-CS-24-08

Abstract

Usually, when a deep neural network is used to classify objects, its last layer computes the softmax. Our empirical results show we can improve the classification results if instead, we have linear or sigmoid last layer. In this paper, we provide an explanation for this empirical phenomenon.