Why Kolmolgorov-Arnold Networks (KAN) Work So Well: A Qualitative Explanation

Authors

Hung T. Nguyen, New Mexico State UniversityFollow
Vladik Kreinovich, University of Texas at El PasoFollow
Olga Kosheleva, University of Texas at El PasoFollow

Publication Date

9-1-2024

Comments

Technical Report: UTEP-CS-24-47

Abstract

In the usual deep neural network, weights are adjusted during training, but the activation function remains the same. Lately, it was experimentally shown that if, instead of using the same activation function always, we train the activation functions as well, we get a much better results -- i.e., for the networks with the same number of parameters, we get a much better accuracy. Such networks are called Kolmogorov-Arnold networks. In this paper, we provide a general explanation of why these new networks work so well.