Why Deep Learning Is More Efficient than Support Vector Machines, and How It Is Related to Sparsity Techniques in Signal Processing

Authors

Laxman Bokati, The University of Texas at El PasoFollow
Olga Kosheleva, The University of Texas at El PasoFollow
Vladik Kreinovich, The University of Texas at El PasoFollow

Publication Date

11-2019

Comments

Technical Report: UTEP-CS-19-104b

To appear in: Proceedings of the 2020 4th International Conference on Intelligent Systems, Metaheuristics & Swarm Intelligence ISMSI'2020, Thimpu, Bhutan, March 21-22, 2020.

Abstract

Several decades ago, traditional neural networks were the most efficient machine learning technique. Then it turned out that, in general, a different technique called support vector machines is more efficient. Reasonably recently, a new technique called deep learning has been shown to be the most efficient one. These are empirical observations, but how we explain them -- thus making the corresponding conclusions more reliable? In this paper, we provide a possible theoretical explanation for the above-described empirical comparisons. This explanation enables us to explain yet another empirical fact -- that sparsity techniques turned out to be very efficient in signal processing.