Why Max and Average Poolings are Optimal in Convolutional Neural Networks

Authors

Ahnaf Farhan, The University of Texas at El PasoFollow
Olga Kosheleva, The University of Texas at El PasoFollow
Vladik Kreinovich, The University of Texas at El PasoFollow

Publication Date

9-2018

Comments

Technical Report: UTEP-CS-18-65

Abstract

In many practical situations, we do not know the exact relation between different quantities; this relation needs to be determined based on the empirical data. This determination is not easy -- especially in the presence of different types of uncertainty. When the data comes in the form of time series and images, many efficient techniques for such determination use algorithms for training convolutional neural network. As part of this training, such networks "pool" several values corresponding to nearby temporal or spatial points into a single value. Empirically, the most efficient pooling algorithm consists of taking the maximum of the pooled values; the next optimal is taking the arithmetic mean. In this paper, we provide a theoretical explanation for this empirical optimality.