Why Deep Learning Is Under-Determined? Why Usual Numerical Methods for Solving Partial Differential Equations Do Not Preserve Energy? The Answers May Be Related to Chevalley-Warning Theorem (and Thus to Fermat Last Theorem)
In this paper, we provide a possible explanation to two seemingly unrelated phenomena: (1) that in deep learning, under-determined systems of equations perform much better than the over-determined one -- which are typical in data processing, and that (2) usual numerical methods for solving partial differential equations do not preserve energy. Our explanation is related to the intuition of Fermat behind his Last Theorem and of Euler about more general statements, intuition that led to the proof of Chevalley-Warning Theorem in number theory.