Publication Date
3-1-2022
Abstract
The main idea behind artificial neural networks is to simulate how data is processed in the data processing devoice that has been optimized by million-years natural selection -- our brain. Such networks are indeed very successful, but interestingly, the most recent successes came when researchers replaces the original biology-motivated sigmoid activation function with a completely different one -- known as rectified linear function. In this paper, we explain that this somewhat unexpected function actually naturally appears in physics-based data processing.
Comments
Technical Report: UTEP-CS-22-39