Optimized Learning Using Fuzzy-Inference-Assisted Algorithms for Deep Learning
Abstract
For years, researchers in Artificial Intelligence (AI) and Deep Learning (DL) observed that performance of a Deep Learning Network (DLN) could be improved by using larger and larger datasets coupled with complex network architectures. Although these strategies yield remarkable results, they have limits, dictated by data quantity and quality, rising costs by the increased computational power, or, more frequently, by long training times on networks that are very large. Training DLN requires laborious work involving multiple layers of densely connected neurons, updates to millions of network parameters, while potentially iterating thousands of times through millions of entries in a big dataset. Reducing DLN training time is an important challenge to address and it is the goal of this research. This study provides innovations at the learning algorithm level to improve the efficiency of the training process; specifically, it optimizes the Backpropagation (BP) algorithm by using fuzzy-inference assisted learning to reduce the number of required operations completed during the training phase while at the same time maintaining performance accuracy. The created two-phase fuzzy inference system is integrated into the BP algorithm to provide decision support, and when appropriate, utilize a speed-up technique of skipping training operations. The results for the proposed model trained with benchmark datasets show remarkable savings of up to 82%, effectively reducing the execution time and accomplishing the desired speedup, at times reaching convergence 600 epochs earlier than baseline case which provides considerable extra savings and optimizes training even further. Remarkably, FIL-BP model reaches same level of system error minimization as the traditional implementation; it achieves same high classification accuracy, and it improves generalization capability by averting unnecessary weight updates that result from overtraining. The speed-up of the training process provides savings that increase as the complexity, size, and challenge of the dataset increases.
Subject Area
Electrical engineering|Computer Engineering|Artificial intelligence
Recommended Citation
Barua, Miroslava, "Optimized Learning Using Fuzzy-Inference-Assisted Algorithms for Deep Learning" (2022). ETD Collection for University of Texas, El Paso. AAI30000222.
https://scholarworks.utep.edu/dissertations/AAI30000222