The Double Descent Behavior in Two Layer Neural Network for Binary Classification

• Published in: Journal of Data Science Vol. 23; no. 2; pp. 370 - 388
• Main Authors: Abeykoon, Chathurika S., Beknazaryan, Aleksandr, Sang, Hailin
• Format: Journal Article
• Language: English
• Published: 中華資料採礦協會 01.04.2025
• Subjects: over and under parameterization; ReLU activation; testing error; generalization error; model complexity

Recent studies observed a surprising concept on model test error called the double descent phenomenon where the increasing model complexity decreases the test error first and then the error increases and decreases again. To observe this, we work on a two-layer neural network model with a ReLU activa...