A Hybrid Regularized Multilayer Perceptron for Input Noise Immunity

• Published in: IEEE transactions on artificial intelligence Vol. 5; no. 1; pp. 115 - 126
• Main Authors: Mondal, Rahul, Pal, Tandra, Dey, Prasenjit
• Format: Journal Article
• Language: English
• Published: IEEE 01.01.2024
• Subjects: <named-content xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ali="http://www.niso.org/schemas/ali/1.0/" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" content-type="math" xlink:type="simple"> <inline-formula> <tex-math notation="LaTeX"> l_{2}</tex-math> </inline-formula> </named-content> regularizer; Additive noise; Artificial intelligence; DropConnect; dropout; Feature extraction; multilayer perceptron (MLP); Multilayer perceptrons; multiplicative noise; Neurons; noise immunity; noise injection; Noise measurement; outliers; Robustness
• ISSN: 2691-4581; 2691-4581
• DOI: 10.1109/TAI.2022.3225124

The immunity of multilayer perceptron (MLP) is less effective toward input noise. In this article, we have focused on the robustness of MLP with respect to input noise where noise can be additive or multiplicative. Here, we have proposed a DropConnect-based regularized MLP to reduce the coadaptation among the neurons of the hidden layer. At first, we have empirically and statistically shown that by reducing coadaptation among the hidden neurons, an MLP can achieve better noise immunity. We have also empirically shown that an MLP with input noise injection and <inline-formula><tex-math notation="LaTeX">l_{2}</tex-math></inline-formula> regularizer is an effective approach to improve its noise immunity. However, the results indicate that it does not adjust the coadaptation among the hidden neurons. Therefore, for further improvement, we have proposed a hybrid regularized MLP (HRMLP), where DropConnect is combined with the noise injection and <inline-formula><tex-math notation="LaTeX">l_{2}</tex-math></inline-formula> regularizer. In addition to input noise, we have also verified the robustness of HRMLP with respect to <inline-formula><tex-math notation="LaTeX">\mathbf {20\%}</tex-math></inline-formula> outliers in the dataset. To justify the effectiveness of the proposed HRMLP, we have compared it with MLP, MLP with noise injection, MLP with <inline-formula><tex-math notation="LaTeX">l_{2}</tex-math></inline-formula> regularizer, MLP with noise injection and <inline-formula><tex-math notation="LaTeX">l_{2}</tex-math></inline-formula> regularizer, and MLP with DropConnect along with two state-of-the-art works based on <inline-formula><tex-math notation="LaTeX">\mathbf {20}</tex-math></inline-formula> standard datasets. The experimental results for both noisy inputs and outliers confirm that the performance of HRMLP is significant compared to other methods.