Implicit Neural Network for Implicit Data Regression Problems

• Published in: Neural Information Processing pp. 187 - 195
• Main Authors: Miao, Zhibin, Zhong, Jinghui, Yang, Peng, Wang, Shibin, Liu, Dong
• Format: Book Chapter
• Language: English
• Published: Cham Springer International Publishing
• Series: Communications in Computer and Information Science
• Subjects: Data regression; Evolutionary algorithm; Implicit data regression; Neural network
• ISBN: 9783030923068; 3030923061
• ISSN: 1865-0929; 1865-0937
• DOI: 10.1007/978-3-030-92307-5_22

Artificial neural network (ANN) is one of the most common methods for data regression. However, existing ANN based methods focus on fitting data with explicit relationships, where the output y can be explicitly expressed by the inputs x in the form of y=f(x)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$y = f(x)$$\end{document}. In contrast, implicit relationships (i.e., f(x,y)=0\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$f(x,y)=0$$\end{document}) are more expressive in that they can concisely present complex closed surfaces and mathematical functions with multiple outputs. However, so far, little effort has been made on applying ANN to fit data with implicit relationships of variables. In this paper, for the first time, we propose an implicit neural network (INN) for implicit data regression. In this framework, an evolutionary implicit neural network (EINN) module is proposed, which is trained by the regression data to capture the implicit relationships among variables. Then, an explicit-implicit cooperate (EIC) mechanism is proposed based on the EINN component to train an explicit ANN model to predict the outputs of new unseen inputs. The proposed framework is tested on eight benchmark problems and the experimental results have demonstrated the efficacy of the proposed method.