A comparative investigation of neural networks in solving differential equations

• Published in: Journal of algorithms & computational technology Vol. 15
• Main Authors: Shi, Enze, Xu, Chuanju
• Format: Journal Article
• Language: English
• Published: London, England SAGE Publications 01.03.2021; Sage Publications Ltd; SAGE Publishing
• Subjects: Approximation; Boundary conditions; Differential equations; Neural networks; Polynomials; Training; loss function; parameter sensitivity; Neural networks; differential equations
• ISSN: 1748-3018; 1748-3026
• DOI: 10.1177/1748302621998605

Methods for solving differential equations based on neural networks have been widely proposed in recent years. However, limited open literature to date has reported the choice of loss functions and the hyperparameters of the network and how it influences the quality of numerical solutions. In the present work we intend to address this issue. Precisely we will focus on possible choices of loss functions and compare their efficiency in solving differential equations through a series of numerical experiments. In particular, a comparative investigation is performed between the natural neural networks and Ritz neural networks, with and without penalty for the boundary conditions. The sensitivity on the accuracy of the neural networks with respect to the size of training set, the number of nodes, and the penalty parameter is also studied. In order to better understand the training behavior of the proposed neural networks, we further investigate the approximation properties of the neural networks in function fitting. A particular attention is paid to approximating Müntz polynomials by neural networks.