Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators

• Published in: Nature machine intelligence Vol. 3; no. 3; pp. 218 - 229
• Main Authors: Lu, Lu, Jin, Pengzhan, Pang, Guofei, Zhang, Zhongqiang, Karniadakis, George Em
• Format: Journal Article
• Language: English
• Published: London Nature Publishing Group UK 01.03.2021; Nature Publishing Group; The Author(s), Springer Nature
• Subjects: 639/705/1041; 639/705/1042; Accuracy; Approximation; Artificial neural networks; Banach spaces; Complex systems; Computer Science; Continuity (mathematics); Deep learning; Differential equations; Dynamical systems; Engineering; Errors; Function space; Machine learning; Mathematical analysis; Neural networks; Operators (mathematics); Optimization; Ordinary differential equations; Partial differential equations; Robotics; Theorems
• ISSN: 2522-5839; 2522-5839
• DOI: 10.1038/s42256-021-00302-5

It is widely known that neural networks (NNs) are universal approximators of continuous functions. However, a less known but powerful result is that a NN with a single hidden layer can accurately approximate any nonlinear continuous operator. This universal approximation theorem of operators is suggestive of the structure and potential of deep neural networks (DNNs) in learning continuous operators or complex systems from streams of scattered data. Here, we thus extend this theorem to DNNs. We design a new network with small generalization error, the deep operator network (DeepONet), which consists of a DNN for encoding the discrete input function space (branch net) and another DNN for encoding the domain of the output functions (trunk net). We demonstrate that DeepONet can learn various explicit operators, such as integrals and fractional Laplacians, as well as implicit operators that represent deterministic and stochastic differential equations. We study different formulations of the input function space and its effect on the generalization error for 16 different diverse applications. Neural networks are known as universal approximators of continuous functions, but they can also approximate any mathematical operator (mapping a function to another function), which is an important capability for complex systems such as robotics control. A new deep neural network called DeepONet can lean various mathematical operators with small generalization error.