Solving and learning nonlinear PDEs with Gaussian processes

• Published in: Journal of computational physics Vol. 447; p. 110668
• Main Authors: Chen, Yifan, Hosseini, Bamdad, Owhadi, Houman, Stuart, Andrew M.
• Format: Journal Article
• Language: English
• Published: Cambridge Elsevier Inc 15.12.2021; Elsevier Science Ltd
• Subjects: Algorithms; Approximation; Burgers equation; Collocation; Collocation methods; Computational physics; Convergence; Eikonal equation; Gaussian process; Gaussian processes; Inverse problems; Kernel methods; Kernels; Newton methods; Nonlinear differential equations; Nonlinear partial differential equations; Optimal recovery; Optimization; Parameter identification; Partial differential equations; Kernel methods; Nonlinear partial differential equations; Optimal recovery; Inverse problems; Gaussian processes
• ISSN: 0021-9991; 1090-2716
• DOI: 10.1016/j.jcp.2021.110668

We introduce a simple, rigorous, and unified framework for solving nonlinear partial differential equations (PDEs), and for solving inverse problems (IPs) involving the identification of parameters in PDEs, using the framework of Gaussian processes. The proposed approach: (1) provides a natural generalization of collocation kernel methods to nonlinear PDEs and IPs; (2) has guaranteed convergence for a very general class of PDEs, and comes equipped with a path to compute error bounds for specific PDE approximations; (3) inherits the state-of-the-art computational complexity of linear solvers for dense kernel matrices. The main idea of our method is to approximate the solution of a given PDE as the maximum a posteriori (MAP) estimator of a Gaussian process conditioned on solving the PDE at a finite number of collocation points. Although this optimization problem is infinite-dimensional, it can be reduced to a finite-dimensional one by introducing additional variables corresponding to the values of the derivatives of the solution at collocation points; this generalizes the representer theorem arising in Gaussian process regression. The reduced optimization problem has the form of a quadratic objective function subject to nonlinear constraints; it is solved with a variant of the Gauss–Newton method. The resulting algorithm (a) can be interpreted as solving successive linearizations of the nonlinear PDE, and (b) in practice is found to converge in a small number of iterations (2 to 10), for a wide range of PDEs. Most traditional approaches to IPs interleave parameter updates with numerical solution of the PDE; our algorithm solves for both parameter and PDE solution simultaneously. Experiments on nonlinear elliptic PDEs, Burgers' equation, a regularized Eikonal equation, and an IP for permeability identification in Darcy flow illustrate the efficacy and scope of our framework. •A rigorous and unified framework for solving and learning nonlinear PDEs.•The proposed framework is based on techniques derived from Gaussian process regression and kernel methods.•It is provably convergent and inherits complexity vs accuracy guarantees of state of the art dense kernel matrix solvers.•It is interpretable and amenable to numerical analysis.