Optimal weighted least-squares methods

• Published in: SMAI Journal of Computational Mathematics Vol. 3; pp. 181 - 203
• Main Authors: Cohen, Albert, Migliorati, Giovanni
• Format: Journal Article
• Language: English
• Published: Société de Mathématiques Appliquées et Industrielles (SMAI) 2017
• Subjects: Mathematics; Numerical Analysis; ran-dom matrices; error analysis; weighted least squares; polynomial approximation Math classification 41A10; multivariate approximation; convergence rates; conditional sampling
• ISSN: 2426-8399; 2426-8399
• DOI: 10.5802/smai-jcm.24

We consider the problem of reconstructing an unknown bounded function u defined on a domain X ⊂ R d from noiseless or noisy samples of u at n points (x i)i=1,...,n. We measure the reconstruction error in a norm L 2 (X, dρ) for some given probability measure dρ. Given a linear space Vm with dim(Vm) = m ≤ n, we study in general terms the weighted least-squares approximations from the spaces Vm based on independent random samples. It is well known that least-squares approximations can be inaccurate and unstable when m is too close to n, even in the noiseless case. Recent results from [4, 5] have shown the interest of using weighted least squares for reducing the number n of samples that is needed to achieve an accuracy comparable to that of best approximation in Vm, compared to standard least squares as studied in [3]. The contribution of the present paper is twofold. From the theoretical perspective, we establish results in expectation and in probability for weighted least squares in general approximation spaces Vm. These results show that for an optimal choice of sampling measure dµ and weight w, which depends on the space Vm and on the measure dρ, stability and optimal accuracy are achieved under the mild condition that n scales linearly with m up to an additional logarithmic factor. In contrast to [3], the present analysis covers cases where the function u and its approximants from Vm are unbounded, which might occur for instance in the relevant case where X = R d and dρ is the Gaussian measure. From the numerical perspective, we propose a sampling method which allows one to generate independent and identically distributed samples from the optimal measure dµ. This method becomes of interest in the multivariate setting where dµ is generally not of tensor product type. We illustrate this for particular examples of approximation spaces Vm of polynomial type, where the domain X is allowed to be unbounded and high or even infinite dimensional, motivated by certain applications to parametric and stochastic PDEs.