The lower tail of random quadratic forms with applications to ordinary least squares

• Published in: Probability theory and related fields Vol. 166; no. 3-4; pp. 1175 - 1194
• Main Author: Oliveira, Roberto Imbuzeiro
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.12.2016; Springer Nature B.V
• Subjects: Bayesian analysis; Economics; Estimating techniques; Estimators; Finance; Insurance; Least squares; Least squares method; Management; Mathematical analysis; Mathematical and Computational Biology; Mathematical and Computational Physics; Mathematics; Mathematics and Statistics; Matrices (mathematics); Matrix methods; Operations Research/Decision Theory; Optimization; Probability; Probability theory; Probability Theory and Stochastic Processes; Quadratic forms; Quantitative Finance; Random variables; Regression analysis; Sample size; Statistics for Business; Studies; Theoretical; 94A15; Random covariance matrices; 60F99; 62J05; Linear regression
• ISSN: 0178-8051; 1432-2064
• DOI: 10.1007/s00440-016-0738-9

Finite sample properties of random covariance-type matrices have been the subject of much research. In this paper we focus on the “lower tail” of such a matrix, and prove that it is sub-Gaussian under a simple fourth moment assumption on the one-dimensional marginals of the random vectors. A similar result holds for more general sums of random positive semidefinite matrices, and our (relatively simple) proof uses a variant of the so-called PAC-Bayesian method for bounding empirical processes. Using this bound, we obtain a nearly optimal finite-sample result for the ordinary least squares estimator under random design.