Cauchy noise loss for stochastic optimization of random matrix models via free deterministic equivalents

• Published in: Journal of mathematical analysis and applications Vol. 483; no. 2; p. 123597
• Main Author: Hayase, Tomohiro
• Format: Journal Article
• Language: English
• Published: Elsevier Inc 15.03.2020
• Subjects: Dimensionality recovery; Free probability theory; Random matrix theory; Rank estimation; Stochastic optimization; Free probability theory; Dimensionality recovery; Random matrix theory; Stochastic optimization; Rank estimation
• ISSN: 0022-247X; 1096-0813
• DOI: 10.1016/j.jmaa.2019.123597

For random matrix models, the parameter estimation based on the traditional likelihood functions is not straightforward in particular when we have only one sample matrix. We introduce a new parameter optimization method for random matrix models which works even in such a case. The method is based on the spectral distribution instead of the traditional likelihood. In the method, the Cauchy noise has an essential role because the free deterministic equivalent, which is a tool in free probability theory, allows us to approximate the spectral distribution perturbed by Cauchy noises by a smooth and accessible density function. Moreover, we study an asymptotic property of determination gap, which has a similar role as generalization gap. Besides, we propose a new dimensionality recovery method for the signal-plus-noise model, and experimentally demonstrate that it recovers the rank of the signal part even if the true rank is not small. It is a simultaneous rank selection and parameter estimation procedure.