Principle of Minimizing Empirical Risk and Averaging Aggregate Functions

• Published in: Journal of mathematical sciences (New York, N.Y.) Vol. 253; no. 4; pp. 583 - 598
• Main Author: Shibzukhov, Z. M.
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.03.2021; Springer; Springer Nature B.V
• Subjects: Iterative methods; Mathematics; Mathematics and Statistics; Optimization; Parameter estimation; Parameter robustness; Penalty function; Risk assessment; Robustness (mathematics); loss function; empirical risk; aggregation function; 68T05; iterative reweighing algorithm; averaging function
• ISSN: 1072-3374; 1573-8795
• DOI: 10.1007/s10958-021-05256-y

In this paper, we propose an extended version of the principle of minimizing empirical risk (ER) based on the use of averaging aggregating functions (AAF) for calculating the ER instead of the arithmetic mean. This is expedient if the distribution of losses has outliers and hence risk assessments are biased. Therefore, a robust estimate of the average risk should be used for optimization of the parameters. Such estimates can be constructed by using AAF that are solutions of the problem of minimizing the penalty function for deviating from the mean value. We also propose an iterative reweighting scheme for the numerical solution of the ER minimization problem. We give examples of constructing a robust procedure for estimating parameters in a linear regression problem and a linear separation problem for two classes based on the use of an averaging aggregating function that replaces the α -quantile.