General risk measures for robust machine learning

• Published in: Foundations of data science Vol. 1; no. 3; pp. 249 - 269
• Main Authors: Chouzenoux, Émilie, Gérard, Henri, Pesquet, Jean-Christophe
• Format: Journal Article
• Language: English
• Published: American Institute of Mathematical Sciences 01.09.2019
• Subjects: Machine Learning; Mathematics; Optimization and Control; Statistics; convex optimization; Wasserstein distance; machine learning; divergences; Risk measures; robust statistics
• ISSN: 2639-8001; 2639-8001
• DOI: 10.3934/fods.2019011

A wide array of machine learning problems are formulated as the minimization of the expectation of a convex loss functionon some parameter space. Since the probability distribution of the data of interest is usually unknown, it is is often estimated from training sets, which may lead to poor out-of-sample performance. In this work, we bring new insights in this problem by using the framework which has been developed in quantitative finance for risk measures. We show that the original min-max problem can be recast asa convex minimization problem under suitable assumptions. We discuss several important examples ofrobust formulations, in particular by defining ambiguity sets based on $\varphi$-divergences and the Wasserstein metric.We also propose an efficient algorithm for solving the corresponding convex optimization problems involving complex convex constraints. Through simulation examples, we demonstrate that this algorithm scales wellon real data sets.