Generalization bounds for learning under graph-dependence: a survey

• Published in: Machine learning Vol. 113; no. 7; pp. 3929 - 3959
• Main Authors: Zhang, Rui-Ray, Amini, Massih-Reza
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.07.2024; Springer Nature B.V; Springer Verlag
• Subjects: Algorithms; Artificial Intelligence; Cognitive tasks; Combinatorial analysis; Combinatorics; Computer Science; Control; Expected values; Graphs; Learning theory; Machine Learning; Mechatronics; Natural Language Processing (NLP); Random variables; Robotics; Simulation and Modeling; Statistical analysis; Generalization bounds; Dependency graphs; Bipartite ranking; Rademacher complexity; Uniform stability
• ISSN: 0885-6125; 1573-0565
• DOI: 10.1007/s10994-024-06536-9

Traditional statistical learning theory relies on the assumption that data are identically and independently distributed (i.i.d.). However, this assumption often does not hold in many real-life applications. In this survey, we explore learning scenarios where examples are dependent and their dependence relationship is described by a dependency graph , a commonly utilized model in probability and combinatorics. We collect various graph-dependent concentration bounds, which are then used to derive Rademacher complexity and stability generalization bounds for learning from graph-dependent data. We illustrate this paradigm through practical learning tasks and provide some research directions for future work. To our knowledge, this survey is the first of this kind on this subject.