Optimal Teaching Curricula with Compositional Simplicity Priors

• Published in: Machine Learning and Knowledge Discovery in Databases. Research Track Vol. 12975; pp. 705 - 721
• Main Authors: Garcia-Piqueras, Manuel, Hernández-Orallo, José
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer International Publishing AG 2021; Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Interposition; Kolmogorov complexity; Machine teaching
• ISBN: 3030864855; 9783030864859
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-030-86486-6_43

Machine teaching under strong simplicity priors can teach any concept in universal languages. Remarkably, recent experiments suggest that the teaching sets are shorter than the concept description itself. This raises many important questions about the complexity of concepts and their teaching size, especially when concepts are taught incrementally. In this paper we put a bound to these surprising experimental findings and reconnect teaching size and concept complexity: complex concepts do require large teaching sets. Also, we analyse teaching curricula, and find a new interposition phenomenon: the teaching size of a concept can increase because examples are captured by simpler concepts built on previously acquired knowledge. We provide a procedure that not only avoids interposition but builds an optimal curriculum. These results indicate novel curriculum design strategies for humans and machines.