Generalized Oracle for Testing Machine Learning Computer Programs

• Published in: Software Engineering and Formal Methods Vol. 10729; pp. 174 - 179
• Main Author: Nakajima, Shin
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer International Publishing AG 2018; Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Improving Runtime Efficiencies; Input Training Dataset; Metamorphic Properties; Metamorphic Testing (MT); Test Corner Cases
• ISBN: 3319747800; 9783319747804
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-319-74781-1_13

Computation results of machine learning programs are not possible to be anticipated, because the results are sensitive to distribution of data in input dataset. Additionally, these computer programs sometimes adopt randomized algorithms for finding sub-optimal solutions or improving runtime efficiencies to reach solutions. The computation is probabilistic and the results vary from execution to execution even for a same input. The characteristics imply that no deterministic test oracle exists to check correctness of programs. This paper studies how a notion of oracles is elaborated so that these programs can be tested, and shows a systematic way of deriving testing properties from mathematical formulations of given machine learning problems.