Symbolic execution of complex program driven by machine learning based constraint solving

• Published in: 2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE) pp. 554 - 559
• Main Authors: Xin Li, Yongjuan Liang, Hong Qian, Yi-Qi Hu, Lei Bu, Yang Yu, Xin Chen, Xuandong Li
• Format: Conference Proceeding
• Language: English
• Published: ACM 01.09.2016
• Subjects: Algorithm design and analysis; Complicated Path Condition; Constraint Solving; Engines; Java; Libraries; Machine Learning; Machine learning algorithms; Optimization; Symbolic Execution; Transforms
• DOI: 10.1145/2970276.2970364

Symbolic execution is a widely-used program analysis technique. It collects and solves path conditions to guide the program traversing. However, due to the limitation of the current constraint solvers, it is difficult to apply symbolic execution on programs with complex path conditions, like nonlinear constraints and function calls. In this paper, we propose a new symbolic execution tool MLB to handle such problem. Instead of relying on the classical constraint solving, in MLB, the feasibility problems of the path conditions are transformed into optimization problems, by minimizing some dissatisfaction degree. The optimization problems are then handled by the underlying optimization solver through machine learning guided sampling and validation. MLB is implemented on the basis of Symbolic PathFinder and encodes not only the simple linear path conditions, but also nonlinear arithmetic operations, and even black-box function calls of library methods, into symbolic path conditions. Experiment results show that MLB can achieve much better coverage on complex real-world programs.