Foundational Property-Based Testing

• Published in: Interactive Theorem Proving pp. 325 - 343
• Main Authors: Paraskevopoulou, Zoe, Hriţcu, Cătălin, Dénès, Maxime, Lampropoulos, Leonidas, Pierce, Benjamin C.
• Format: Book Chapter
• Language: English
• Published: Cham Springer International Publishing 19.08.2015
• Series: Lecture Notes in Computer Science
• ISBN: 9783319221014; 3319221019
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-319-22102-1_22

Integrating property-based testing with a proof assistant creates an interesting opportunity: reusable or tricky testing code can be formally verified using the proof assistant itself. In this work we introduce a novel methodology for formally verified property-based testing and implement it as a foundational verification framework for QuickChick, a port of QuickCheck to Coq. Our framework enables one to verify that the executable testing code is testing the right Coq property. To make verification tractable, we provide a systematic way for reasoning about the set of outcomes a random data generator can produce with non-zero probability, while abstracting away from the actual probabilities. Our framework is firmly grounded in a fully verified implementation of QuickChick itself, using the same underlying verification methodology. We also apply this methodology to a complex case study on testing an information-flow control abstract machine, demonstrating that our verification methodology is modular and scalable and that it requires minimal changes to existing code.