Programming with a read-eval-synth loop

• Published in: Proceedings of ACM on programming languages Vol. 4; no. OOPSLA; pp. 1 - 30
• Main Authors: Peleg, Hila, Gabay, Roi, Itzhaky, Shachar, Yahav, Eran
• Format: Journal Article
• Language: English
• Published: New York, NY, USA ACM 13.11.2020
• Subjects: Automatic programming; Compilers; Software and its engineering; Software creation and management; Software development techniques; Software notations and tools; Source code generation; program synthesis; specification predicates; read-eval-print loops
• ISSN: 2475-1421; 2475-1421
• DOI: 10.1145/3428227

A frequent programming pattern for small tasks, especially expressions, is to repeatedly evaluate the program on an input as its editing progresses. The Read-Eval-Print Loop (REPL) interaction model has been a successful model for this programming pattern. We present the new notion of Read-Eval-Synth Loop (RESL) that extends REPL by providing in-place synthesis on parts of the expression marked by the user. RESL eases programming by synthesizing parts of a required solution. The underlying synthesizer relies on a partial solution from the programmer and a few examples. RESL hinges on bottom-up synthesis with general predicates and sketching, generalizing programming by example. To make RESL practical, we present a formal framework that extends observational equivalence to non-example specifications. We evaluate RESL by conducting a controlled within-subjects user-study on 19 programmers from 8 companies, where programmers are asked to solve a small but challenging set of competitive programming problems. We find that programmers using RESL solve these problems with far less need to edit the code themselves and by browsing documentation far less. In addition, they are less likely to leave a task unfinished and more likely to be correct.