LLM4TDD: Best Practices for Test Driven Development Using Large Language Models

In today's society, we are becoming increasingly dependent on software systems. However, we also constantly witness the negative impacts of buggy software. Program synthesis aims to improve software correctness by automatically generating the program given an outline of the expected behavior. F...

Full description

Saved in:
Bibliographic Details
Published in2024 IEEE/ACM International Workshop on Large Language Models for Code (LLM4Code) pp. 14 - 21
Main Authors Piya, Sanyogita, Sullivan, Allison
Format Conference Proceeding
LanguageEnglish
Published ACM 20.04.2024
Subjects
Best practices
Chatbots
Codes
Conferences
Encoding
Large language models
LLM
Software development management
Software systems
Test-driven development
Online AccessGet full text
DOI10.1145/3643795.3648382

Cover

More Information
Summary:In today's society, we are becoming increasingly dependent on software systems. However, we also constantly witness the negative impacts of buggy software. Program synthesis aims to improve software correctness by automatically generating the program given an outline of the expected behavior. For decades, program synthesis has been an active research field, with recent approaches looking to incorporate Large Language Model. This paper explores the concept of LLM4TDD, where we guide Large Language Models to generate code iteratively using a test-driven development methodology. We conduct an empirical evaluation using ChatGPT and coding problems from LeetCode to investigate the impact of different test, prompt and problem attributes on the efficacy of LLM4TDD.CCS CONCEPTS* Software and its engineering → Software development techniques.
DOI:10.1145/3643795.3648382