Leveraging Large Language Models for Enhancing the Understandability of Generated Unit Tests

• Published in: Proceedings / International Conference on Software Engineering pp. 1449 - 1461
• Main Authors: Deljouyi, Amirhossein, Koohestani, Roham, Izadi, Maliheh, Zaidman, Andy
• Format: Conference Proceeding
• Language: English
• Published: IEEE 26.04.2025
• Subjects: Automated Test Generation; Computer bugs; Generators; Industries; Large language models; Readability; Retrieval augmented generation; Software; Software engineering; Software testing; Test pattern generators; Understandability; Unit Testing; Writing
• ISSN: 1558-1225
• DOI: 10.1109/ICSE55347.2025.00032

Automated unit test generators, particularly searchbased software testing tools like EvoSuite, are capable of generating tests with high coverage. Although these generators alleviate the burden of writing unit tests, they often pose challenges for software engineers in terms of understanding the generated tests. To address this, we introduce UTGen, which combines searchbased software testing and large language models to enhance the understandability of automatically generated test cases. We achieve this enhancement through contextualizing test data, improving identifier naming, and adding descriptive comments. Through a controlled experiment with 32 participants from both academia and industry, we investigate how the understandability of unit tests affects a software engineer's ability to perform bug-fixing tasks. We selected bug-fixing to simulate a real-world scenario that emphasizes the importance of understandable test cases. We observe that participants working on assignments with UTGen test cases fix up to 33 % more bugs and use up to 20 % less time when compared to baseline test cases. From the post-test questionnaire, we gathered that participants found that enhanced test names, test data, and variable names improved their bugfixing process.