DSKIPP: A Prompt Method to Enhance the Reliability in LLMs for Java API Recommendation Task

• Published in: Software testing, verification & reliability Vol. 35; no. 2
• Main Authors: Yang, Jingbo, Wu, Wenjun, Ren, Jian
• Format: Journal Article
• Language: English
• Published: Chichester Wiley Subscription Services, Inc 01.03.2025
• Subjects: Ablation; API recommendation; Application programming interface; DSKIPP; Java; Large language models; LLM; Programming environments; reliability; Software development; Software reliability
• ISSN: 0960-0833; 1099-1689
• DOI: 10.1002/stvr.1913

ABSTRACT In the realm of software development, selecting the appropriate Java application programming interfaces (APIs) from a vast pool remains a significant challenge for developers. This research addresses this complexity by tackling the limitations of current API recommendation methods, which often struggle to align API suggestions with the specific queries and development contexts. In this paper, we introduce a novel prompt method named DSKIPP (Development Scenario, key Knowledge and Intention's Progressive Prompt), designed to enhance the efficiency of large language models (LLMs) in Java API recommendations. Firstly, we devise an overview of DSKIPP which conducts LLMs through a sequential process: first, inferring the package level, followed by the class level, and ultimately the method level as an API comprises three distinct components at varying levels—package, class and method. Secondly, at each level, DSKIPP assists LLMs in deducing the development scenario associated with a query and the essential key knowledge relevant to that scenario. This approach enables LLMs to gain a more profound contextual understanding of the query's intention. Moreover, during the inference process at the class and method level, we implement a self‐check mechanism enabling LLMs to validate the results and ensure a more reasoned and reliable outcome. To validate the efficiency of DSKIPP, comparison and ablation experiments are both conducted within Java programming environment. The comparison results affirm that our method outperforms the current state‐of‐the‐art technologies in API recommendation tasks, while the ablation results shed light on why DSKIPP can enhance the reliability of API recommendations in LLMs. This research contributes to the field by offering a more reliable and context‐sensitive solution for API recommendation in software development. The Development Scenario, key Knowledge and Intention's Progressive Prompt (DSKIPP) first infers the package level, followed by the class level, and ultimately the method level, as an application programming interface (API) comprises three distinct components at varying levels—package, class and method. By sequentially addressing these levels, the method can obtain a comprehensive understanding of an API query and more accurately recommend appropriate APIs.