DocChecker: Bootstrapping Code Large Language Model for Detecting and Resolving Code-Comment Inconsistencies
EACL 2024 - Demonstration track Comments within source code are essential for developers to comprehend the code's purpose and ensure its correct usage. However, as codebases evolve, maintaining an accurate alignment between the comments and the code becomes increasingly challenging. Recognizing...
Saved in:
Main Authors | , , |
---|---|
Format | Journal Article |
Language | English |
Published |
10.06.2023
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | EACL 2024 - Demonstration track Comments within source code are essential for developers to comprehend the
code's purpose and ensure its correct usage. However, as codebases evolve,
maintaining an accurate alignment between the comments and the code becomes
increasingly challenging. Recognizing the growing interest in automated
solutions for detecting and correcting differences between code and its
accompanying comments, current methods rely primarily on heuristic rules. In
contrast, this paper presents DocChecker, a tool powered by deep learning.
DocChecker is adept at identifying inconsistencies between code and comments,
and it can also generate synthetic comments. This capability enables the tool
to detect and correct instances where comments do not accurately reflect their
corresponding code segments. We demonstrate the effectiveness of DocChecker
using the Just-In-Time and CodeXGlue datasets in different settings.
Particularly, DocChecker achieves a new State-of-the-art result of 72.3%
accuracy on the Inconsistency Code-Comment Detection (ICCD) task and 33.64
BLEU-4 on the code summarization task against other Large Language Models
(LLMs), even surpassing GPT 3.5 and CodeLlama.
DocChecker is accessible for use and evaluation. It can be found on our
GitHub https://github.com/FSoft-AI4Code/DocChecker and as an Online Tool
http://4.193.50.237:5000/. For a more comprehensive understanding of its
functionality, a demonstration video is available on YouTube
https://youtu.be/FqnPmd531xw. |
---|---|
DOI: | 10.48550/arxiv.2306.06347 |