DART: An automated end-to-end object detection pipeline with data Diversification, open-vocabulary bounding box Annotation, pseudo-label Review, and model Training

• Published in: Expert systems with applications Vol. 258; p. 125124
• Main Authors: Xin, Chen, Hartel, Andreas, Kasneci, Enkelejda
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 15.12.2024
• Subjects: Data diversification; Large multimodal model (LMM); Open-vocabulary object detection (OVD); Pseudo-label; Stable diffusion; YOLO; YOLO; Large multimodal model (LMM); Data diversification; Stable diffusion; Open-vocabulary object detection (OVD); Pseudo-label
• ISSN: 0957-4174
• DOI: 10.1016/j.eswa.2024.125124

Accurate real-time object detection is vital across numerous industrial applications, from safety monitoring to quality control. Traditional approaches, however, are hindered by arduous manual annotation and data collection, struggling to adapt to ever-changing environments and novel target objects. To address these limitations, this paper presents DART, an innovative automated end-to-end pipeline that revolutionizes object detection workflows from data collection to model evaluation. It eliminates the need for laborious human labeling and extensive data collection while achieving outstanding accuracy across diverse scenarios. DART encompasses four key stages: (1) Data Diversification using subject-driven image generation (DreamBooth with SDXL), (2) Annotation via open-vocabulary object detection (Grounding DINO) to generate bounding box and class labels, (3) Review of generated images and pseudo-labels by large multimodal models (InternVL-1.5 and GPT-4o) to guarantee credibility, and (4) Training of real-time object detectors (YOLOv8 and YOLOv10) using the verified data. We apply DART to a self-collected dataset of construction machines named Liebherr Product, which contains over 15K high-quality images across 23 categories. The current instantiation of DART significantly increases average precision (AP) from 0.064 to 0.832. Its modular design ensures easy exchangeability and extensibility, allowing for future algorithm upgrades, seamless integration of new object categories, and adaptability to customized environments without manual labeling and additional data collection. The code and dataset are released at https://github.com/chen-xin-94/DART. •DART, an automated end-to-end object detection pipeline without manual labeling.•DART streamlines data diversification, annotation, review, and training stages.•DART boosts AP from 0.064 to 0.832 on a collected dataset of construction machines.•DART’s modular design ensures flexibility, robustness, and customization.•Code and dataset are made publicly available for further research.