Navigating the Testing of Evolving Deep Learning Systems: An Exploratory Interview Study

Deep Learning (DL) systems have been widely adopted across various industrial domains such as autonomous driving and intelligent healthcare. As with traditional software, DL systems also need to constantly evolve to meet ever-changing user requirements. However, ensuring the quality of these continu...

Full description

Saved in:
Bibliographic Details
Published inProceedings / International Conference on Software Engineering pp. 2726 - 2738
Main Authors You, Hanmo, Wang, Zan, Lin, Bin, Chen, Junjie
Format Conference Proceeding
LanguageEnglish
Published IEEE 26.04.2025
Subjects
Deep learning
Faces
Fault diagnosis
Industries
Interview Study
Interviews
Medical services
Navigation
Software
Software engineering
Software Evolution
Testing
Online AccessGet full text

Cover

More Information
Summary:Deep Learning (DL) systems have been widely adopted across various industrial domains such as autonomous driving and intelligent healthcare. As with traditional software, DL systems also need to constantly evolve to meet ever-changing user requirements. However, ensuring the quality of these continuously evolving systems presents significant challenges, especially in the context of testing. Understanding how industry developers address these challenges and what extra obstacles they are facing could provide valuable insights for further safeguarding the quality of DL systems. To reach this goal, we conducted semi-structured interviews with 22 DL developers from diverse domains and backgrounds. More specifically, our study focuses on exploring the challenges developers encounter in testing evolving DL systems, the practical solutions they employ, and their expectations for extra support. Our results highlight the difficulties in testing evolving DL systems (e.g., regression faults, online-offline differences, and test data collection) and identify the best practices for D L developers to address these challenges. Additionally, we pinpoint potential future research directions to enhance testing effectiveness in evolving DL systems.
ISSN:1558-1225
DOI:10.1109/ICSE55347.2025.00106