Incorporating clinical knowledge with constrained classifier chain into a multimodal deep network for melanoma detection

• Published in: Computers in biology and medicine Vol. 137; p. 104812
• Main Authors: Wang, Yuheng, Cai, Jiayue, Louie, Daniel C., Wang, Z. Jane, Lee, Tim K.
• Format: Journal Article
• Language: English
• Published: Oxford Elsevier Ltd 01.10.2021; Elsevier Limited
• Subjects: 7-Point checklist; Accuracy; Algorithms; Automation; Chains; Check lists; Classification; Classifiers; Constrained classifier chain; Criteria; Deep learning; Dermatology; Disease; Feature extraction; Knowledge; Learning algorithms; Machine learning; Medical imaging; Melanoma; Melanoma detection; Morphology; Multi-modality; Neural networks; Physicians; Principal components analysis; Skin cancer; Skin diseases; Smartphones; Melanoma detection; Constrained classifier chain; 7-Point checklist; Multi-modality
• ISSN: 0010-4825; 1879-0534
• DOI: 10.1016/j.compbiomed.2021.104812

In recent years, vast developments in Computer-Aided Diagnosis (CAD) for skin diseases have generated much interest from clinicians and other eventual end-users of this technology. Introducing clinical domain knowledge to these machine learning strategies can help dispel the black box nature of these tools, strengthening clinician trust. Clinical domain knowledge also provides new information channels which can improve CAD diagnostic performance. In this paper, we propose a novel framework for malignant melanoma (MM) detection by fusing clinical images and dermoscopic images. The proposed method combines a multi-labeled deep feature extractor and clinically constrained classifier chain (CC). This allows the 7-point checklist, a clinician diagnostic algorithm, to be included in the decision level while maintaining the clinical importance of the major and minor criteria in the checklist. Our proposed framework achieved an average accuracy of 81.3% for detecting all criteria and melanoma when testing on a publicly available 7-point checklist dataset. This is the highest reported results, outperforming state-of-the-art methods in the literature by 6.4% or more. Analyses also show that the proposed system surpasses the single modality system of using either clinical images or dermoscopic images alone and the systems without adopting the approach of multi-label and clinically constrained classifier chain. Our carefully designed system demonstrates a substantial improvement over melanoma detection. By keeping the familiar major and minor criteria of the 7-point checklist and their corresponding weights, the proposed system may be more accepted by physicians as a human-interpretable CAD tool for automated melanoma detection. •A novel method for melanoma detection is proposed based on clinically constrained classifier chain and deep learning.•Both dermoscopy and clinical images are considered to improve the diagnostic performance.•Clinical features are split into major and minor categories following the actual clinical diagnostic standards.•The results achieve state-of-the-art performance in the malignant melanoma and individual 7-point criteria detection.