Two-stream collaborative network for multi-label chest X-ray Image classification with lung segmentation

• Published in: Pattern recognition letters Vol. 135; pp. 221 - 227
• Main Authors: Chen, Bingzhi, Zhang, Zheng, Lin, Jianyong, Chen, Yi, Lu, Guangming
• Format: Journal Article
• Language: English
• Published: Amsterdam Elsevier B.V 01.07.2020; Elsevier Science Ltd
• Subjects: Chest; Classification; Collaboration; Emphysema; Feature extraction; Image analysis; Image classification; Image processing; Image segmentation; Lung diseases; Lung segmentation; Multi-label CXR image classification; Optimization; Pneumothorax; Self-adaptive weighted fusion; Two-stream collaborative network; Two-stream collaborative network; Self-adaptive weighted fusion; 41A10; Lung segmentation; 65D05; 65D17; 41A05; Multi-label CXR image classification
• ISSN: 0167-8655; 1872-7344
• DOI: 10.1016/j.patrec.2020.04.016

•A dual-stream collaborative network framework is proposed.•The global and local modalities of the CXR images are jointly considered.•A novel adaptive weighting scheme is developed.•The importance of different modalities is self-tuned for robust disease analysis.•Extensive experiments demonstrate the effectiveness of the proposed learning scheme, yielding encouraging performance. Automated chest X-ray (CXR) image analysis is often subject to serious disruption and misguided by its imaging artifacts and noise regions. To minimize such negative effects, many state-of-the-art works have made great efforts in the precise segmentation of lung fields. Based on these works, some image-based features can be extracted directly from lung filed to provide clues for many types of lung diseases such as lung nodule, cardiomegaly, pneumothorax, or emphysema. In this paper, we propose a novel two-stream collaborative network call TSCN for multi-label CXR image classification based on lung segmentation. Specifically, we first train a robust lung segmentor with U-Net and apply it to capture the lung filed from the original CXR image. Then we perform a two-stream feature fusion operation to aggregate the contextual information in both the global image and lung field for complementary feature learning. By taking advantage of the two-stream deep structures and two types of image inputs, a novel self-adaptive weighted fusion scheme is designed to jointly learn these two feature streams in the end-to-end training phase, in order to realize adaptive two-stream feature subsets selection and optimization. Extensive experiments on the ChestX-ray14 dataset demonstrate the effectiveness of the proposed method as compared with the state-of-the-art baselines.