Large scale tissue histopathology image classification, segmentation, and visualization via deep convolutional activation features

• Published in: BMC bioinformatics Vol. 18; no. 1; p. 281
• Main Authors: Xu, Yan, Jia, Zhipeng, Wang, Liang-Bo, Ai, Yuqing, Zhang, Fang, Lai, Maode, Chang, Eric I-Chao
• Format: Journal Article
• Language: English
• Published: England BioMed Central 26.05.2017; BMC
• Subjects: Activation analysis; Algorithms; Automation; Brain; Brain cancer; Brain Neoplasms - diagnosis; Brain Neoplasms - pathology; Brain tumors; Cancer; Carcinoma - diagnosis; Carcinoma - pathology; Classification; Colon; Colon cancer; Colonic Neoplasms - diagnosis; Colonic Neoplasms - pathology; Datasets; Deep convolution activation feature; Deep learning; Design; Diagnosis; Digital imaging; Feature extraction; Feature learning; Feature recognition; Histopathology; Humans; Image analysis; Image classification; Image processing; Image Processing, Computer-Assisted - methods; Image segmentation; Knowledge management; Methodology; Neural networks; Neural Networks, Computer; Object recognition; Pathology; Representations; Segmentation; State of the art; Support Vector Machine; Training; Visualization; Working conditions; Workload; Deep learning; Feature learning; Segmentation; Classification; Deep convolution activation feature
• ISSN: 1471-2105; 1471-2105
• DOI: 10.1186/s12859-017-1685-x

Histopathology image analysis is a gold standard for cancer recognition and diagnosis. Automatic analysis of histopathology images can help pathologists diagnose tumor and cancer subtypes, alleviating the workload of pathologists. There are two basic types of tasks in digital histopathology image analysis: image classification and image segmentation. Typical problems with histopathology images that hamper automatic analysis include complex clinical representations, limited quantities of training images in a dataset, and the extremely large size of singular images (usually up to gigapixels). The property of extremely large size for a single image also makes a histopathology image dataset be considered large-scale, even if the number of images in the dataset is limited. In this paper, we propose leveraging deep convolutional neural network (CNN) activation features to perform classification, segmentation and visualization in large-scale tissue histopathology images. Our framework transfers features extracted from CNNs trained by a large natural image database, ImageNet, to histopathology images. We also explore the characteristics of CNN features by visualizing the response of individual neuron components in the last hidden layer. Some of these characteristics reveal biological insights that have been verified by pathologists. According to our experiments, the framework proposed has shown state-of-the-art performance on a brain tumor dataset from the MICCAI 2014 Brain Tumor Digital Pathology Challenge and a colon cancer histopathology image dataset. The framework proposed is a simple, efficient and effective system for histopathology image automatic analysis. We successfully transfer ImageNet knowledge as deep convolutional activation features to the classification and segmentation of histopathology images with little training data. CNN features are significantly more powerful than expert-designed features.