Visual interpretability for deep learning: a survey

• Published in: Frontiers of information technology & electronic engineering Vol. 19; no. 1; pp. 27 - 39
• Main Authors: Zhang, Quan-shi, Zhu, Song-chun
• Format: Journal Article
• Language: English
• Published: Hangzhou Zhejiang University Press 01.01.2018; Springer Nature B.V
• Subjects: Annotations; Artificial intelligence; Artificial neural networks; Communications Engineering; Computer Hardware; Computer Science; Computer Systems Organization and Communication Networks; Deep learning; Electrical Engineering; Electronics and Microelectronics; Explainable artificial intelligence; Instrumentation; Machine learning; Networks; Neural networks; Representations; Review; Semantics; Visualization; Deep learning; Interpretable model; TP391; Artificial intelligence
• ISSN: 2095-9184; 2095-9230
• DOI: 10.1631/FITEE.1700808

This paper reviews recent studies in understanding neural-network representations and learning neural networks with interpretable/disentangled middle-layer representations. Although deep neural networks have exhibited superior performance in various tasks, interpretability is always Achilles’ heel of deep neural networks. At present, deep neural networks obtain high discrimination power at the cost of a low interpretability of their black-box representations. We believe that high model interpretability may help people break several bottlenecks of deep learning, e.g., learning from a few annotations, learning via human–computer communications at the semantic level, and semantically debugging network representations. We focus on convolutional neural networks (CNNs), and revisit the visualization of CNN representations, methods of diagnosing representations of pre-trained CNNs, approaches for disentangling pre-trained CNN representations, learning of CNNs with disentangled representations, and middle-to-end learning based on model interpretability. Finally, we discuss prospective trends in explainable artificial intelligence.