Concept-based AI interpretability in physiological time-series data: Example of abnormality detection in electroencephalography

• Published in: Computer methods and programs in biomedicine Vol. 257; p. 108448
• Main Authors: Brenner, Alexander, Knispel, Felix, Fischer, Florian P., Rossmanith, Peter, Weber, Yvonne, Koch, Henner, Röhrig, Rainer, Varghese, Julian, Kutafina, Ekaterina
• Format: Journal Article
• Language: English
• Published: Ireland Elsevier B.V 01.12.2024
• Subjects: Artificial neural networks; Decision support systems; Electroencephalography; Explainable artificial intelligence; Supervised learning; TCAV; Decision support systems; Explainable artificial intelligence; Electroencephalography; TCAV; Supervised learning; Artificial neural networks
• ISSN: 0169-2607; 1872-7565; 1872-7565
• DOI: 10.1016/j.cmpb.2024.108448

•Testing with concept activation vectors (TCAV) employs concept-based explanations of deep neural networks built for the classification of electroencephalography (EEG) data.•TCAV analysis confirms the medical expectation with high concept sensitivity towards typical epileptogenic signal patterns for the classification of abnormal EEGs.•Concept samples can be created from labeled signal sections, using metadata, as well as by extracting or simulating signal characteristics in the form of frequencies.•TCAV analysis yields consistent results for concept samples from different sources. Despite recent performance advancements, deep learning models are not yet adopted in clinical practice on a wide scale. The intrinsic intransparency of such systems is commonly cited as one major reason for this reluctance. This has motivated methods that aim to provide explanations of model functioning. Known limitations of feature-based explanations have led to an increased interest in concept-based interpretability. Testing with Concept Activation Vectors (TCAV) employs human-understandable, abstract concepts to explain model behavior. The method has previously been applied to the medical domain in the context of electronic health records, retinal fundus images and magnetic resonance imaging. We explore the usage of TCAV for building interpretable models on physiological time series, using an example of abnormality detection in electroencephalography (EEG). For this purpose, we adopt the XceptionTime model, which is suitable for multi-channel physiological data of variable sizes. The model provides state-of-the-art performance on raw EEG data and is publically available. We propose and test several ideas regarding concept definition through metadata mining, using additional labeled EEG data and extracting interpretable signal characteristics in the form of frequencies. By including our own hospital data with analog labeling, we further evaluate the robustness of our approach. The tested concepts show a TCAV score distribution that is in line with the clinical expectations, i.e. concepts known to have strong links with EEG pathologies (such as epileptiform discharges) received higher scores than the neutral concepts (e.g. sex). The scores were consistent across the applied concept generation strategies. TCAV has the potential to improve interpretability of deep learning applied to multi-channel signals as well as to detect possible biases in the data. Still, further work on developing the strategies for concept definition and validation on clinical physiological time series is needed to better understand how to extract clinically relevant information from the concept sensitivity scores.