SleepTransformer: Automatic Sleep Staging With Interpretability and Uncertainty Quantification

• Published in: IEEE transactions on biomedical engineering Vol. 69; no. 8; pp. 2456 - 2467
• Main Authors: Phan, Huy, Mikkelsen, Kaare, Chen, Oliver Y., Koch, Philipp, Mertins, Alfred, De Vos, Maarten
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.08.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Automatic sleep staging; Brain modeling; Decisions; deep neural network; EEG; Electroencephalography; Entropy; Inspection; interpretability; sequence-to-sequence; Sleep; Task analysis; transformer; Transformers; Uncertainty; uncertainty estimation
• ISSN: 0018-9294; 1558-2531; 1558-2531
• DOI: 10.1109/TBME.2022.3147187

Background: Black-box skepticism is one of the main hindrances impeding deep-learning-based automatic sleep scoring from being used in clinical environments. Methods: Towards interpretability, this work proposes a sequence-to-sequence sleep-staging model, namely SleepTransformer. It is based on the transformer backbone and offers interpretability of the model's decisions at both the epoch and sequence level. We further propose a simple yet efficient method to quantify uncertainty in the model's decisions. The method, which is based on entropy, can serve as a metric for deferring low-confidence epochs to a human expert for further inspection. Results: Making sense of the transformer's self-attention scores for interpretability, at the epoch level, the attention scores are encoded as a heat map to highlight sleep-relevant features captured from the input EEG signal. At the sequence level, the attention scores are visualized as the influence of different neighboring epochs in an input sequence (i.e. the context) to recognition of a target epoch, mimicking the way manual scoring is done by human experts. Conclusion: Additionally, we demonstrate that SleepTransformer performs on par with existing methods on two databases of different sizes. Significance: Equipped with interpretability and the ability of uncertainty quantification, SleepTransformer holds promise for being integrated into clinical settings.