SAITS: Self-attention-based imputation for time series

Missing data in time series is a pervasive problem that puts obstacles in the way of advanced analysis. A popular solution is imputation, where the fundamental challenge is to determine what values should be filled in. This paper proposes SAITS, a novel method based on the self-attention mechanism f...

Full description

Saved in:
Bibliographic Details
Published inExpert systems with applications Vol. 219; p. 119619
Main Authors Du, Wenjie, Côté, David, Liu, Yan
Format Journal Article
LanguageEnglish
Published Elsevier Ltd 01.06.2023
Subjects
Imputation model
Missing values
Neural network
Self-attention
Time series
Neural network
Missing values
Imputation model
Time series
Self-attention
Online AccessGet full text

Cover

More Information
Summary:Missing data in time series is a pervasive problem that puts obstacles in the way of advanced analysis. A popular solution is imputation, where the fundamental challenge is to determine what values should be filled in. This paper proposes SAITS, a novel method based on the self-attention mechanism for missing value imputation in multivariate time series. Trained by a joint-optimization approach, SAITS learns missing values from a weighted combination of two diagonally-masked self-attention (DMSA) blocks. DMSA explicitly captures both the temporal dependencies and feature correlations between time steps, which improves imputation accuracy and training speed. Meanwhile, the weighted-combination design enables SAITS to dynamically assign weights to the learned representations from two DMSA blocks according to the attention map and the missingness information. Extensive experiments quantitatively and qualitatively demonstrate that SAITS outperforms the state-of-the-art methods on the time-series imputation task efficiently and reveal SAITS’ potential to improve the learning performance of pattern recognition models on incomplete time-series data from the real world. •Address the missing data problem in time-series analysis tasks by deep learning.•A novel self-attention model imputes missing values in incomplete time series.•Our method solves the disadvantages of previous RNN-based imputation models.•SAITS has a better imputation model architecture than Transformer.•SAITS achieves the state-of-the-art performance on the time-series imputation task.
ISSN:0957-4174
1873-6793
DOI:10.1016/j.eswa.2023.119619