Multi-head attention-based model for reconstructing continuous missing time series data

• Published in: The Journal of supercomputing Vol. 79; no. 18; pp. 20684 - 20711
• Main Authors: Wu, Huafeng, Zhang, Yuxuan, Liang, Linian, Mei, Xiaojun, Han, Dezhi, Han, Bing, Weng, Tien-Hsiung, Li, Kuan-Ching
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.12.2023; Springer Nature B.V
• Subjects: Compilers; Computer Science; Interference; Interpreters; Missing data; Processor Architectures; Programming Languages; Time series; Underwater communication; Wireless sensor networks; UWSNs; Multi-head self-attention mechanism; Bi-GRU; Continuous data missing problem; Recovery method
• ISSN: 0920-8542; 1573-0484
• DOI: 10.1007/s11227-023-05465-z

Time series data sensed by underwater wireless sensor networks (UWSNs) play a crucial role in prediction and decision-making in marine applications. Unfortunately, equipment and environmental precision and interference problems in UWSNs may lead to a large amount of missing data in a specific time period. In this work, we propose a multi-head attention-based sequence-to-sequence model (MSSM) for reconstructing continuous missing data. It can reduce the negative impact of missing data due to the harsh underwater communication environment. MSSM has a dual encoder architecture that can process known data on both sides of missing values. Multi-head self-attention mechanism and bidirectional gate recurrent unit (Bi-GRU) can thoroughly learn the temporal patterns and the inter-sequence dependencies; moreover, soft thresholding can also reduce noise interference. Datasets are used to test the performance, and experimental results show that metrics are lower than other relevant alternatives, demonstrating that MSSM is an effective model with solid generalization ability.