Information-aware attention dynamic synergetic network for multivariate time series long-term forecasting

• Published in: Neurocomputing (Amsterdam) Vol. 500; pp. 143 - 154
• Main Authors: He, Xiaoyu, Shi, Suixiang, Geng, Xiulin, Xu, Lingyu
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 21.08.2022
• Subjects: Attention; Long-term forecasting; Multivariate time series; Trend similarity; Long-term forecasting; Trend similarity; Multivariate time series; Attention
• ISSN: 0925-2312; 1872-8286
• DOI: 10.1016/j.neucom.2022.04.124

Multivariate time series forecasting is widely used in a variety of fields, such as cyber-physical systems and financial market analysis. Recently, attention-based recurrent neural networks (RNNs) have been paid attention by scholars for its ability to reduce the accumulative errors. Although attention-based RNNs are proved effective, there are still some challenges: 1) the noise in raw data will hurt model performance, 2) the traditional attentions are easy to discard low-weight input vectors, thereby leading to poor accuracy, and 3) encoder-focused attentions are not conducive to maintaining the trend consistency between the prediction and the original sequence. To tackle these problems, we propose a novel Information-aware Attention Dynamic Synergy Network (IADSN), which contains a specially designed information-aware long short-term memory network (IALSTM), a multi-dimensional attention (MA) that fine-grainedly assigns weight to your attention, and an attention dynamic synergy strategy. The novelty of MA lies in assigning a weight vector to the input instead of a single scalar. Therefore, it can identify the importance of each dimension of the input vector, thus alleviating the problem of ignoring low-weight inputs. IALSTM employs internal MA and a gated fusion unit to weaken the influence of input untrusted features on prediction. The attention dynamic synergy strategy maintains the similarity between the predicted and the original series by establishing an association among the current decoder unit, the previous decoder units, and the encoder. Experiments on three fields of energy, air quality, and ecological datasets demonstrate that IADSN not only achieves the state-of-the-art performance, but also effectively maintains the dynamic tendency of the forecast series.