Short-Term Electrical Load Forecasting Based on Time Augmented Transformer

• Published in: International journal of computational intelligence systems Vol. 15; no. 1; pp. 1 - 11
• Main Authors: Zhang, Guangqi, Wei, Chuyuan, Jing, Changfeng, Wang, Yanxue
• Format: Journal Article
• Language: English
• Published: Dordrecht Springer Netherlands 18.08.2022; Springer Nature B.V; Springer
• Subjects: Artificial Intelligence; Artificial neural networks; Computational Intelligence; Control; Deep learning; Deep neural network; Electrical loads; Engineering; Forecasting; Machine learning; Mathematical Logic and Foundations; Mathematical models; Mechatronics; Natural language processing; Neural networks; Power management; Recurrent neural networks; Representations; Research Article; Robotics; Short-term load forecast; Time augmentation; Transformer; Transformers; Time augmentation; Deep neural network; Transformer; Short-term load forecast
• ISSN: 1875-6883; 1875-6891; 1875-6883
• DOI: 10.1007/s44196-022-00128-y

Electrical load forecasting is of vital importance in intelligent power management and has been a hot spot in industrial Internet application field. Due to the complex patterns and dynamics of the data, accurate short-term load forecasting is still a challenging task. Currently, many tasks use deep neural networks for power load forecasting, and most use recurrent neural network as the basic architecture, including Long Short-Term Memory (LSTM), Sequence to Sequence (Seq2Seq), etc. However, the performance of these models is not as good as expected due to the gradient vanishing problem in recurrent neural network. Transformer is a deep learning model initially designed for natural language processing, it calculates input–output representations and captures long dependencies entirely on attention mechanisms which has great performance for capturing the complex dynamic nonlinear sequence dependence on long sequence input. In this work, we proposed a model Time Augmented Transformer (TAT) for short-term electrical load forecasting. A temporal augmented module in TAT is designed to learn the temporal relationships representation between the input history series to adapt to the short-term power load forecasting task. We evaluate our approach on a real-word dataset for electrical load and extensively compared it to the performance of the existed electrical load forecasting model including statistical approach, traditional machine learning and deep learning methods, the experimental results show that the proposed TAT model results in higher precision and accuracy in short-term load forecasting.