Expert Incorporated Deep Reinforcement Learning Approach for Market Arbitrage Strategy of the Battery Energy Storage

Peak-valley arbitrage is one of the important ways for energy storage systems to make profits. Traditional optimization methods have shortcomings such as long solution time, poor universality, and difficulty in applying to non-convex problems. This study addresses this issue by utilizing Deep Reinfo...

Full description

Saved in:
Bibliographic Details
Published in2023 IEEE 7th Conference on Energy Internet and Energy System Integration (EI2) pp. 1709 - 1714
Main Authors Zhang, Bohan, Yi, Zhongkai, Xu, Ying, Xu, Jianing, Lu, Yu, Zhou, Yuhao
Format Conference Proceeding
LanguageEnglish
Published IEEE 15.12.2023
Subjects
Deep reinforcement learning
energy storage system
expert experience
market arbitrage
Optimal control
Optimization methods
Power control
Profitability
System integration
Training
Online AccessGet full text

Cover

More Information
Summary:Peak-valley arbitrage is one of the important ways for energy storage systems to make profits. Traditional optimization methods have shortcomings such as long solution time, poor universality, and difficulty in applying to non-convex problems. This study addresses this issue by utilizing Deep Reinforcement Learning (DRL) to optimize the market arbitrage of battery storage system (BSS). Firstly, the market arbitrage problem is presented as a typical Markov Decision Process (MDP). Secondly, an expert incorporated DRL approach is proposed to seek for the optimal control strategy for the energy storage systems. Finally, the proposed algorithm is verified based on the U.K. electricity market background. The numerical simulation result show that the proposed method can achieve refined power control on the premise of preventing overcharge and over-discharge of the battery storage system, and achieve higher training efficiency comparing with existing deep reinforcement learning approach.
DOI:10.1109/EI259745.2023.10513107