Intelligent EV Charging for Urban Prosumer Communities: An Auction and Multi-Agent Deep Reinforcement Learning Approach

• Published in: IEEE eTransactions on network and service management Vol. 19; no. 4; pp. 4384 - 4407
• Main Authors: Zou, Luyao, Munir, Md. Shirajum, Tun, Yan Kyaw, Kang, Seokwon, Hong, Choong Seon
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.12.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; breakeven-based double auction; Deep learning; Dynamic scheduling; Economics; Electric vehicle charging; Electric vehicles; EV charging; Machine learning; multi-agent deep reinforcement learning; Multiagent systems; Optimization; Power grids; Reinforcement learning; Renewable energy sources; urban prosumer communities
• ISSN: 1932-4537; 1932-4537
• DOI: 10.1109/TNSM.2022.3160210

Recently, the deployment of electric vehicles supply equipment (EVSE) and its market is expanding rapidly to support the massive penetration of electric vehicles (EVs). However, to accomplish an effective EV charging mechanism for urban prosumer communities, it is imperative to tackle the challenges of distinct energy generation among the communities, dependency of the total purchasable energy price of each EV based on the distance between EV and EVSE, and extreme uncertainty among the energy demand and generation. Therefore, in this paper, the problem of EV charging of urban prosumer communities is studied. In particular, a joint optimization problem is proposed to maximize both the social welfare and EV charging achieved rate of the considered urban prosumer communities. Consequently, the formulated problem is decomposed into 1) truthful double auction problem for determining the unit price and winners by maximizing social welfare, and 2) EV auction losers charging problem for improving EVs charging achieved rate by purchasing energy from the power grid. Then the breakeven-based double auction (BDA) mechanism is proposed to find the unit price and EV winners' for charging. Sequentially, a multi-agent deep reinforcement learning-based asynchronous advantage actor-critic algorithm with a long short-term memory layer (A3C-LSTM) is adopted to achieve the optimal grid energy buying decision for ensuring the charging of the losers. Finally, the experimental results demonstrate the efficacy of the proposed model that can increase the number of EV charging up to 57.31%, and prosumer communities have gained 86.04% of their income compared to baseline methods.