Energy Management Strategy for Fuel Cell/Battery/Ultracapacitor Hybrid Electric Vehicles Using Deep Reinforcement Learning With Action Trimming

• Published in: IEEE transactions on vehicular technology Vol. 71; no. 7; pp. 7171 - 7185
• Main Authors: Fu, Zhumu, Wang, Haocong, Tao, Fazhan, Ji, Baofeng, Dong, Yongsheng, Song, Shuzhong
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.07.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adaptive control; Adaptive filters; Consumption; Dater driven; Deep learning; deep reinforcement learning; Dynamic programming; Electric vehicles; Energy efficiency; Energy management; energy management strategy; fuel cell hybrid electric vehicle; Fuel cells; Fuel consumption; Fuel economy; Fuzzy control; heuristic technique; Hybrid electric vehicles; Hydrogen; Life span; Machine learning; Optimization; Power consumption; Power management; Power sources; Real-time systems; Structural hierarchy; Supercapacitors; Topology; Trimming; Vehicle dynamics; Voltage
• ISSN: 0018-9545; 1939-9359
• DOI: 10.1109/TVT.2022.3168870

As for fuel cell hybrid electric vehicle equipped with battery (BAT) and ultracapacitor (UC), its dynamic topology structure is complex and different characteristics of three power sources induce challenges in energy management for fuel economy, power sources lifespan, and dynamic performance of the vehicle. In this paper, an energy management strategy (EMS) based on a hierarchical power splitting structure and deep reinforcement learning (DRL) is proposed. In the higher layer strategy of the proposed EMS, the UC is employed to supply peak power and recover braking energy through the adaptive filter based on fuzzy control. Then, the integrated DRL and equivalent consumption minimization strategy framework is proposed to optimize the power allocation of fuel cell (FC) and BAT in the lower layer, to ensure the highly efficient operation of FC and reduce hydrogen consumption. And the action trimming based on heuristic technique is proposed to further restrain the adverse effect of sudden peak power on FC lifespan. The simulation results show the proposed EMS can make the output of FC smoother, improve its working efficiency to alleviate the stress of BAT, and increase by 14.8% compared with the Q-learning strategy in fuel economy under WLTP driving cycle. Meanwhile, the obtained results under UDDSHDV show fuel economy of the proposed EMS can reach dynamic programming (DP) benchmark level of 89.7<inline-formula><tex-math notation="LaTeX">\%</tex-math></inline-formula>.