Deep Neural Networks Algorithms for Stochastic Control Problems on Finite Horizon: Numerical Applications

• Published in: Methodology and computing in applied probability Vol. 24; no. 1; pp. 143 - 178
• Main Authors: Bachouch, Achref, Huré, Côme, Langrené, Nicolas, Pham, Huyên
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.03.2022; Springer Nature B.V
• Subjects: Algorithms; Artificial neural networks; Business and Management; Deep learning; Distributed generation; Economics; Electrical Engineering; Energy storage; Iterative methods; Life Sciences; Machine learning; Mathematics and Statistics; Neural networks; Optimal control; Partial differential equations; Statistics; Stochastic processes; Deep learning; Quantization; Performance iteration; Value iteration; Monte Carlo; Policy learning
• ISSN: 1387-5841; 1573-7713
• DOI: 10.1007/s11009-019-09767-9

This paper presents several numerical applications of deep learning-based algorithms for discrete-time stochastic control problems in finite time horizon that have been introduced in Huré et al. ( 2018 ). Numerical and comparative tests using TensorFlow illustrate the performance of our different algorithms, namely control learning by performance iteration (algorithms NNcontPI and ClassifPI), control learning by hybrid iteration (algorithms Hybrid-Now and Hybrid-LaterQ), on the 100-dimensional nonlinear PDEs examples from Weinan et al. ( 2017 ) and on quadratic backward stochastic differential equations as in Chassagneux and Richou ( 2016 ). We also performed tests on low-dimension control problems such as an option hedging problem in finance, as well as energy storage problems arising in the valuation of gas storage and in microgrid management. Numerical results and comparisons to quantization-type algorithms Qknn, as an efficient algorithm to numerically solve low-dimensional control problems, are also provided.