Deep Direct Reinforcement Learning for Financial Signal Representation and Trading

• Published in: IEEE transaction on neural networks and learning systems Vol. 28; no. 3; pp. 653 - 664
• Main Authors: Deng, Yue, Bao, Feng, Kong, Youyong, Ren, Zhiquan, Dai, Qionghai
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.03.2017; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Artificial neural networks; Back propagation; Deep learning (DL); Feature extraction; financial signal processing; Learning (artificial intelligence); Machine learning; neural network (NN) for finance; Neural networks; Optimization; Reinforcement; reinforcement learning (RL); Representations; Robustness; Signal representation; Training; Unknown environments
• ISSN: 2162-237X; 2162-2388; 2162-2388
• DOI: 10.1109/TNNLS.2016.2522401

Can we train the computer to beat experienced traders for financial assert trading? In this paper, we try to address this challenge by introducing a recurrent deep neural network (NN) for real-time financial signal representation and trading. Our model is inspired by two biological-related learning concepts of deep learning (DL) and reinforcement learning (RL). In the framework, the DL part automatically senses the dynamic market condition for informative feature learning. Then, the RL module interacts with deep representations and makes trading decisions to accumulate the ultimate rewards in an unknown environment. The learning system is implemented in a complex NN that exhibits both the deep and recurrent structures. Hence, we propose a task-aware backpropagation through time method to cope with the gradient vanishing issue in deep training. The robustness of the neural system is verified on both the stock and the commodity future markets under broad testing conditions.