DeepScalper: A Risk-Aware Reinforcement Learning Framework to Capture Fleeting Intraday Trading Opportunities
Reinforcement learning (RL) techniques have shown great success in many challenging quantitative trading tasks, such as portfolio management and algorithmic trading. Especially, intraday trading is one of the most profitable and risky tasks because of the intraday behaviors of the financial market t...
Saved in:
Published in | arXiv.org |
---|---|
Main Authors | , , , , , , |
Format | Paper |
Language | English |
Published |
Ithaca
Cornell University Library, arXiv.org
21.08.2022
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Abstract | Reinforcement learning (RL) techniques have shown great success in many challenging quantitative trading tasks, such as portfolio management and algorithmic trading. Especially, intraday trading is one of the most profitable and risky tasks because of the intraday behaviors of the financial market that reflect billions of rapidly fluctuating capitals. However, a vast majority of existing RL methods focus on the relatively low frequency trading scenarios (e.g., day-level) and fail to capture the fleeting intraday investment opportunities due to two major challenges: 1) how to effectively train profitable RL agents for intraday investment decision-making, which involves high-dimensional fine-grained action space; 2) how to learn meaningful multi-modality market representation to understand the intraday behaviors of the financial market at tick-level. Motivated by the efficient workflow of professional human intraday traders, we propose DeepScalper, a deep reinforcement learning framework for intraday trading to tackle the above challenges. Specifically, DeepScalper includes four components: 1) a dueling Q-network with action branching to deal with the large action space of intraday trading for efficient RL optimization; 2) a novel reward function with a hindsight bonus to encourage RL agents making trading decisions with a long-term horizon of the entire trading day; 3) an encoder-decoder architecture to learn multi-modality temporal market embedding, which incorporates both macro-level and micro-level market information; 4) a risk-aware auxiliary task to maintain a striking balance between maximizing profit and minimizing risk. Through extensive experiments on real-world market data spanning over three years on six financial futures, we demonstrate that DeepScalper significantly outperforms many state-of-the-art baselines in terms of four financial criteria. |
---|---|
AbstractList | Reinforcement learning (RL) techniques have shown great success in many challenging quantitative trading tasks, such as portfolio management and algorithmic trading. Especially, intraday trading is one of the most profitable and risky tasks because of the intraday behaviors of the financial market that reflect billions of rapidly fluctuating capitals. However, a vast majority of existing RL methods focus on the relatively low frequency trading scenarios (e.g., day-level) and fail to capture the fleeting intraday investment opportunities due to two major challenges: 1) how to effectively train profitable RL agents for intraday investment decision-making, which involves high-dimensional fine-grained action space; 2) how to learn meaningful multi-modality market representation to understand the intraday behaviors of the financial market at tick-level. Motivated by the efficient workflow of professional human intraday traders, we propose DeepScalper, a deep reinforcement learning framework for intraday trading to tackle the above challenges. Specifically, DeepScalper includes four components: 1) a dueling Q-network with action branching to deal with the large action space of intraday trading for efficient RL optimization; 2) a novel reward function with a hindsight bonus to encourage RL agents making trading decisions with a long-term horizon of the entire trading day; 3) an encoder-decoder architecture to learn multi-modality temporal market embedding, which incorporates both macro-level and micro-level market information; 4) a risk-aware auxiliary task to maintain a striking balance between maximizing profit and minimizing risk. Through extensive experiments on real-world market data spanning over three years on six financial futures, we demonstrate that DeepScalper significantly outperforms many state-of-the-art baselines in terms of four financial criteria. |
Author | Xue, Wanqi Xu, He Zhu, Junlei An, Bo Sun, Shuo Wang, Rundong Li, Jian |
Author_xml | – sequence: 1 givenname: Shuo surname: Sun fullname: Sun, Shuo – sequence: 2 givenname: Wanqi surname: Xue fullname: Xue, Wanqi – sequence: 3 givenname: Rundong surname: Wang fullname: Wang, Rundong – sequence: 4 givenname: He surname: Xu fullname: Xu, He – sequence: 5 givenname: Junlei surname: Zhu fullname: Zhu, Junlei – sequence: 6 givenname: Jian surname: Li fullname: Li, Jian – sequence: 7 givenname: Bo surname: An fullname: An, Bo |
BookMark | eNqNissKgkAUQIco6OU_DLQWdEx77KKSgiAw9zLULcb0znRnJPr7FPqAVgfOOWPWR43QYyMRRaG_nAsxZJ61ZRAEIlmIOI5GrN4BmMtVVgZozTc8U_bpb96SgGeg8K7pCjWg4yeQhAofPCVZw1vTkzvNt9K4pn3TCsB19YiO5E1-eN6iE2djNLkGlVNgp2xwl5UF78cJm6X7fHvwDelXA9YVpW4I21SIRIhkFSzjMPrv-gJRtUt5 |
ContentType | Paper |
Copyright | 2022. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
Copyright_xml | – notice: 2022. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
DBID | 8FE 8FG ABJCF ABUWG AFKRA AZQEC BENPR BGLVJ CCPQU DWQXO HCIFZ L6V M7S PIMPY PQEST PQQKQ PQUKI PRINS PTHSS |
DatabaseName | ProQuest SciTech Collection ProQuest Technology Collection Materials Science & Engineering Collection ProQuest Central (Alumni) ProQuest Central ProQuest Central Essentials ProQuest Central Technology Collection ProQuest One Community College ProQuest Central Korea SciTech Premium Collection ProQuest Engineering Collection Engineering Database Publicly Available Content Database ProQuest One Academic Eastern Edition (DO NOT USE) ProQuest One Academic ProQuest One Academic UKI Edition ProQuest Central China Engineering Collection |
DatabaseTitle | Publicly Available Content Database Engineering Database Technology Collection ProQuest Central Essentials ProQuest One Academic Eastern Edition ProQuest Central (Alumni Edition) SciTech Premium Collection ProQuest One Community College ProQuest Technology Collection ProQuest SciTech Collection ProQuest Central China ProQuest Central ProQuest Engineering Collection ProQuest One Academic UKI Edition ProQuest Central Korea Materials Science & Engineering Collection ProQuest One Academic Engineering Collection |
DatabaseTitleList | Publicly Available Content Database |
Database_xml | – sequence: 1 dbid: 8FG name: ProQuest Technology Collection url: https://search.proquest.com/technologycollection1 sourceTypes: Aggregation Database |
DeliveryMethod | fulltext_linktorsrc |
Discipline | Physics |
EISSN | 2331-8422 |
Genre | Working Paper/Pre-Print |
GroupedDBID | 8FE 8FG ABJCF ABUWG AFKRA ALMA_UNASSIGNED_HOLDINGS AZQEC BENPR BGLVJ CCPQU DWQXO FRJ HCIFZ L6V M7S M~E PIMPY PQEST PQQKQ PQUKI PRINS PTHSS |
ID | FETCH-proquest_journals_26226908513 |
IEDL.DBID | BENPR |
IngestDate | Thu Oct 10 18:17:56 EDT 2024 |
IsOpenAccess | true |
IsPeerReviewed | false |
IsScholarly | false |
Language | English |
LinkModel | DirectLink |
MergedId | FETCHMERGED-proquest_journals_26226908513 |
OpenAccessLink | https://www.proquest.com/docview/2622690851?pq-origsite=%requestingapplication% |
PQID | 2622690851 |
PQPubID | 2050157 |
ParticipantIDs | proquest_journals_2622690851 |
PublicationCentury | 2000 |
PublicationDate | 20220821 |
PublicationDateYYYYMMDD | 2022-08-21 |
PublicationDate_xml | – month: 08 year: 2022 text: 20220821 day: 21 |
PublicationDecade | 2020 |
PublicationPlace | Ithaca |
PublicationPlace_xml | – name: Ithaca |
PublicationTitle | arXiv.org |
PublicationYear | 2022 |
Publisher | Cornell University Library, arXiv.org |
Publisher_xml | – name: Cornell University Library, arXiv.org |
SSID | ssj0002672553 |
Score | 3.4233325 |
SecondaryResourceType | preprint |
Snippet | Reinforcement learning (RL) techniques have shown great success in many challenging quantitative trading tasks, such as portfolio management and algorithmic... |
SourceID | proquest |
SourceType | Aggregation Database |
SubjectTerms | Coders Deep learning Embedding Encoders-Decoders Machine learning Portfolio management Reagents Risk |
Title | DeepScalper: A Risk-Aware Reinforcement Learning Framework to Capture Fleeting Intraday Trading Opportunities |
URI | https://www.proquest.com/docview/2622690851 |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwfV3PS8MwFH64FcGbP_HHHAG9Fm2WttaLzNk6hc1RFXYbaZOKiG1tO8SLf7svIdWDsGMIhOSRvPfly_deAE45dx3up5nNlTgMEQG38QoW2NTjjPqeoJlQucOTqTd-Zvdzd24It9rIKlufqB21KFLFkZ9RD4FCoADCVflhq1-j1Ouq-UKjAxZ1mHqmta7D6Sz-ZVmo5yNmHvxztDp6RJtgzXgpqy1Yk_k2rGvRZVrvwPuNlOUjGgk7L8mQxK_1mz385JUksdQVTVNN3hFTBPWFRK2UijQFGfFS0f8ETa_zlsmdImoF_yIYgFRIIg-lQtfLXFdN3YWTKHwaje12jguzj-rF36oHe9DNi1zuA3E54gjBBNoTr2ciS2ggA4clNGNSusHgAHqrRjpc3X0EG1RJ_M_xBDk96DbVUh5j4G2SPnQuotu-sTG2Jt_hD_Ruj4k |
link.rule.ids | 786,790,12792,21416,33406,33777,43633,43838 |
linkProvider | ProQuest |
linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwfV3fS8MwED50Q_TNn_hjakBfgzZNW-uLjGntdJsyJ-ytZE06RGxr2yH-915Cqw_CngMhOS53X758dwE4F8KxhBcnVGhxGCICQfEK5lPmCs48V7JE6trh4cgNX_nD1JnWhFtZyyqbmGgCtcxizZFfMBeBgq8Bwk3-SfWvUfp1tf5CYxXa3HZt7edXwf0vx8JcDxGz_S_MmtwRbEL7WeSq2IIVlW7DmpFcxuUOfNwqlb-giXDwmnTJ-K18p90vUSgyVqafaWyoO1K3QJ2ToBFSkSojPZFr8p-g4U3VMulrmlaKb4LpRyck8pRrbL1ITc_UXTgL7ia9kDZrjGovKqO_Pdt70EqzVO0DcQSiCMklWhMvZzKZMV_5Fp-xhCvl-PYBdJbNdLh8-BTWw8lwEA36o8cj2GBa7H-JZ8nqQKsqFuoYU3A1OzF2_gGqSo75 |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=DeepScalper%3A+A+Risk-Aware+Reinforcement+Learning+Framework+to+Capture+Fleeting+Intraday+Trading+Opportunities&rft.jtitle=arXiv.org&rft.au=Sun%2C+Shuo&rft.au=Xue%2C+Wanqi&rft.au=Wang%2C+Rundong&rft.au=Xu%2C+He&rft.date=2022-08-21&rft.pub=Cornell+University+Library%2C+arXiv.org&rft.eissn=2331-8422 |