Deep Reinforcement Learning for Algorithmic Trading Strategies
DOI:
https://doi.org/10.15662/IJRAI.2024.0702003Keywords:
Deep Reinforcement Learning, Algorithmic Trading, Deep Q-Network, Proximal Policy Optimization, Financial Markets, Stock TradingAbstract
In an ever more complex world of globalized market structures, the academic literature and practitioners in finance have been drawn into applying state-of-the-art artificial intelligence (AI) techniques to algorithmic trading tasks. Conventional methods include technical indicators and rule based systems which have short comings in changing market environment. We study exploiting DRL for building adaptive and profitable trading strategies in this paper. We utilized a DQN and PPO model to compare their performance under U.S. stock market environment. The features included in the models were computed from historical high-frequency data of the S&P 500 index, such as price returns, moving averages and volatility measures.
The method was implemented as such that an agent had buy, hold or sell actions in an environment where the rewards were measured by the cumulative portfolio returns considering transaction costs. DQN was a strong performer in stable markets, while PPO overall performed better when markets were more volatile. The experimental results demonstrated that PPO significantly outperformed DQN with satisfactory Sharpe ratio 1.75 and average annualized return over 18%, comparing to the 1.12 of DQN. Both of these models outperformed the buy-and-hold and moving-average crossover type baselines.
The results demonstrate that, in dynamic environment, DRL which can learn optimal trading policies adaptively is possible to achieve a notable enhancement of the risk-adjusted return. These findings also emphasize the generalization capability of DRL on the development of intelligent trading systems that address financial market volatility and uncertainty.
References
[1] M. Pasqual High Frequency Trading: an analysis of the phenomenon and the effect of the human component Università Ca’Foscari Venezia (2020)
[2] K.Z. Zaharudin, M.R. Young, W.-H. Hsu High-frequency trading: Definition, implications, and controversies J Econ Surv, 36 (1) (2022), pp. 75-107
[3] G. Sonkavde, D.S. Dharrao, A.M. Bongale, S.T. Deokate, D. Doreswamy, S.K. Bhat Forecasting stock market prices using machine learning and deep learning models: A systematic review, performance analysis and discussion of implications Int J Financ Stud, 11 (3) (2023), p. 94
[4] K. Sharifani, M. Amini Machine learning and deep learning: A review of methods and applications World Inf Technol Eng J, 10 (07) (2023), pp. 3897-3904
[5] T. Singh, R. Kalra, S. Mishra, Satakshi, M. Kumar An efficient real-time stock prediction exploiting incremental learning and deep learning Evol Syst, 14 (6) (2023), pp. 919-937
[6] A. Thakkar, K. Chaudhari A comprehensive survey on portfolio optimization, stock price and trend prediction using particle swarm optimization Arch Comput Methods Eng, 28 (4) (2021), pp. 2133-2164
[7] U.R. Chinthapalli, R.K. Bommisetti, B.R. Kondamudi, G. Bagale, R. Satyanarayana Isolated stakeholders’ behavior towards fintech assisted by artificial intelligence technology Ann Oper Res (2021), pp. 1-27
[8] M.M. Kumbure, C. Lohrmann, P. Luukka, J. Porras Machine learning techniques and data for stock market forecasting: A literature review Expert Syst Appl, 197 (2022), Article 116659
[9] A.M. Rahmani, B. Rezazadeh, M. Haghparast, W.-C. Chang, S.G. Ting Applications of artificial intelligence in the economy, including applications in stock trading, market analysis, and risk management IEEE Access (2023)
[10] A. Salehpour, K. Samadzamini Machine learning applications in algorithmic trading: a comprehensive systematic review Int J Educ Manag Eng, 13 (6) (2023), p. 41.
[11] M. Ayitey Junior, P. Appiahene, O. Appiah, C.N. Bombie Forex market forecasting using machine learning: Systematic literature review and meta-analysis J Big Data, 10 (1) (2023), p. 9
[12] A. Bhardwaj Time series forecasting with recurrent neural networks: An in-depth analysis and comparative study Perform Eval, 2 (4) (2023)
[13] T.H. Aldhyani, A. Alzahrani Framework for predicting and modeling stock market prices based on deep learning algorithms Electronics, 11 (19) (2022), p. 3149
[14] J. Zhang, L. Ye, Y. Lai Stock price prediction using CNN-BiLSTM-Attention model Mathematics, 11 (9) (2023), p. 1985
[15] B. Gülmez, Stock price prediction with optimized deep LSTM network with artificial rabbits optimization algorithm Expert Syst Appl, 227 (2023), Article 120346
[16] M. Dogariu, L.-D. Ştefan, B.A. Boteanu, C. Lamba, B. Kim, B. Ionescu Generation of realistic synthetic financial time-series ACM Trans Multimed Comput Commun Appl (TOMM), 18 (4) (2022), pp. 1-27.
