Enhancing Time Series Forecasting using DRL with attention-based neural architectures
DOI:
https://doi.org/10.63282/3050-9246/ICRTCSIT-111Keywords:
TSF, Deep Learning, DRL, Attention Based Neural ArchitecturesAbstract
Time series forecasting (TSF) is key to decision-making in finance, retail, supply chain management, healthcare, climate among others where accurate predictions inform resource allocation, risk management and strategic planning. While traditional statistical models such as ARIMA handle linear dependencies, they fail to capture the nonlinear and multivariate complexities of modern datasets. Deep learning models such as RNNs, LSTMs, GRUs, CNNs, and Transformers, have advanced forecasting accuracy by capturing temporal patterns and cross-variable interactions. However, these models are static and unable to adapt dynamically to regime shifts, shocks or evolving trends once trained. In addressing this gap, deep reinforcement learning (DRL) offers adaptivity by treating forecasting as sequential decision-making where agents iteratively refine predictions through reward feedback. Attention mechanisms further enhance interpretability and accuracy by highlighting critical time steps and features. This white paper critically reviewed DL and DRL models for multivariate TSF and evaluated their application in finance, retail, supply chains, climate forecasting and healthcare using research studies and datasets. Case studies demonstrate that attention-LSTM and Transformer variants outperform traditional deep models while hybrid DRL–DL approaches achieve greater adaptability. A proposed hybrid architecture integrates attention-based forecasting with DRL agents to combine predictive accuracy, adaptive learning, and interpretability. Although challenges on data, model structure and tasks remain, the approach has the potential to transform TSF into adaptive and decision-support systems across domains
Downloads
References
[1] Casolaro, V. Capone, G. Iannuzzo and F. Camastra, “Deep Learning for TSF: Advances and Open Problems,” Information, vol. 14, no. 11, 598, 2023. https://doi.org/10.3390/info14110598
[2] A. Pölz, A.P. Blaschke, J. Komma, A.H. Farnleitner and J. Derx, “Transformer versus LSTM: A comparison of deep learning models for karst spring discharge forecasting,” Water Resources Research, vol. 60, no. 4, pp. 1-19, 2024. DOI:10.1029/2022WR032602.
[3] Vaswani, “Attention is all you need,” in Proc. Adv. Neural Inf. Process. Syst., vol. 30, pp. 5998–6008, 2017.
[4] Chen and J. Dong, “Deep learning approaches for time series prediction in climate resilience applications”, Sec. Big Data, AI, and the Environment, vol. 13, 2025, https://doi.org/10.3389/fenvs.2025.1574981
[5] Chen, Y. Vincent and D. Laird, “Deep q-learning with recurrent neural networks,” Stanford cs229 course report, vol. 4, no. 3, 2024. https://cs229.stanford.edu/proj2016/report/ChenYingLaird-DeepQLearningWithRecurrentNeuralNetwords-report.pdf
[6] Andres, “Advanced Time Series Forecasting Methods,” Machine learning pills, https://mlpills.dev/time-series/advanced-time-series-forecasting-methods/ [Accessed: 3rd October, 2025]
[7] Oluwagbade, “Reinforcement Learning for Time-Series Data Analysis,” University of Cape Coast, 2025. https://www.researchgate.net/publication/395269263
[8] Ke, “An Efficient and Accurate DDPG-Based Recurrent Attention Model for Object Localization,” IEEE Access, vol. 99, pp. 1-12, 2020. DOI:10.1109/ACCESS.2020.3008171
[9] Arushana, E.A.A. Priyadarshanie, R.T. Weliwatta and S.A.H. Amarasena, “Deep Learning Architectures for TSF,”, pp. 1-15, 2024 DOI:10.13140/RG.2.2.19904.75524
[10] Pan, Y. Tang and G. Wang, “A Stock index futures price prediction approach based on the MULTI-GARCH-LSTM mixed model,” Mathematics, vol. 12, no. 11, pp. 1677-1694, 2024.
[11] Moreira, J. Rivas, F. Cruz, R. Dazeley, A. Ayala and B. Fernandes, “DRL with interactive feedback in a human–robot environment” Applied Sciences, vol. 10, no. 16, pp. 5574-5593, 2020. https://doi.org/10.3390/app10165574
[12] Terven, “DRL: A Chronological Overview and Methods,” AI, vol. 6, no. 3, pp. 46-54. https://doi.org/10.3390/ai6030046
[13] Alzubaidi, J. Zhang, A.J. Humaidi, A. Al-Dujaili, Y. Duan… and L. Farhan, “Review of deep learning: concepts, CNN architectures, challenges, applications, future directions,” Journal of Big Data, vol. 8, no. 53, 2021. https://doi.org/10.1186/s40537-021-00444-8
[14] Cheng, Z. Liu, X. Tao, Q. Liu, J. Zhang, T. Pan, ... and E. Chen, “A comprehensive survey of TSF: Concepts, challenges, and future directions,” Authorea Preprints, PP. 1-29, 2025. https://www.techrxiv.org/doi/full/10.36227/techrxiv.174430535.53879341
[15] M.D. Pra, “TSF with Deep Learning and Attention Mechanism,” Medium, https://medium.com/data-science/time-series-forecasting-with-deep-learning-and-attention-mechanism-2d001fc871fc [Accessed: 2nd October 2025]
[16] M.H. Alharbi, “Prediction of the Stock Market Using LSTM, ARIMA, and Hybrid of LSTM-ARIMA Models,” Journal of Knowledge Management Application and Practice: An International Journal, vol. 7, no. 1, pp. 15-22, 2025.
[17] M.S.A. Bhuiyan, J. Maua, S.R. Sultana, M.N.H. Mir, S.M. Sanchary and K. Nur, “Hybrid LSTM-CNN with Attention Mechanism for Accurate and Scalable Grocery Sales Forecasting,” In 2025 International Conference on Electrical, Computer and Communication Engineering (ECCE), pp. 1-6, 2025. DOI:10.1109/ECCE64574.2025.11012951
[18] N. Forghani and M. Forouzanfar, “Performance Comparison of Transformer, LSTM, and ARIMA TSF Models: A Healthcare Application,” In the 1st International Online Conference on Bioengineering, 16–18 October 2024, MDPI: Basel, Switzerland.
[19] N. Makarov, M. Bordukova, P. Quengdaeng, D. Garger, R. Rodriguez-Esteban… and M.P. Menden, “Large language models forecast patient health trajectories enabling digital twins,” NPJ Digit. Med. Vol. 8, no. 588, 2025. https://doi.org/10.1038/s41746-025-02004-3
[20] N. Smith, “A review of Reinforcement learning for financial time series prediction and portfolio optimization”, Medium, https://medium.com/journal-of-quantitative-finance/a-review-of-reinforcement-learning-for-financial-time-series-prediction-and-portfolio-optimisation-4cb2e92a23f3, 2021, [Accessed: 3rd October 2025]
[21] S. Amin, Part I: Understanding Long Short-Term Memory (LSTM): LSTM Implementation from Scratch, https://medium.com/@samina.amin/understanding-lstms-lstm-implementation-from-scratch-18965a150eca, 2024, Accessed [2nd October 2025].
[22] S. Du and H. Shen, “A Stock Prediction Method Based on DRL and Sentiment Analysis,” Applied Sciences, vol. 14, no. 19, pp. 8747-8762. https://doi.org/10.3390/app14198747
[23] S.-J. Bu and S.-B. Cho, “TSF with Multi-Headed Attention-Based Deep Learning for Residential Energy Consumption,” Energies, vol. 13, no. 18, pp. 4722-4743, 2020. https://doi.org/10.3390/en13184722
[24] T.S. Madhulatha and A.S. Ghori, “Deep neural network approach integrated with reinforcement learning for forecasting exchange rates using time series data and influential factors,” Scientific reports, vol. 15, no. 29009, 2025. https://doi.org/10.1038/s41598-025-12516-3
[25] V.R. Thota, “Comparative Study of TSF for Trucking Shipments: Evaluating Arima, LSTM, and Hybrid Arima-LSTM Models,” Master's thesis, State University of New York at Binghamton.
[26] X. Kong, Z. Chen, W. Liu, K. Ning, L. Zhang... and F. Xia, “Deep learning for TSF: a survey,” Int. J. Mach. Learn. & Cyber, vol.16, pp. 5079–5112, 2025, https://doi.org/10.1007/s13042-025-02560-w
[27] X. Liu and W. Wang, “Deep TSF Models: A Comprehensive Survey,” Mathematics, vol.12, no. 10, pp. 1504-1523, 2024. https://doi.org/10.3390/math12101504
[28] X. Zhang, P. Li, X. Han, Y. Yang and Y. Cui, "Enhancing Time Series Product Demand Forecasting with Hybrid Attention-Based Deep Learning Models," in IEEE Access, vol. 12, pp. 190079-190091, 2024, doi: 10.1109/ACCESS.2024.
[29] Y.H. Gu, D. Jin, H. Yin, R. Zheng, X. Piao and S.J. Yoo, “Forecasting agricultural commodity prices using dual input attention LSTM,” Agriculture, vol. 12, no. 2, 256. https://doi.org/10.3390/agriculture12020256
[30] Y. Liu, C. Gong, L. Yang and Y. Chen, “DSTP-RNN: A dual-stage two-phase attention-based recurrent neural network for long-term and multivariate time series prediction,” Expert Systems with Applications, vol. 143, no. 113082, 2020. https://doi.org/10.1016/j.eswa.2019.113082
[31] Thirunagalingam, A. (2024). Transforming real-time data processing: the impact of AutoML on dynamic data pipelines. Available at SSRN 5047601.
[32] Maroju, P. K. (2024). Advancing synergy of computing and artificial intelligence with innovations challenges and future prospects. FMDB Transactions on Sustainable Intelligent Networks, 1(1), 1-14.
[33] Priscila, S. S., Celin Pappa, D., Banu, M. S., Soji, E. S., Christus, A. T., & Kumar, V. S. (2024). Technological Frontier on Hybrid Deep Learning Paradigm for Global Air Quality Intelligence. In P. Paramasivan, S. Rajest, K. Chinnusamy, R. Regin, & F. John Joseph (Eds.), Cross-Industry AI Applications (pp. 144-162). IGI Global Scientific Publishing. https://doi.org/10.4018/979-8-3693-5951-8.ch010
[34] Panyaram, S. (2024). Utilizing quantum computing to enhance artificial intelligence in healthcare for predictive analytics and personalized medicine. FMDB Transactions on Sustainable Computing Systems, 2(1), 22-31.
[35] Mohanarajesh Kommineni. (2023/6). Investigate Computational Intelligence Models Inspired By Natural Intelligence, Such As Evolutionary Algorithms And Artificial Neural Networks. Transactions On Latest Trends In Artificial Intelligence. 4. P30. Ijsdcs.
