AI-Powered Open Banking Ecosystem: Real-World Applications of Machine and Deep Learning with Gradient Boosting and LLM-Enhanced Cloud APIs on Azure and Databricks
DOI:
https://doi.org/10.15662/IJRAI.2025.0806809Keywords:
open banking, machine learning, deep learning, gradient boosting, large language model, cloud API, Azure, Databricks, credit scoring, fraud detection, personalization, explainable AIAbstract
The open banking paradigm, facilitated by standardized APIs and customer-consented data sharing, offers unprecedented opportunities for innovation in financial services. This paper proposes a comprehensive ecosystem in which machine learning (ML), deep learning (DL) and large language models (LLMs) are harnessed via gradient boosting frameworks and cloud APIs (e.g., on Azure Machine Learning and Databricks) to deliver real-world applications in risk prediction, personalization, fraud detection and credit scoring. We examine how open banking data (transactions, balances, textual descriptions) enable high-performing gradient boosting and deep nets, and how LLM-enhanced cloud APIs support natural language and explanation layers. After designing the architecture and implementing a pilot across retail banking and SME lending use cases, we report results: gradient boosting models achieved significantly higher discriminatory power (AUC improvements ~5-10 %) versus traditional models; a deep-learning NLP module operating on transaction description text improved early default detection by ~15 %. The integration of LLM-driven conversational APIs enabled more intuitive customer interactions and automated compliance summarization. Key advantages include improved accuracy, faster time-to-insight, richer personalization and enhanced transparency via explainable AI (XAI) tools. Limitations involve data governance/regulatory complexity, model interpretability, and infrastructure cost/complexity. We conclude by outlining a roadmap for scaling across multiple institutions, exploring federated learning, and embedding real-time streaming architectures.
References
1. Chen, T., & Guestrin, C. (2016). XGBoost: A scalable tree boosting system. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.
2. Clements, J. M., Xu, D., Yousefi, N., & Efimov, D. (2020). Sequential deep learning for credit risk monitoring with tabular financial data. arXiv preprint arXiv:2012.15330.
3. Hjelkrem, L. O., de Lange, P. E., & Nesset, E. (2022). The value of open banking data for application credit scoring: Case study of a Norwegian bank. Journal of Risk and Financial Management, 15(12), 597. https://doi.org/10.3390/jrfm15120597
4. Reddy, B. T. K., & Sugumar, R. (2025, June). Effective forest fire detection by UAV image using Resnet 50 compared over Google Net. In AIP Conference Proceedings (Vol. 3267, No. 1, p. 020274). AIP Publishing LLC.
5. Shashank, P. S. R. B., Anand, L., & Pitchai, R. (2024, December). MobileViT: A Hybrid Deep Learning Model for Efficient Brain Tumor Detection and Segmentation. In 2024 International Conference on Progressive Innovations in Intelligent Systems and Data Science (ICPIDS) (pp. 157-161). IEEE.
6. Binu, C. T., Kumar, S. S., Rubini, P., & Sudhakar, K. (2024). Enhancing Cloud Security through Machine Learning-Based Threat Prevention and Monitoring: The Development and Evaluation of the PBPM Framework. https://www.researchgate.net/profile/Binu-C-T/publication/383037713_Enhancing_Cloud_Security_through_Machine_Learning-Based_Threat_Prevention_and_Monitoring_The_Development_and_Evaluation_of_the_PBPM_Framework/links/66b99cfb299c327096c1774a/Enhancing-Cloud-Security-through-Machine-Learning-Based-Threat-Prevention-and-Monitoring-The-Development-and-Evaluation-of-the-PBPM-Framework.pdf
7. Adari, V. K. (2024). APIs and open banking: Driving interoperability in the financial sector. International Journal of Research in Computer Applications and Information Technology (IJRCAIT), 7(2), 2015–2024.
8. Manda, P. (2024). THE ROLE OF MACHINE LEARNING IN AUTOMATING COMPLEX DATABASE MIGRATION WORKFLOWS. International Journal of Research Publications in Engineering, Technology and Management (IJRPETM), 7(3), 10451-10459.
9. Sridhar Kakulavaram. (2022). Life Insurance Customer Prediction and Sustainbility Analysis Using Machine Learning Techniques. International Journal of Intelligent Systems and Applications in Engineering, 10(3s), 390 –.Retrieved from https://ijisae.org/index.php/IJISAE/article/view/7649
10. Amuda, K. K., Kumbum, P. K., Adari, V. K., Chunduru, V. K., & Gonepally, S. (2024). Evaluation of crime rate prediction using machine learning and deep learning for GRA method. Data Analytics and Artificial Intelligence, 4 (3).
11. Kandula, N. Machine Learning Techniques in Fracture Mechanics a Comparative Study of Linear Regression, Random Forest, and Ada Boost Model.
12. HV, M. S., & Kumar, S. S. (2024). Fusion Based Depression Detection through Artificial Intelligence using Electroencephalogram (EEG). Fusion: Practice & Applications, 14(2).
13. Raju, L. H. V., & Sugumar, R. (2025, June). Improving jaccard and dice during cancerous skin segmentation with UNet approach compared to SegNet. In AIP Conference Proceedings (Vol. 3267, No. 1, p. 020271). AIP Publishing LLC.
14. Kesavan, E. (2025). A Comprehensive Review of Automated Software Testing Tools and Techniques. International Journal of Innovations in Science, Engineering And Management, 14-20. https://ijisem.com/journal/index.php/ijisem/article/view/279
15. Poornima, G., & Anand, L. (2024, April). Effective strategies and techniques used for pulmonary carcinoma survival analysis. In 2024 1st International Conference on Trends in Engineering Systems and Technologies (ICTEST) (pp. 1-6). IEEE.
16. Bussu, V. R. R. Leveraging AI with Databricks and Azure Data Lake Storage. https://pdfs.semanticscholar.org/cef5/9d7415eb5be2bcb1602b81c6c1acbd7e5cdf.pdf
17. Lin, T. (2024). The role of generative AI in proactive incident management: Transforming infrastructure operations. International Journal of Innovative Research in Science, Engineering and Technology, 13(12), Article — . https://doi.org/10.15680/IJIRSET.2024.1312014
18. Pachyappan, R., Kotapati, V. B. R., & Shanmugam, L. (2024). TicketGenesis: LLM-Driven Compliance Evidence Extraction and Auto-Assignment Engine. Los Angeles Journal of Intelligent Systems and Pattern Recognition, 4, 325-366.
19. Sivaraju, P. S., & Mani, R. (2024). Private Cloud Database Consolidation in Financial Services: A Comprehensive Case Study on APAC Financial Industry Migration and Modernization Initiatives. International Journal of Research Publications in Engineering, Technology and Management (IJRPETM), 7(3), 10472-10490.
20. Gorle, S., Christadoss, J., & Sethuraman, S. (2025). Explainable Gradient-Boosting Classifier for SQL Query Performance Anomaly Detection. American Journal of Cognitive Computing and AI Systems, 9, 54-87.
21. Hu, L., Chen, J., Vaughan, J., Yang, H., Sudjianto, A., & Nair, V. N. (2020). Supervised machine learning techniques: An overview with applications to banking. arXiv preprint arXiv:2008.04059.
22. Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., … Liu, T.-Y. (2017). LightGBM: A highly efficient gradient boosting decision tree. Advances in Neural Information Processing Systems, 30.
23. Kim, S. D., Andreeva, G., & Rovatsos, M. (2023). The double-edged sword of big data and information technology for the disadvantaged: A cautionary tale from open banking. arXiv preprint arXiv:2307.13408.
24. “Top machine learning algorithms used in financial services.” (2024). eICTA IIT Kharagpur Knowledge Hub. Retrieved from https://www.eicta.iitk.ac.in/knowledge-hub/machine-learning/top-machine-learning-algorithms/feed/
