Next Gen Real Time Fraud Detection with Cloud Scale Stream Processing
DOI:
https://doi.org/10.15662/IJRAI.2025.0801005Keywords:
Real-time fraud detection systems, Stream processing for fraud analytics, Cloud-scale event-driven architectures, Apache Kafka and Flink for fraud detection, Low-latency anomaly detection in streaming data, Machine learning for real-time financial fraud, Scalable cloud-based fraud detection pipelinesAbstract
The sudden rise of digital transactions in financial, retail, and e-commerce platform has made fraudulent enterprises more extensive and sophisticated. The mainstream fraud detection systems, mostly batch-based and rule-based, find it difficult to provide real-time and precise responses to the situation when the data velocity is high, and the nature of the attacks is constantly changing. Such constraints are likely to lead to slower detection, high false positive rates and low adaptability to new fraud patterns.
This paper presents a subsequent generation real-time fraud detection system that is based on cloud-scale stream processing principles in order to deal with these challenges. The suggested architecture will take advantage of the following features: event-driven data ingestion, low-latency stream processing, and online machine learning inference to allow running a continuous fraud evaluation as transactions are received. The framework provides scalable cloud infrastructure coupled with stateful stream processing engines that are capable of executing features in real-time, performing model execution, and generating alerts and being fault tolerant and scalable.
This work has three major contributions. First, it introduces a single, cloud-native system to real-time detect fraud, which integrates streaming data pipelines in real-time with machine learning-based analytics. Second, it presents a stream based feature engineering and inference pipeline, which aims at reducing end-to-end detection latency with a high throughput of transactions. Third, it gives an empirical assessment of the suggested system explaining better detection responsiveness, maintained throughput and consistent performance with increasing load relative to classic batch and micro-batch fraud detection strategies.
The findings show that cloud-scale stream processing is a feasible and efficient basis of the next-generation fraud detection infrastructure and can be used in real-time to make decisions and provide greater operational stability in modern data-intensive operational settings.
References
1. Aarthy, R., Viranth, D., Ramya, S., Rajagopal, S., & Divyashri, S. (2023). Spatiotemporal Anomaly Object Detection Using Edge Cloud Collaborative for Real-Time Surveillance Video. In 2023 9th International Conference on Advanced Computing and Communication Systems, ICACCS 2023 (pp. 554–558). Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/ICACCS57279.2023.10112775
2. Abdallah, A., Maarof, M. A., & Zainal, A. (2016, June 1). Fraud detection system: A survey. Journal of Network and Computer Applications. Academic Press. https://doi.org/10.1016/j.jnca.2016.04.007
3. Ahmed Salamkar, M., & Associate at Morgan Chase, S. J. (2023). Real-time Analytics: Implementing ML algorithms to analyze data streams in real-time. Journal of AI-Assisted Scientific Discovery, 3(2), 587–612. Retrieved from https://scienceacadpress.com/index.php/jaasd/article/view/223
4. Alluri, R. K. (2022). Universal Data Engineering Frameworks for Cross-Platform Fraud Detection. Journal of Artificial Intelligence & Cloud Computing, 1–6. https://doi.org/10.47363/jaicc/2022(1)468
5. Amin Ifath, M. M., Neves, M., & Haque, I. (2023). Fast Prototyping of Distributed Stream Processing Applications with stream2gym. In Proceedings - International Conference on Distributed Computing Systems (Vol. 2023-July, pp. 395–405). Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/ICDCS57875.2023.00034
6. An Event-Driven Mobile Application for Gas Leakage Monitoring and Detection using Firebase and IoT. (2023). International Journal of Advanced Trends in Computer Science and Engineering, 13(1), 24–31. https://doi.org/10.30534/ijatcse/2024/041312024
7. Branco, B., Abreu, P., Gomes, A. S., Almeida, M. S. C., Ascensão, J. T., & Bizarro, P. (2020). Interleaved Sequence RNNs for Fraud Detection. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 3101–3109). Association for Computing Machinery. https://doi.org/10.1145/3394486.3403361
8. Carcillo, F., Dal Pozzolo, A., Le Borgne, Y. A., Caelen, O., Mazzer, Y., & Bontempi, G. (2018). SCARFF: A scalable framework for streaming credit card fraud detection with spark. Information Fusion, 41, 182–194. https://doi.org/10.1016/j.inffus.2017.09.005
9. Dusi, M., D’Heureuse, N., Huici, F., Di Pietro, A., Bonelli, N., Bianchi, G., … Niccolini, S. (2012). Blockmon: Flexible and high-performance big data stream analytics platform and its use cases. NEC Technical Journal, 7(2), 102–106.
10. Fuchs, A., Fuchs, K., Gwinner, F., & Winkelmann, A. (2021). A meta-model for real-time fraud detection in ERP systems. In Proceedings of the Annual Hawaii International Conference on System Sciences (Vol. 2020-January, pp. 7112–7121). IEEE Computer Society. https://doi.org/10.24251/hicss.2021.856
11. Gómez, H. D., Garcia-Rodriguez, J., Azorin-Lopez, J., Tomás, D., Fuster-Guillo, A., & Mora-Mora, H. (2021). IA-CPS: Intelligent architecture for cyber-physical systems management. Journal of Computational Science, 53, 101409. https://doi.org/10.1016/j.jocs.2021.101409
12. Kakaraparthi, N., & Augie, M. A. (2022). under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 (CC BY-NC-ND 4.0) International License Technical Review: Kafka-Driven AI Architectures for Stream Processing. Sarcouncil Journal of Engineering and Computer Sciences. Retrieved from https://doi.org/10.5281/zenodo.15862347
13. Kancharla, C. R., Vanoost, D., Vankeirsbilck, J., Boydens, J., & Hallez, H. (2023). (POSTER) Unsupervised and Condition Invariant Anomaly Detection for The Constrained Edge. In Proceedings - 19th International Conference on Distributed Computing in Smart Systems and the Internet of Things, DCOSS-IoT 2023 (pp. 77–79). Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/DCOSS-IoT58021.2023.00022
14. Kauffman, S., Dunne, M., Gracioli, G., Khan, W., Benann, N., & Fischmeister, S. (2021). Palisade: A framework for anomaly detection in embedded systems. Journal of Systems Architecture, 113. https://doi.org/10.1016/j.sysarc.2020.101876
15. Kumar, R. (2023). Event-Driven Architectures for Real-Time Data Processing: A Deep Dive into System Design and Optimization. IJIRCT: International Journal of Innovative Research and Creative Technology, 9(2), 1–18.
16. Kurt, Ç., & Ayhan Erdem, O. (2020). Real-time anomaly detection and mitigation using streaming telemetry in SDN. Turkish Journal of Electrical Engineering and Computer Sciences, 28(5), 2448–2466. https://doi.org/10.3906/ELK-1909-112
17. Krishna Inampudi, R., & Researcher, I. (2022). AI-Enhanced Fraud Detection in Real-Time Payment Systems: Leveraging Machine Learning and Anomaly Detection to Secure Digital Transactions. Australian Journal of Machine Learning Research & Applications By Sydney Academics 483 Australian Journal of Machine Learning Research & Applications, 2.
18. Manunza, L., Marseglia, S., & Romano, S. P. (2017). Kerberos: A real-time fraud detection system for IMS-enabled VoIP networks. Journal of Network and Computer Applications, 80, 22–34. https://doi.org/10.1016/j.jnca.2016.12.018
19. Mohamed, A., Najafabadi, M. K., Wah, Y. B., Zaman, E. A. K., & Maskat, R. (2020). The state of the art and taxonomy of big data analytics: view from new big data framework. Artificial Intelligence Review, 53(2), 989–1037. https://doi.org/10.1007/s10462-019-09685-9
20. Nakamura, K., Hayashi, A., & Matsutani, H. (2016). An FPGA-based low-latency network processing for spark streaming. In Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016 (pp. 2410–2415). Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/BigData.2016.7840876
21. Pham, V. N., Hossain, M. D., Lee, G. W., & Huh, E. N. (2023). Efficient Data Delivery Scheme for Large-Scale Microservices in Distributed Cloud Environment. Applied Sciences (Switzerland), 13(2). https://doi.org/10.3390/app13020886
22. Pranitha Gadam. (2023). Real-Time Fraud Detection in Serverless Financial Systems Using AI. International Journal of Advanced Research in Science, Communication and Technology, 716–721. https://doi.org/10.48175/ijarsct-13700l
23. Potla, R. T. (2023). "AI in Fraud Detection: Leveraging Real-Time Machine Learning for Financial Security. " Journal of Artificial Intelligence Research and Applications, 3(2), 534–549.
24. Real-time Data Stream Processing - Challenges and Perspectives. (2017). International Journal of Computer Science Issues, 14(5), 6–12. https://doi.org/10.20943/01201705.612
25. Solaimani, M., Iftekhar, M., Khan, L., Thuraisingham, B., Ingram, J., & Seker, S. E. (2016). Online anomaly detection for multi-source VMware using a distributed streaming framework. Software - Practice and Experience, 46(11), 1479–1497. https://doi.org/10.1002/spe.2390
26. Solaimani, M., Iftekhar, M., Khan, L., & Thuraisingham, B. (2014). Statistical technique for online anomaly detection using Spark over heterogeneous data from multi-source VMware performance data. In Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014 (pp. 1086–1094). Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/BigData.2014.7004343
27. Sanjay Lote, Praveena K B, & Durugappa Patrer. (2022). Real-time data stream processing in large-scale systems. World Journal of Advanced Research and Reviews, 15(3), 560–570. https://doi.org/10.30574/wjarr.2022.15.3.0903
28. Shao, W., Wei, Y., Rajapaksha, P., Li, D., Luo, Z., & Crespi, N. (2023). Low-Latency Dimensional Expansion and Anomaly Detection Empowered Secure IoT Network. IEEE Transactions on Network and Service Management, 20(3), 3865–3879. https://doi.org/10.1109/TNSM.2023.3246798
29. Tinini, R. I., Santos, M. R. P. dos, Figueiredo, G. B., & Batista, D. M. (2020). 5GPy: A SimPy-based simulator for performance evaluations in 5G hybrid Cloud-Fog RAN architectures. Simulation Modelling Practice and Theory, 101. https://doi.org/10.1016/j.simpat.2019.102030
30. V.Y., R., & Yu, K. D. (2018). Technological Support of Real-Time Interaction in Web Clients of Analytical Fraud Detection Systems. KnE Social Sciences, 3(2), 450. https://doi.org/10.18502/kss.v3i2.1576
31. Witte, P. A., Louboutin, M., Modzelewski, H., Jones, C., Selvage, J., & Herrmann, F. J. (2020). An event-driven approach to serverless seismic imaging in the cloud. IEEE Transactions on Parallel and Distributed Systems, 31(9), 2032–2049. https://doi.org/10.1109/TPDS.2020.2982626
32. Sannareddy, Sai Bharath. (2024). Autonomous Kubernetes Cluster Healing using Machine Learning. International Journal of Research Publications in Engineering, Technology and Management (IJRPETM), 7(5), 11171–11180. https://doi.org/10.15662/IJRPETM.2024.0705006
33. Sannareddy, Sai Bharath., & Sunkari, Suresh. (2025). A Unified Multi-Signal Correlation Architecture for Proactive Detection of Azure Cloud Platform Outages. Eastasouth Journal of Information System and Computer Science (ESISCS), 3(02), 191–201. https://doi.org/10.58812/esiscs.v3i02.845
34. Sannareddy, Sai Bharath. (2024). GenAI-Driven Observability and Incident Response Control Plane for Cloud-Native Systems. International Journal of Research and Applied Innovations (IJRAI), 7(6), 11817–11828. https://doi.org/10.15662/IJRAI.2024.0706027
35. B. K. Alti, “An AI-Based Control Framework for Secure and Compliant Linux Systems in Financial Services,” Review of Contemporary Philosophy, vol. 24, no. 1, pp. 921–934, 2025. DOI: https://doi.org/10.52783/rcp.1346
36. B. K. Alti, “A Policy-Driven Architecture for Enterprise-Scale Patch and Configuration Governance Using Red Hat Satellite,” Letters in High Energy Physics, vol. 2024, Article ID 8141, Feb. 2024. DOI: https://doi.org/10.52783/lhep.2024.1606
37. B. K. Alti, “Architectural Tradeoffs in Migrating Enterprise File Workloads to Object Storage,” International Journal on Recent and Innovation Trends in Computing and Communication, vol. 12, no. 2, pp. 1163–1172, Aug. 2024. DOI: https://doi.org/10.17762/ijritcc.v12i2.11822
38. B. K. Alti, “Systematic Enforcement of CIS-Aligned Security Controls for Kubernetes Worker Nodes,” The Eastasouth Journal of Information System and Computer Science, Vol. 1, No. 01, August, pp. 156 – 168, Aug. 2023 DOI: https://esj.eastasouth-institute.com/index.php/esiscs/article/view/864
39. B. K. Alti, “Continuous Security Validation of Linux Systems Using Configuration-as-Code,” The Eastasouth Journal of Information System and Computer Science, Vol. 1, No. 02, December, pp. 184-193 DOI: https://esj.eastasouth-institute.com/index.php/esiscs/article/view/863
40. Xu, Y., & Helal, A. (2016). Scalable Cloud-Sensor Architecture for the Internet of Things. IEEE Internet of Things Journal, 3(3), 285–298. https://doi.org/10.1109/JIOT.2015.2455555
41. Yadav, S. (2023). AI-Powered Fraud Detection in Financial Transactions. International Journal on Science and Technology (IJSAT) IJSAT23041216, 14(4), 148–156.
42. Yang, C., Du, Z., Meng, X., Zhang, X., Hao, X., & Bader, D. A. (2023). Anomaly Detection in Catalog Streams. IEEE Transactions on Big Data, 9(1), 294–311. https://doi.org/10.1109/TBDATA.2022.3161925
43. Zorion, P. K., Sachan, L., Chhabra, R., Pandey, V., & Fatima, Dr. H. (2023). Credit Card Financial Fraud Detection Using Deep Learning. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.4629093
