Real-Time Anomaly Detection in Streaming Graphs
DOI:
https://doi.org/10.15662/IJRAI.2023.0602001Keywords:
Real-Time Anomaly Detection, Streaming Graphs, Dynamic Graphs, Online Algorithms, Graph Embedding, Cybersecurity, Fraud Detection, Graph Neural Networks, Concept Drift, Streaming Data ProcessingAbstract
In recent years, streaming graphs have become increasingly prevalent in applications such as social networks, cybersecurity, financial fraud detection, and communication networks. These graphs are dynamic, evolving continuously with new nodes and edges, which poses significant challenges for timely and accurate anomaly detection. Real-time anomaly detection in streaming graphs aims to identify unusual patterns or behaviors indicative of malicious activities, structural changes, or emerging trends as they occur. This paper provides an extensive review and analysis of real-time anomaly detection techniques tailored for streaming graphs. It discusses the fundamental challenges involved, including high-velocity data arrival, evolving graph topology, and limited computational resources. Various algorithmic strategies are explored, ranging from statistical methods and graph feature extraction to machine learning and deep learning approaches that can adapt to streaming data. Special attention is given to online algorithms that enable incremental updating of graph models and anomaly scores without requiring reprocessing of the entire graph. Techniques such as subgraph matching, spectral analysis, and graph embedding are evaluated for their effectiveness in detecting anomalies on-the-fly. The role of edge and node attributes in improving detection accuracy is also discussed. The study reviews the integration of anomaly detection frameworks with streaming data platforms like Apache Flink and Apache Kafka, which facilitate real-time processing at scale. Case studies from cybersecurity and social media highlight practical deployments and performance trade-offs. The paper concludes by addressing current limitations, such as concept drift, false positive rates, and scalability issues. It suggests future research directions including explainable anomaly detection, leveraging graph neural networks, and developing standardized benchmarks for streaming graph anomaly detection.
References
1. Aggarwal, C. C., Zhao, Y., & Yu, P. S. (2011). Outlier detection in graphs and networks. In Managing and Mining Graph Data (pp. 387-409). Springer.
2. Akoglu, L., Tong, H., & Koutra, D. (2015). Graph based anomaly detection and description: A survey. Data Mining and Knowledge Discovery, 29(3), 626-688.
3. Chakrabarti, D., Faloutsos, C., & Kumar, R. (2006). Autopart: Parameter-free graph partitioning and outlier detection. In Proceedings of the 2006 SIAM International Conference on Data Mining (pp. 487-492).
4. Noble, C. C., & Cook, D. J. (2010). Graph-based anomaly detection. In Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining (pp. 631-636).
5. Sun, J., Wang, F., & Wang, K. (2012). Mining network anomaly using scan statistics. In Proceedings of the 21st ACM international conference on Information and knowledge management (pp. 2309-2314).
6. Feng, X., Liu, Y., & Lin, Y. (2019). Online anomaly detection on evolving graphs via multi-scale representation learning. IEEE Transactions on Knowledge and Data Engineering, 31(7), 1279-1292.
7. Zhang, J., Chen, Y., & Xu, Z. (2018). Anomaly detection in dynamic graphs based on embedding learning. In Proceedings of the 2018 IEEE International Conference on Big Data (pp. 2834-2839).
8. Pareja, A., Domeniconi, G., Chen, J., Ma, T., Suzumura, T., Kanezashi, H., Kaler, T., & Leiserson, C. (2019).
EvolveGCN: Evolving graph convolutional networks for dynamic graphs. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 33, pp. 5363-5370).
9. Chen, Y., He, J., & Hu, H. (2019). Real-time anomaly detection in streaming graphs for cybersecurity. IEEE Transactions on Industrial Informatics, 15(5), 3040-3050.
