Stream4Flow: Real-time IP flow host monitoring using Apache Spark

Tomas Jirsik
2018 NOMS 2018 - 2018 IEEE/IFIP Network Operations and Management Symposium  
In this paper, we present Stream4Flow, a framework for cyber situational awareness based on Apache Spark Streaming. We demonstrate utilization of Stream4Flow for real-time IP flow host monitoring in a large campus network. Contemporary IP flow analysis systems are not designed for the continuous host monitoring. Gaining the detailed overview of each host is not straightforward with these systems due to connection-based paradigm and performance challenges. We show that distributed stream
more » ... ng is a natural solution for detailed IP flow host monitoring. Moreover, we describe a real-time host monitoring workflow in data streams in detail and present advantages of flow-based host monitoring in Apache Spark including real-time host profiling, dynamic level of detail and granularity.
doi:10.1109/noms.2018.8406132 dblp:conf/noms/Jirsik18 fatcat:3ox7nthu4jdvvf6w75mx3qj23u