A performance benchmark for NetFlow data analysis on distributed stream processing systems

Milan Cermak, Daniel Tovarnak, Martin Lastovicka, Pavel Celeda
2016 NOMS 2016 - 2016 IEEE/IFIP Network Operations and Management Symposium  
Modern distributed stream processing systems can potentially be applied to real time network flow processing. However, differences in performance make some systems more suitable than others for being applied to this domain. We propose a novel performance benchmark, which is based on common security analysis algorithms of NetFlow data to determine the suitability of distributed stream processing systems. Three of the most used distributed stream processing systems are benchmarked and the results
more » ... ked and the results are compared with NetFlow data processing challenges and requirements. The benchmark results show that each system reached a sufficient data processing speed using a basic deployment scenario with little to no configuration tuning. Our benchmark, unlike any other, enables the performance of small structured messages to be processed on any stream processing system.
doi:10.1109/noms.2016.7502926 dblp:conf/noms/CermakTLC16 fatcat:ztmv4mqd4jeyrnrglnbofy3fy4