In-network, push-based network resource monitoring

Taylor Groves, Dorian Arnold, Yihua He
2013 Proceedings of the Third International Workshop on Network-Aware Data Management - NDM '13  
We present preliminary work from our experiences with distributed, push-based monitoring of networks at Yahoo!. Network switches have grown beyond mere ASICs into machines which support unmodified Linux kernels and familiar user interfaces. These advances have enabled a paradigm shift in network monitoring. In lieu of traditional approaches where network diagnostics were delivered via SNMP we utilize Sysdb of Arista's EOS to implement a push based approach to network monitoring. This leaves the
more » ... individual switches in charge of determining what monitoring data to send and when to send it. With this approach -on-switch collection, dissemination, and analysis of interfaces and protocols become possible. This push based approach reduces the feedback loop of network diagnostics and enables networkaware applications, middleware and resource managers to have access to the freshest available data. Our work utilizes the OpenTSDB monitoring framework to provide a scalable back-end for accessing and storing real-time statistics delivered by on-switch collection agents. OpenTSDB is built on top of Hadoop/HBase, which handles the underlying access and storage for the monitoring system. We wrote two collection agents as prototypes to explore the framework and demonstrate the benefits of push based network monitoring.
doi:10.1145/2534695.2534704 dblp:conf/sc/GrovesAH13 fatcat:o5s75adkqfaxhkhzi44ifhunxq