SAND: A Fault-Tolerant Streaming Architecture for Network Traffic Analytics

Qin Liu, John C.S. Lui, Cheng He, Lujia Pan, Wei Fan, Yunlong Shi
2014 2014 44th Annual IEEE/IFIP International Conference on Dependable Systems and Networks  
Many long-running network analytics applications impose a high-throughput and high reliability requirements on stream processing systems. However, previous stream processing systems cannot sustain high-speed traffic at the core router level. Furthermore, their fault-tolerant schemes cannot provide strong consistency which is essential for network analytics. In this paper, we present the design and implementation of SAND, a fault-tolerant distributed stream processing system for network
more » ... or network analytics. SAND is designed to operate under high-speed network traffic, and it uses a novel checkpointing protocol which can perform failure recovery based on upstream backup and checkpointing. We prove our fault-tolerant scheme provides strong consistency even under multiple node failure. We implement several real-world network analytics applications on SAND, evaluate their performance using network traffic captured from commercial cellular core networks, and demonstrate that SAND can sustain high-speed network traffic and that our fault-tolerant scheme is efficient.
doi:10.1109/dsn.2014.91 dblp:conf/dsn/LiuLHPFS14 fatcat:7hhra6irdnfaljedpsxfrrabl4