A Sequence-Oriented Stream Warehouse Paradigm for Network Monitoring Applications [chapter]

Lukasz Golab, Theodore Johnson, Subhabrata Sen, Jennifer Yates
2012 Lecture Notes in Computer Science  
Network administrators are faced with the increasingly challenging task of monitoring their network's health in real time, drawing upon diverse and voluminous measurement data feeds and extensively mining them. The role of database systems in network monitoring has traditionally been that of data repositories; even if an application uses a database, the application logic is implemented using external programs. While such programs are flexible, they tend to be adhoc, opaque, inefficient and hard
more » ... to maintain over time. In this paper, we propose a new way of implementing network monitoring applications: directly within a database as continually updated tables defined using a declarative query language (SQL). We also address a crucial technical issue with realizing this approach: SQL was designed for set-oriented data transformations, but network monitoring involves sequence-oriented analysis. To solve this problem, we propose an extension to SQL that makes sequence-oriented analysis easier to express and faster to evaluate. Using a prototype implementation in a large-scale production data warehouse, we demonstrate how the declarative sequence-oriented query language simplifies application development and how the associated system optimizations improve application performance.
doi:10.1007/978-3-642-28537-0_6 fatcat:kpxxze73eff47e7ykq5ckuyvh4