User-Constraint and Self-Adaptive Fault Tolerance for Event Stream Processing Systems

Andre Martin, Tiaraju Smaneoto, Tobias Dietze, Andrey Brito, Christof Fetzer
2015 2015 45th Annual IEEE/IFIP International Conference on Dependable Systems and Networks  
Event Stream Processing (ESP) Systems are currently enabling a renaissance in the data processing area as they provide results at low latency compared to the traditional MapReduce approach. Although the majority of ESP systems offer some form of fault tolerance to their users, the provided fault tolerance scheme is often not tailored to the application at hand. For example, active replication is well suited for critical applications where unresponsiveness due to a background recovery process is
more » ... not acceptable. However, for other classes of applications without such tight constraints, the use of passive replication, based on checkpoints and logging, is a better choice as it can save a significant amount of resources compared to active replication. In this paper, we present StreamMine3G, a fault tolerant and elastic ESP system which employs several fault tolerance schemes, such as passive and active replication as well as intermediate alternatives such as active and passive standby. In order to free the user from the burden of choosing the correct scheme for the application at hand, StreamMine3G is equipped with a fault-tolerance controller that transitions between the employed schemes during runtime in response to the evolution of the given workload and the user's provided constraints (recovery time and semantics, i.e., gap or precise). Our evaluation shows that the overall resource footprint for fault tolerance can be considerably reduced using our adaptive approach without consequences to the recovery time.
doi:10.1109/dsn.2015.56 dblp:conf/dsn/MartinSDBF15 fatcat:4bgixhotijdt5cr6qbizdz4bue