10,868 Hits in 5.8 sec

Discretized streams

Matei Zaharia, Tathagata Das, Haoyuan Li, Timothy Hunter, Scott Shenker, Ion Stoica
2013 Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles - SOSP '13  
Running these applications at ever-larger scales requires parallel platforms that automatically handle faults and stragglers.  ...  We propose a new processing model, discretized streams (D-Streams), that overcomes these challenges.  ...  Large-scale Streaming While several recent systems enable streaming computation with high-level APIs similar to D-Streams, they also lack the fault and straggler recovery benefits of the discretized stream  ... 
doi:10.1145/2517349.2522737 dblp:conf/sosp/ZahariaDLHSS13 fatcat:tndskypg6rfhta755bt5q2tk3e

Evaluating the impact of a coordinated checkpointing in distributed data streams processing systems using discrete event simulation

Matheus Bernardelli de Moraes, André Leon Sampaio Gradvohl
2020 Revista Brasileira de Computação Aplicada  
In this paper, we propose a discrete simulation model for investigating the impacts of the Coordinated Checkpoint fault tolerance strategy imposes on Data Stream Processing Systems.  ...  To guarantee systems' dependability and avoid information loss, one must use a fault-tolerance strategy.  ...  Nevertheless, practical evaluation of fault-tolerance mechanisms in large-scale applications such as DaSP systems is challenging.  ... 
doi:10.5335/rbca.v12i2.10295 fatcat:wg22iuzhebhwlhndgkztohz75y

Efficiency of Stream Processing Engines for Processing BIGDATA Streams

B. V. S. Srikanth, V. Krishna Reddy
2016 Indian Journal of Science and Technology  
problems of Fault-Tolerance 10 , Faults & Stragglers within the second-scale latency, and maintain high-availability, data processed scalability with better performance of stream processing.  ...  Keywords: Fault-Tolerance, RDD, Second-Scale Latency and Immutable Datasets, Stragglers In-Memory Data Processing: The concept of in-memory data processing provides the high performance of complex event  ... 
doi:10.17485/ijst/2016/v9i14/84797 fatcat:dji434lx6nb4jdfyvwt63kn5qm

Consistent and efficient output-streams management in optimistic simulation platforms

Francesco Antonacci, Alessandro Pellegrini, Francesco Quaglia
2013 Proceedings of the 2013 ACM SIGSIM conference on Principles of advanced discrete simulation - SIGSIM-PADS '13  
vs CPU-bound behaviors at the application level.  ...  At the same time, the delay for materializing the output stream (making it available for any type of audit activity) is shown to be fairly limited and constant, especially for good mixtures of I/O-bound  ...  Still in the context of fault tolerance, the work in [11] presents an innovative protocol for supporting output commitment specifically aimed at promptly delivering output data, which is one of the target  ... 
doi:10.1145/2486092.2486133 dblp:conf/pads/AntonacciPQ13 fatcat:6zkvvdpdqvbxpgho4mpndoybb4

Maintaining balanced trees for structured distributed streaming systems

F. Giroire, R. Modrzejewski, N. Nisse, S. Pérennes
2017 Discrete Applied Mathematics  
Moreover, to minimize the delay of the streaming, the depth of the diffusion tree must also be controlled.  ...  The motivation comes from live distributed streaming systems in which a source diffuses a content to peers via a tree, a node forwarding the data to its children.  ...  The algorithm proposed in this paper aims at tolerating the high churn in a live distributed streaming system.  ... 
doi:10.1016/j.dam.2017.07.006 fatcat:gxlusd3ffba7vokpxuey3qobzy

Cloud-based data stream processing

Thomas Heinze, Leonardo Aniello, Leonardo Querzoni, Zbigniew Jerzak
2014 Proceedings of the 8th ACM International Conference on Distributed Event-Based Systems - DEBS '14  
Specifically, we focus on novel approaches for (1) scalability and (2) fault tolerance in large scale distributed streaming systems.  ...  In general, new fault tolerance mechanisms strive to be more robust and at the same time introduce less overhead.  ...  Data streaming systems aimed at cloud platforms are no exception, and must thus be designed to efficiently cope with faults. Faults may appear in a data streaming system at several different places.  ... 
doi:10.1145/2611286.2611309 dblp:conf/debs/HeinzeAQJ14 fatcat:u67igik235fqdjngoraiukucr4

An Improved Stream Processing Access

Many real time streaming approaches are emerging to utilize or process large real-time data by replacing legacy centralized scenarios which are causing more memory utilization, delay and fault tolerance  ...  In this paper we are focusing on improving stream processing techniques, limitations and future research directions for real-time stream processing  ...  Some of the API's are lagging in terms of backpressure, flow control and time stamps, fault tolerance, state and memory utilization, stream partitioning and so on.  ... 
doi:10.35940/ijitee.k1147.09811s19 fatcat:hxbafhbeabbxpigf3mystx4h6m

The Stream Software Connector Design Space: Frameworks and Languages for Distributed Stream Processing

Masiar Babazadeh, Cesare Pautasso
2014 2014 IEEE/IFIP Conference on Software Architecture  
On the other side, the gaps in the design space we identify point at future research directions in the area of distributed stream processing.  ...  This paper introduces the design space of the stream software connector by analyzing recent stream processing engine frameworks and domain specific languages featuring native streaming support.  ...  Design Issue: Fault Tolerance Fault tolerance is a very important aspect for a stream processing system.  ... 
doi:10.1109/wicsa.2014.42 dblp:conf/wicsa/BabazadehP14 fatcat:kkd7mmuglfh2desuhot2hjdviy

Fault Tolerance for Stream Processing Engines [article]

Muhammad Anis Uddin Nasir
2020 arXiv   pre-print
Finally, we discuss implications of the fault tolerance techniques for different streaming application requirements.  ...  We discuss fault tolerance techniques that are used in modern stream processing engines that are Storm, S4, Samza, SparkStreaming and MillWheel.  ...  The set of computational slices defines a computational state of an operator, where each slice is a computationally-independent unit associated with the input stream.  ... 
arXiv:1605.00928v3 fatcat:kvdgebicrfbktogtew77mv7ppy

ChronoStream: Elastic stateful stream computation in the cloud

Yingjun Wu, Kian-Lee Tan
2015 2015 IEEE 31st International Conference on Data Engineering  
tenants. 1 add (remove) computing nodes in a system, aka scale out (in) 2 add (remove) resources at a single node in a system, aka scale up (down)  ...  We introduce ChronoStream, a distributed system specifically designed for elastic stateful stream computation in the cloud.  ...  Fault tolerance. Several authors have discussed possible fault-tolerance strategies for DSPS. As a pioneering team in exploring the stream database field, Hwang et al.  ... 
doi:10.1109/icde.2015.7113328 dblp:conf/icde/WuT15 fatcat:7l236mp4krgxljami3tptelkee

Scalable stateful stream processing for smart grids

Raul Castro Fernandez, Matthias Weidlich, Peter Pietzuch, Avigdor Gal
2014 Proceedings of the 8th ACM International Conference on Distributed Event-Based Systems - DEBS '14  
In addition, our solution is fault-tolerant, ensuring that the large processing state of stream operators is not lost after failure.  ...  When we scale out the system, the time reduces linearly to 30 minutes before the system bottlenecks at the data source.  ...  Naiad [10] can scale to many nodes but is not designed to be fault tolerant when the managed state is large.  ... 
doi:10.1145/2611286.2611326 dblp:conf/debs/FernandezWPG14 fatcat:3q2gnupre5ephi7tli6zzumkpe

Revenue Models for Streaming Applications over Shared Clouds

Rafael Tolosana-Calasanz, Jose Angel Banares, Congduc Pham, Omer F. Rana
2012 2012 IEEE 10th International Symposium on Parallel and Distributed Processing with Applications  
When multiple users execute their streaming applications over a shared Cloud infrastructure, the provider typically captures the Quality of Service (QoS) for each application at a Service Level Agreement  ...  In this paper, we analyse revenue models for in-transit streaming applications, executed over a shared Cloud infrastructure under the presence of faulty computational resources.  ...  Data streams in such applications are generally large-scale and distributed, and generated continuously at a rate that cannot be estimated in advance.  ... 
doi:10.1109/ispa.2012.67 dblp:conf/ispa/Tolosana-CalasanzBPR12 fatcat:wyhn76z2t5fobggwkm7qisvh7m

Distributed Streaming Analytics on Large-scale Oceanographic Data using Apache Spark [article]

Janak Dahal, Elias Ioup, Shaikh Arifuzzaman, Mahdi Abdelguerfi
2019 arXiv   pre-print
We also verify the fault tolerance by stopping nodes in the middle of a job and making sure that the job is rescheduled and completed on other nodes.  ...  In this paper, we analyze large-scale geo-temporal data collected from the USGODAE (United States Global Ocean Data Assimilation Experiment) data catalog, and showcase and assess the ability of Spark stream  ...  We measure performance of a system running streaming job with various node failures to access the fault tolerance capability of the Apache Spark streaming.  ... 
arXiv:1907.13264v2 fatcat:ubwr334mwre2pesjs6bdxn23ly

Autonomous, failure-resilient orchestration of distributed discrete event simulations

Matthew Malensek, Zhiquan Sui, Neil Harvey, Shrideep Pallickara
2013 Proceedings of the 2013 ACM Cloud and Autonomic Computing Conference on - CAC '13  
In this paper, we propose an autonomous agent that provides fault tolerance functionality for discrete event simulations by predicting state changes in the simulation and adjusting its fault tolerance  ...  compared to conventional distributed fault tolerance schemes.  ...  To our knowledge, this is the first attempt to incorporate fault tolerance into a discrete event simulation orchestrated using a stream processing engine.  ... 
doi:10.1145/2494621.2494625 dblp:conf/cac/MalensekSHP13 fatcat:gpiqqp76eraava2u7efefzwir4

Tolerating Transient Late-Timing Faults in Cloud-Based Real-Time Stream Processing

Peter Garraghan, Stuart Perks, Xue Ouyang, David McKee, Ismael Solis Moreno
2016 2016 IEEE 19th International Symposium on Real-Time Distributed Computing (ISORC)  
However, such systems struggle with transient late-timing faults -a fault classification challenging to effectively tolerate -that manifests increasingly within large-scale distributed systems.  ...  This work proposes a fault-tolerant approach for QoS-aware data prediction to tolerate transient late-timing faults.  ...  [10] develop a new processing model termed discretized streams (D-Streams). The main objective is a scalable means to tolerate both faults and task stragglers.  ... 
doi:10.1109/isorc.2016.24 dblp:conf/isorc/GarraghanPOMM16 fatcat:szi7w7k3ybeoblsq5hmlrkaeim
« Previous Showing results 1 — 15 out of 10,868 results