5,137 Hits in 8.1 sec

Integrated recovery and task allocation for stream processing

Hongliang Li, Jie Wu, Zhen Jiang, Xiang Li, Xiaohui Wei, Yuan Zhuang
2017 2017 IEEE 36th International Performance Computing and Communications Conference (IPCCC)  
The failure-free tasks slow down to accelerate a task recovery rather than suspending their actions and waiting for the recovery to finish; waiting causes a complete halt of the application.  ...  Our approach enables continuous processing results and seamless failure recoveries with a constrained slowdown ratio.  ...  In our previous work [23] , we study a task allocation problem that uses the recovery latency as a constraint.  ... 
doi:10.1109/pccc.2017.8280443 dblp:conf/ipccc/LiWJLWZ17 fatcat:6wfqwe7snfb2fp3qk4xe3vouny

Minimum Backups for Stream Processing With Recovery Latency Guarantees

Hongliang Li, Jie Wu, Zhen Jiang, Xiang Li, Xiaohui Wei
2017 IEEE Transactions on Reliability  
With this model, we design an algorithm to compute FTCs for different types of stream topologies according to recovery latency requirements.  ...  This paper introduces the FT Configuration (FTC) problem and presents a solution for guaranteed recovery latency with minimum backups.  ...  Processing Latency and Recovery Latency Modeling Chain [27] is one of the earliest works focusing on the modeling of processing latency and task allocation strategies for stream processing systems.  ... 
doi:10.1109/tr.2017.2712563 fatcat:krmako3yifgi5ps5jf4ve7uz6q

Phoebe: QoS-Aware Distributed Stream Processing through Anticipating Dynamic Workloads [article]

Morgan K. Geldenhuys, Dominik Scheinert, Odej Kao, Lauritz Thamsen
2022 arXiv   pre-print
They are characterized by the high-throughput processing of near to real-time event streams with the goal of delivering low-latency results and thus enabling time-sensitive decision making.  ...  In this paper we present Phoebe, a proactive approach to system auto-tuning for Distributed Stream Processing jobs executing on dynamic workloads.  ...  ACKNOWLEDGMENT This work has been supported through grants by the German Ministry for Education and Research (BMBF) as BIFOLD (funding mark 01IS18025A) and WaterGridSense 4.0 (funding mark 02WIK1475D).  ... 
arXiv:2206.09679v1 fatcat:64tszsslj5hibfja3empjba7re

Avoiding request–request type message-dependent deadlocks in networks-on-chips

Xiaohang Wang, Peng Liu, Mei Yang, Yingtao Jiang
2013 Parallel Computing  
In the literature, several message dependent deadlock avoidance/recovery methods for NoCs have been proposed, including stricting order and end-to-end flow control [8] .  ...  These tasks can be executed in multiple parallel processing units organized in a pipelined fashion to enable higher processing throughput and flexibility.  ...  All the tasks will be executed repeatedly for a number of iterations to process the incoming data stream.  ... 
doi:10.1016/j.parco.2013.05.002 fatcat:rcmpgerrtjdf7obvtwn7kju5re

Reactive Monitoring Adaptation for Dynamic Dataflow on Variable Infrastructure

Chethana R M, Megha G S, Gavina C G, Veeranna Kotagi
2020 Zenodo  
Among these according to the user requirements scheduling the resource is very challenging in cloud environment indeed for the low latency over high velocity data streams.  ...  We propose a Ranking with deadline based algorithm that will implement in the alternative path to provide the end users with more sophisticated control and resource mapping heuristics for communal of dataflow  ...  Many import applications need to process data arriving in real time for distributed stream processing is expensive that treats streaming as a series of short batch jobs to down the latency.  ... 
doi:10.5281/zenodo.3749470 fatcat:4vmhiswr5bfibmpcjonpba6uyi

Liquid Stream Processing Across Web Browsers and Web Servers [chapter]

Masiar Babazadeh, Andrea Gallidabino, Cesare Pautasso
2015 Lecture Notes in Computer Science  
In this paper we present the decentralized control approach followed by the Web Liquid Streams (WLS) framework, a novel framework for streaming applications running on Web browsers, Web servers and smart  ...  While streaming applications can be adapted to run on Web browsers, it remains difficult to deal with temporary disconnections, energy consumption on mobile devices and a potentially very large number  ...  browsers a graph of data stream operators in order to reduce the end to end latency of the stream while enforcing given deployment constraints.  ... 
doi:10.1007/978-3-319-19890-3_3 fatcat:cvi2m77mwvg57e7esjyrbu7gxu

Fault-tolerant stream processing using a distributed, replicated file system

YongChul Kwon, Magdalena Balazinska, Albert Greenberg
2008 Proceedings of the VLDB Endowment  
We present SGuard, a new fault-tolerance technique for distributed stream processing engines (SPEs) running in clusters of commodity servers.  ...  SGuard is less disruptive to normal stream processing and leaves more resources available for normal stream processing than previous proposals.  ...  Geambasu, and the anonymous reviewers for helpful comments on drafts of this paper. This work was partially supported by NSF Grants IIS-0713123, IIS-0454425, and a gift from Cisco Systems Inc.  ... 
doi:10.14778/1453856.1453920 fatcat:4icvod6hwbc6dhvraa5nr3d3xu


Eric Boutin, Paul Brett, Xiaoyu Chen, Jaliya Ekanayake, Tao Guan, Anna Korsun, Zhicheng Yin, Nan Zhang, Jingren Zhou
2015 Proceedings of the VLDB Endowment  
Besides the challenges in massive scalability and low latency distributed query processing, it is imperative to achieve all these requirements with effective fault tolerance and efficient recovery, as  ...  Interactive, reliable, and rich data analytics at cloud scale is a key capability to support low latency data exploration and experimentation over terabytes of data for a wide range of business scenarios  ...  Detecting failures and providing effective failure recovery in a timely manner is crucial for low latency query processing systems.  ... 
doi:10.14778/2824032.2824066 fatcat:bl4yhf6j5zgb5npxzsufzumpgm

Social-insect-inspired adaptive task allocation for many-core systems

Matthew Rowlings, Andy M. Tyrrell, Martin A. Trefzer
2016 2016 IEEE Congress on Evolutionary Computation (CEC)  
This paper not only shows that effective decentralised task allocation is achieved, but also that such a scheme can adapt to faults and alter its behaviour to meet soft real-time constraints.  ...  To investigate this we have explored biological models of task allocation in ant colonies and applied this to a 36-core Network on Chip.  ...  introduces causality into the model and is a more realistic processing stream.  ... 
doi:10.1109/cec.2016.7743887 dblp:conf/cec/RowlingsTT16 fatcat:ain3tisnefcwnbbryavxyepgom

Series Editorial The Fourth Issue of the Series on Machine Learning in Communications and Networks

Geoffrey Y. Li, Walid Saad, Ayfer Ozgur, Peter Kairouz, Zhijin Qin, Jakob Hoydis, Zhu Han, Deniz Gunduz, Jaafar Elmirghani
2022 IEEE Journal on Selected Areas in Communications  
The authors also provide effective pruning methods for the proposed neural network structure.  ...  In [A2], Hussain and Michelusi provide an approach for beam training technique in mm-Wave systems with low overhead.  ...  A novel two-timescale training method is also developed for the proposed DNN with a binary layer.  ... 
doi:10.1109/jsac.2021.3126188 fatcat:6aohhlq55fco5gnndq6cusjbbi

Strategies for Big Data Analytics through Lambda Architectures in Volatile Environments [article]

Alexandre Da Silva Veith, Edison Pignaton de Freitas AVALON
2017 arXiv   pre-print
%An application of batch and stream-based methods are proposed to reduce the latency.  ...  The proposed strategies make use of services such as migration, replication, MapReduce simulation, and combined processing methods (batch- and streaming-based).  ...  A batch method for the stream processing enables to overcome the limitations of BIGhybrid and adapting it to low latency processing.  ... 
arXiv:1708.04796v1 fatcat:ieeqnouvifg5fgecl2ac7yuooa

Mapping the Big Data landscape: Technologies, Platforms and Paradigms for Real-Time Analytics of Data Streams

Timothee Dubuc, Frederic Stahl, Etienne B. Roesch
2020 IEEE Access  
We conclude with a discussion of the field of data stream processing/analytics, future directions and research challenges.  ...  In this article, we evaluate a particular type of solution, which focuses on streaming data, and processing pipelines that permit online analysis of data streams that cannot be stored as-is on the computing  ...  This enables the rollback and recovery of a task in cases where one element is faulty. Apache Samza (see Table 2 ) [63] is another relatively recent stream processing engine.  ... 
doi:10.1109/access.2020.3046132 fatcat:evptdnhmpfahlluemryl5gwouy

Strategies for Big Data Analytics through Lambda Architectures in Volatile Environments

Veith Alexandre da Silva, Anjos Julio C.S. dos, de Freitas Edison Pignaton, Lampoltshammer Thomas J., Geyer Claudio F.
2016 IFAC-PapersOnLine  
The proposed strategies make use of services such as migration, replication, MapReduce simulation, and combined processing methods (batch-and streaming-based).  ...  Via these services, a distribution of tasks for the best balance of computational resources is achieved, while monitoring and management can be performed asynchronously in the background.  ...  A batch method for the stream processing enables to overcome the limitations of BIGhybrid and adapting it to low latency processing.  ... 
doi:10.1016/j.ifacol.2016.11.138 fatcat:zy45z7ztc5f6zbmx4pskr7w5a4

A Survey on Collaborative DNN Inference for Edge Intelligence [article]

Weiqing Ren, Yuben Qu, Chao Dong, Yuqian Jing, Hao Sun, Qihui Wu, Song Guo
2022 arXiv   pre-print
tasks.  ...  Then, we classify four typical collaborative DNN inference paradigms for EI, and analyze the characteristics and key technologies of them.  ...  For EI with low latency, edge combines DNN partitioning and early-exit mechanism to reduce the latency of inference tasks.  ... 
arXiv:2207.07812v1 fatcat:yahjwuowz5erhctsgbderya65m

Passive and Partially Active Fault Tolerance for Massively Parallel Stream Processing Engines

Li Su, Yongluan Zhou
2017 IEEE Transactions on Knowledge and Data Engineering  
a non-negligible latency for streaming data applications.  ...  Even with a PPA plan, postponing the recovery of the passively replicated tasks until the arrival of all the necessary resources would incur a significant latency to a streaming data applications.  ... 
doi:10.1109/tkde.2017.2720602 fatcat:77evpu3oprdurgr43azif3ge2q
« Previous Showing results 1 — 15 out of 5,137 results