Filters








30,500 Hits in 5.8 sec

DYNAMIC STREAMS FOR EFFICIENT COMMUNICATIONS BETWEEN MIGRATING PROCESSES IN A CLUSTER

PASCAL GALLARD, CHRISTINE MORIN
2003 Parallel Processing Letters  
b 2 FIFO LIFO Direct Socket Pipe Char FIFO Low−level Point−to−Point communication system KerNet Dynamic stream TCP/IP Myrinet Netdevice w y ¡ k u i ¶ ¿ B R u s u i l f ¤ R p r ¤ 4 w u í § h | p s j u s  ...  j h | p i u s j l E £ ® j l h s h Y n h s u s ¡ p i w l u f ª listen MPI (TCP) FastEthernet MPI (TCP/KerNet) FastEthernet () accept() socket() bind() socket() bind() connect() Child process  ... 
doi:10.1142/s0129626403001549 fatcat:d2qtndxssbcvjpynq2le6nnafy

Dynamic Streams for Efficient Communications between Migrating Processes in a Cluster [chapter]

Pascal Gallard, Christine Morin
2003 Lecture Notes in Computer Science  
b 2 FIFO LIFO Direct Socket Pipe Char FIFO Low−level Point−to−Point communication system KerNet Dynamic stream TCP/IP Myrinet Netdevice w y ¡ k u i ¶ ¿ B R u s u i l f ¤ R p r ¤ 4 w u í § h | p s j u s  ...  j h | p i u s j l E £ ® j l h s h Y n h s u s ¡ p i w l u f ª listen MPI (TCP) FastEthernet MPI (TCP/KerNet) FastEthernet () accept() socket() bind() socket() bind() connect() Child process  ... 
doi:10.1007/978-3-540-45209-6_129 fatcat:62qbbsq67ze7ba3sto7nbi47oe

Supporting Dynamic Migration in Tightly Coupled Grid Applications

Liang Chen, Qian Zhu, Gagan Agrawal
2006 ACM/IEEE SC 2006 Conference (SC'06)  
This paper considers the problem of supporting and efficiently implementing dynamic resource allocation for tightly-coupled and pipelined applications in a grid environment.  ...  We have extensively evaluated our implementation using three stream data processing applications, and show that the use of LSS allows efficient process migration.  ...  The focus in this community has largely been on centralized processing of a single data stream.  ... 
doi:10.1109/sc.2006.58 fatcat:hdr74xp5b5bs5noctiyck2oo4a

Cluster-based Communication and Load Balancing for Simulations on Dynamically Adaptive Grids

Martin Schreiber, Hans-Joachim Bungartz
2014 Procedia Computer Science  
For data migration, the stack-based intra-cluster communication allows a very low memory footprint for data migration and the RLE leads to efficient updates of connectivity information.  ...  The present paper introduces a new communication and load-balancing scheme based on a clustering of the grid which we use for the efficient parallelization of simulations on dynamically adaptive grids.  ...  Acknowledgements We like to thank the Munich Centre of Advanced Computing for for funding this project by providing computing time on the MAC Cluster.  ... 
doi:10.1016/j.procs.2014.05.209 fatcat:iguhbpp4rnhnddvefqtod57tfi

More on Pipelined Dynamic Scheduling of Big Data Streams

Stavros Souravlas, Sofia Anastasiadou, Stefanos Katsavounis
2020 Applied Sciences  
In this regard, it is important to develop efficient task scheduling algorithms, which reduce the stream processing costs.  ...  In this work, we provide a fast and fair task migration policy while maintaining load balancing and low latency times.  ...  Table 1 . 1 Migration policies in dynamic stream scheduling schemes. Table 2 . 2 Class Table (CT) for N  ... 
doi:10.3390/app11010061 fatcat:mlytqfcgsrcpde3z6gf2trzhqy

Flink-ER: An Elastic Resource-Scheduling Strategy for Processing Fluctuating Mobile Stream Data on Flink

Ziyang Li, Jiong Yu, Chen Bian, Yonglin Pu, Yuefei Wang, Yitian Zhang, Binglei Guo
2020 Mobile Information Systems  
As real-time and immediate feedback becomes increasingly important in tasks related to mobile information, big data stream processing systems are increasingly applied to process massive amounts of mobile  ...  The strategy improves the performance of the platform by dynamically rescaling the cluster and increasing the parallelism of operators based on the processing load.  ...  In conclusion, by dynamically rescheduling resources and efficiently migrating stateful data, Flink-ER promotes cluster throughput via latency constraints.  ... 
doi:10.1155/2020/5351824 fatcat:nb4igucv7vf2ldhiy42tj4tkbe

Transparent Network Connectivity in Dynamic Cluster Environments [chapter]

Xiaodong Fu, Hua Wang, Vijay Karamcheti
2000 Lecture Notes in Computer Science  
and can effectively select the best substrate for implementing application communication requirements. keywords Workstation cluster, process migration, network connectivity, adaptation, API interception  ...  This paper presents the design, implementation, and performance of a transparent network connectivity layer for dynamic cluster environments.  ...  Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright annotation thereon.  ... 
doi:10.1007/10720115_3 fatcat:424n4innjrbxpd6wd2v74b6mle

AutoFlow: Hotspot-Aware, Dynamic Load Balancing for Distributed Stream Processing [article]

Pengqi Lu and Liang Yuan and Yunquan Zhang and Hang Cao and Kun Li
2021 arXiv   pre-print
It incorporates a centralized scheduler which monitors the load balance in the entire dataflow dynamically and implements state migrations correspondingly.  ...  We introduce AutoFlow, an automatic, hotspot-aware dynamic load balance system for streaming dataflows.  ...  Each operator in the logic graph of dataflow will be deployed in a cluster combined with a Python process.  ... 
arXiv:2103.08888v1 fatcat:peu5wnkbbvaeljsfywryoiizja

Online Scheduling and Interference Alleviation for Low-Latency, High-Throughput Processing of Data Streams

Thilina Buddhika, Ryan Stern, Kira Lindburg, Kathleen Ericson, Shrideep Pallickara
2017 IEEE Transactions on Parallel and Distributed Systems  
Data Streams occur naturally in several observational settings and often need to be processed with a low latency.  ...  As the number of streams increases, stream processing computations need to be orchestrated over a collection of machines. Achieving timeliness and high throughput in such settings is a challenge.  ...  Existing work has also relied on the properties of the stream processing graph itself, such as the communication patterns between tasks, in order to derive a more efficient placement plan.  ... 
doi:10.1109/tpds.2017.2723403 fatcat:dltosdpcyzaqnhzzif226yyqri

Dynamic Task Scheduling Scheme for Processing Real-Time Stream Data in Storm Environments

Dojin Choi, Hyeonwook Jeon, Jongtae Lim, Kyoungsoo Bok, Jaesoo Yoo
2021 Applied Sciences  
The proposed scheme performs dynamic scheduling using a heterogeneous cluster consisting of various nodes with different performances.  ...  In this paper, we propose a dynamic task scheduling scheme considering task deadlines and node resources.  ...  In this study, the amount of communication between worker nodes is calculated by monitoring and plotted as a graph.  ... 
doi:10.3390/app11177942 fatcat:ythsi7z2d5bfrahb6bx7ynf64q

Biophysical controls on cluster dynamics and architectural differentiation of microbial biofilms in contrasting flow environments

Iris Hödl, Lorenzo Mari, Enrico Bertuzzo, Samir Suweis, Katharina Besemer, Andrea Rinaldo, Tom J. Battin
2013 Environmental Microbiology  
To explore the potential contribution of demographic processes to these architectures, we propose a sizestructured population model to simulate the dynamics of biofilm growth and cluster size distribution  ...  Ecology, with a traditional focus on plants and animals, seeks to understand the mechanisms underlying structure and dynamics of communities.  ...  Acknowledgements We are grateful to Herman Hofreiter and Engelbert Sollböck for the operation of the experimental flumes and to Gabriel Singer for support and advice.  ... 
doi:10.1111/1462-2920.12205 pmid:23879839 pmcid:PMC4231231 fatcat:npfxrykpjvca7bviurk3enjwxu

Predicting the Cost and Benefit of Adapting Data Parallel Applications in Clusters

Jon B. Weissman
2002 Journal of Parallel and Distributed Computing  
This paper examines the problem of adapting data parallel applications in a shared dynamic environment of PC or workstation clusters.  ...  In addition, we dispel the conventional wisdom that migration is too expensive, and show that it can be beneficial even for running parallel applications with non-trivial communication. # 2002 Elsevier  ...  When ghosting is enabled, migration and processor removal will allow the process image to persist for the worker but in a sleep state.  ... 
doi:10.1006/jpdc.2002.1838 fatcat:m5obcfh2m5ckzdpannf5xjw7jy

Persistence and communication state transfer in an asynchronous pipe mechanism

P. Chan, D. Abramson
2007 2007 International Conference on Parallel and Distributed Systems  
These communication issues are addressed by combining adaptive caching with data streaming for efficient and fault-tolerant communication.  ...  This makes it possible for different segments (from cache and from writer) of the pipe data to be concurrently streamed to the migrated reader, reducing the retrieval time.  ...  ACKNOWLEDGMENT We thank the Victorian Partnership for Advanced Computing (VPAC) for the use of their facilities and for their continuing funding to support this project.  ... 
doi:10.1109/icpads.2007.4447734 dblp:conf/icpads/ChanA07 fatcat:2apuxex7vbdoden4zwd3wa4hmq

Persistence and Communication State Transfer in an Asynchronous Pipe Mechanism

Philip Chan, David Abramson
2009 International Journal of Grid and High Performance Computing  
These communication issues are addressed by combining adaptive caching with data streaming for efficient and fault-tolerant communication.  ...  This makes it possible for different segments (from cache and from writer) of the pipe data to be concurrently streamed to the migrated reader, reducing the retrieval time.  ...  ACKNOWLEDGMENT We thank the Victorian Partnership for Advanced Computing (VPAC) for the use of their facilities and for their continuing funding to support this project.  ... 
doi:10.4018/jghpc.2009070102 fatcat:hrfflykpynchtftnstrq7hzvuq

Elasticutor: Rapid Elasticity for Realtime Stateful Stream Processing [article]

Li Wang, Tom Z. J. Fu, Richard T. B. Ma, Marianne Winslett, Zhenjie Zhang
2017 arXiv   pre-print
Elasticity is highly desirable for stream processing systems to guarantee low latency against workload dynamics, such as surges in data arrival rate and fluctuations in data distribution.  ...  Our results show that Elasticutor doubles the throughput and achieves an average processing latency up to 2 orders of magnitude lower than previous methods, for a dynamic workload of real-world applications  ...  However, whenever a shard is assigned to a new task, inter-task communications are needed to migrate the state of the shard between tasks.  ... 
arXiv:1711.01046v1 fatcat:gfdcb5xigza63eyfae4snahvrq
« Previous Showing results 1 — 15 out of 30,500 results