5,771 Hits in 3.9 sec

In support of workload-aware streaming state management

Vasiliki Kalavri, John Liagouris
2020 USENIX Workshop on Hot Topics in Storage and File Systems  
This paper surfaces the limitations of established practices for streaming state management and advocates for configurable streaming backends, tailored to the state requirements of each operator.  ...  We question the suitability of such general-purpose stores for streaming workloads and argue that they incur unnecessary overheads in exchange for state management capabilities.  ...  A flexible testbed for state management To explore the potentials of a more flexible state management approach, we have implemented a testbed on top of the Timely Dataflow stream processor [19] .  ... 
dblp:conf/hotstorage/KalavriL20 fatcat:4qebhihdmjbixci6z7mfwedkku

Dataflow Management in the Internet of Things: Sensing, Control, and Security

Dawei Wei, Huansheng Ning, Feifei Shi, Yueliang Wan, Jiabo Xu, Shunkun Yang, Li Zhu
2021 Tsinghua Science and Technology  
Then, we illustrate and compare representative tools or platforms for IoT dataflow management.  ...  In this paper, we provide an overall review of IoT dataflow management.  ...  To elaborate on the state of the art of management platforms for IoT dataflow, we compare the above mentioned platforms.  ... 
doi:10.26599/tst.2021.9010029 fatcat:a7j6puazezge5ifnnruyqmimp4

Failure Analysis in a highly parallel processor for Ll Triggering

G. Cancelo, E. Gottschalk, V. Pavlicek, M. Wang, J. Wu
2003 2003 IEEE Nuclear Science Symposium. Conference Record (IEEE Cat. No.03CH37515)  
The failure analysis is crucial for a system with over 2500 processing nodes and a number of storage units and communication links of the same order of magnitude.  ...  The failure analysis is based on models of the L1 Trigger architecture and shows the dynamics of the architecture's dataflow.  ...  That is, faster processors or a larger number of them per Farmlet will lower the time constants of the exponentials and the system approaches faster to its steady state.  ... 
doi:10.1109/nssmic.2003.1351928 fatcat:aihomhz4uzhatm4fhvj5ud6lxy

An AppGallery for dataflow computing

Nemanja Trifunovic, Veljko Milutinovic, Nenad Korolija, Georgi Gaydadjiev
2016 Journal of Big Data  
For state of the art, see [15] [16] .  ...  Therefore, the times have arrived for the technology to enable the dataflow approach to be effective.  ... 
doi:10.1186/s40537-015-0038-8 fatcat:plawxfn3ivdkncaxxiub27ibdq

Fast Hardware Implementation of an Hadamard Transform Using RVC-CAL Dataflow Programming

Khaled Jerbi, Matthieu Wipliez, Mickael Raulet, Olivier Deforges, Marie Babel, Mohamed Abid
2010 2010 5th International Conference on Embedded and Multimedia Computing  
The objective was to spend the most part of the conception time in an open source software platform.  ...  The final step consists in an automatic generation of an efficient hardware implementation from the dataflow program.  ...  These structures have to be manually modified into several actions managed by finite state machine.  ... 
doi:10.1109/emc.2010.5575731 fatcat:fl6uwhfx4fhdnpu2l6bhztgzjm

Cavs: A Vertex-centric Programming Interface for Dynamic Neural Networks [article]

Hao Zhang, Shizhen Xu, Graham Neubig, Wei Dai, Qirong Ho, Guangwen Yang, Eric P. Xing
2017 arXiv   pre-print
Existing dataflow-based programming models for DL---both static and dynamic declaration---either cannot readily express these dynamic models, or are inefficient due to repeated dataflow graph construction  ...  Experiments comparing Cavs to two state-of-the-art frameworks for dynamic NNs (TensorFlow Fold and DyNet) demonstrate the efficacy of this approach: Cavs achieves a near one order of magnitude speedup  ...  We plot for one epoch, both the (averaged) absolute time for graph construction and it percentage of the overall time.  ... 
arXiv:1712.04048v1 fatcat:uha5kzwolzh6xidsk3lgqfd6y4

Cavs: An Efficient Runtime System for Dynamic Neural Networks

Shizhen Xu, Hao Zhang, Graham Neubig, Wei Dai, Jin Kyu Kim, Zhijie Deng, Qirong Ho, Guangwen Yang, Eric P. Xing
2018 USENIX Annual Technical Conference  
Experiments comparing Cavs to state-of-the-art frameworks for dynamic NNs (TensorFlow Fold, PyTorch and DyNet) demonstrate the efficacy of our approach: Cavs achieves a near one order of magnitude speedup  ...  However, existing DL programming models are inefficient in handling dynamic network architectures because of: (1) substantial overhead caused by repeating dataflow graph construction and processing every  ...  Similarly, in ∂ F, the gradient derivation for model parameters are mostly lazy -their execution can be deferred as long as the gradients of hidden states are derived and propagated in time.  ... 
dblp:conf/usenix/XuZN0KDHYX18 fatcat:tmihvtu625d7bax3wfpd5uh5sa

Mobile Apps: It's Time to Move Up to CondOS

David Chu, Aman Kansal, Jie Liu, Feng Zhao
2011 USENIX Workshop on Hot Topics in Operating Systems  
Memory Management: Memory is scarce on mobile devices, yet users are demanding multi-tasking and faster app startup times.  ...  For example, in the context standing, the memory manager might learn over time to prefer preloading games for some users, and ebook apps for others since it is easy to identify what the user actually loads  ... 
dblp:conf/hotos/ChuKLZ11 fatcat:h7da2k6seneqpcjshecmzmmpzu

Apache Flink™: Stream and Batch Processing in a Single Engine

Paris Carbone, Asterios Katsifodimos, Stephan Ewen, Volker Markl, Seif Haridi, Kostas Tzoumas
2015 IEEE Data Engineering Bulletin  
These continuous streams of data come for example from web logs, application logs, sensors, or as changes to application state in databases (transaction log records).  ...  (machine learning, graph analysis) can be expressed and executed as pipelined fault-tolerant dataflows.  ...  Stream Analytics on Top of Dataflows Flink's DataStream API implements a full stream-analytics framework on top of Flink's runtime, including the mechanisms to manage time such as out-of-order event processing  ... 
dblp:journals/debu/CarboneKEMHT15 fatcat:xzgvdr6pljctzb75xecvg74m3q

Rapid prototyping for digital signal processing systems using Parameterized Synchronous Dataflow graphs

Hsiang-Huang Wu, Hojin Kee, Nimish Sane, William Plishker, Shuvra S. Bhattacharyya
2010 Proceedings of 2010 21st IEEE International Symposium on Rapid System Protyping  
By building on the DIF (Dataflow Interchange Format), which is a design language and associated software package for developing and experimenting with dataflow-based design techniques for signal processing  ...  Parameterized Synchronous Dataflow (PSDF) has been used previously for abstract scheduling and as a model for architecting embedded software and FPGA implementations.  ...  National Science Foundation, Award Number 0720596, and the Laboratory for Telecommunications Science.  ... 
doi:10.1109/rsp.2010.5656423 dblp:conf/rsp/WuKSPB10 fatcat:p3lm2v5cybgjrke7lhf5cwysaq

Shared Arrangements: practical inter-query sharing for streaming dataflows [article]

Frank McSherry and Andrea Lattuada and Malte Schwarzkopf and Timothy Roscoe
2020 arXiv   pre-print
We implement shared arrangements in a modern stream processor and show order-of-magnitude improvements in query response time and resource consumption for interactive queries against high-throughput streams  ...  Current systems for data-parallel, incremental processing and view maintenance over high-rate streams isolate the execution of independent queries.  ...  We thank Natacha Crooks, Jon Howell, Michael Isard, and the MIT PDOS group for their valuable feedback, and the many users of DD who exercised and informed its design.  ... 
arXiv:1812.02639v3 fatcat:7jvlhrceofahpikttojhcxsepq

A Relational Approach to Complex Dataflows

Yannis Chronis, Yannis Foufoulas, Vaggelis Nikolopoulos, Alexandros Papadopoulos, Lefteris Stamatogiannakis, Christoforos Svingos, Yannis E. Ioannidis
2016 International Conference on Extending Database Technology  
Exareme is designed to take advantage of clouds by dynamically allocating and deallocating compute resources, offering trade-offs between execution time and monetary cost.  ...  Clouds have become an attractive platform for highly scalable processing of Big Data, especially due to the concept of elasticity, which characterizes them.  ...  The Resource Manager is responsible for the allocation and deallocation of VMs based on the demand.  ... 
dblp:conf/edbt/ChronisFNPSSI16 fatcat:ff4fk4kfmrf6zaiiblqr3vdwfy

$$\mathbb {ECHO}$$ : An Adaptive Orchestration Platform for Hybrid Dataflows across Cloud and Edge [chapter]

Pushkara Ravindra, Aakash Khochare, Siva Prakash Reddy, Sarthak Sharma, Prateeksha Varshney, Yogesh Simmhan
2017 Lecture Notes in Computer Science  
It manages the application's lifecycle, including container-based deployment and a registry for state management.  ...  The Internet of Things (IoT) is offering unprecedented observational data that are used for managing Smart City utilities.  ...  We would also like to thank Venkatesh Babu and Avishek from the VAL lab at IISc for inputs on the video analytics deep-learning model.  ... 
doi:10.1007/978-3-319-69035-3_28 fatcat:5uspil6byrc7par2i7gnylow7q

The Multi-Dataflow Composer Tool: an open-source tool suite for Optimized Coarse-Grain Reconfigurable Hardware Accelerators and Platform Design

Carlo Sau, Tiziana Fanni, Claudio Rubattu, Luigi Raffo, Francesca Palumbo
2020 Microprocessors and microsystems  
Such requirements pushed designers towards the adoption of heterogeneous and reconfigurable substrates, which development and management is not that straightforward.  ...  This paper is focused on the Multi-Dataflow Composer (MDC) tool, that intends to solve issues related to design, optimization and operation of coarse-grain reconfigurable hardware accelerators and their  ...  Luca Fanni, a former employ of the University of Sassari, for providing the initial MATLAB code used to derived the  ... 
doi:10.1016/j.micpro.2020.103326 fatcat:rtjvmftv5rbipc74yg2wq3vorm

Fluχ: a quality-driven dataflow model for data intensive computing

Sérgio Esteves, João Nuno Silva, Luís Veiga
2013 Journal of Internet Services and Applications  
Such data processing schemes are often governed by complex dataflow systems, which are deployed atop highly-scalable infrastructures that need to manage data efficiently in order to enhance performance  ...  Also, this notion can be specially beneficial in cloud computing, where a dataflow computing service (SaaS) may provide certain QoD levels for different budgets.  ...  Session Management It manages the configurations of the QoD constraints, over data objects, through the meta-data that is provided, along with the dataflow specification, and defined for each different  ... 
doi:10.1186/1869-0238-4-12 fatcat:jz3mjxua4zau7mgvlhdxczaomq
« Previous Showing results 1 — 15 out of 5,771 results