44,784 Hits in 5.2 sec

Monitoring Distributed Data Streams through Node Clustering [chapter]

Maria Barouti, Daniel Keren, Jacob Kogan, Yaakov Malinovsky
2014 Lecture Notes in Computer Science  
The proposed approach enables monitoring values of an arbitrary threshold function over distributed data streams through a set of constraints applied independently on each stream and/or clusters of streams  ...  Monitoring data streams in a distributed system is a challenging problem with profound applications.  ...  Conclusions and Future Research Directions In this paper we propose to monitor threshold functions over distributed data streams through clustering nodes whose data fluctuations "cancel out" each other  ... 
doi:10.1007/978-3-319-08979-9_12 fatcat:llamtm5ysjeklcnkst5fyukisu

Monitoring Large Cloud-Based Systems

Mauro Andreolini, Marcello Pietri, Stefania Tosi, Andrea Balboni
2014 Proceedings of the 4th International Conference on Cloud Computing and Services Science  
It integrates a hierarchical scheme to monitor the resources in a cluster with a distributed hash table (DHT) to broadcast system state information among different monitors.  ...  This architecture strives to obtain high scalability, effectiveness and resilience, as well as the possibility of monitoring services spanning across different clusters or even different data centers of  ...  In particular, we aim to find out: • how many resource data streams can be monitored per node (intra-node scalability); • how many nodes can be monitored in a cluster (intra-cluster scalability).  ... 
doi:10.5220/0004794003410351 dblp:conf/closer/AndreoliniPTB14 fatcat:4qjvrxen3vdd7k56nlsfkz2tjy

A Scalable Architecture for Real-Time Monitoring of Large Information Systems

Mauro Andreolini, Michele Colajanni, Marcello Pietri
2012 2012 Second Symposium on Network Cloud Computing and Applications  
We evaluate its scalability by means of an analytical models that is validated through a set of experimental results.  ...  We propose a novel monitoring architecture that, by combining a hierarchical approach with decentralized monitors, addresses the aforementioned challenges.  ...  The distributed cluster data filter. The resource data streams gathered by the collection agent are sent to the Distributed cluster data filter, shown in Figure 2 .  ... 
doi:10.1109/ncca.2012.24 dblp:conf/ncca/AndreoliniCP12 fatcat:73vcti5nvrhwzidpvek6for6ca

Distributed clustering of ubiquitous data streams

Pedro Pereira Rodrigues, João Gama
2013 Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery  
Two subproblems exist: clustering streaming data observations and clustering streaming data sources.  ...  We characterize clustering algorithms for ubiquitous data streams, discussing advantages and disadvantages of distributed procedures.  ...  ACKNOWLEDGMENTS This work is financed by the ERDF-European Regional Development Fund through the COMPETE Programme (operational programme for competitiveness) and by National Funds through the FCT-Fundação  ... 
doi:10.1002/widm.1109 fatcat:4ybb44a6szgkxkhtwni3z3apw4

Distributed AI embedded cluster for real-time video analysis systems with edge computing

Wei Li, Zhiyuan Han, Jian Shen, Dandan Luo, Bo Gao, Jin Xie, J. Zhu
2022 MATEC Web of Conferences  
With ARM cluster server as the hardware platform, a distributed software platform is constructed.  ...  Herein, on the basis of a distributed AI cluster, a real-time video analysis system is proposed for edge computing.  ...  Not only is it involved in task distribution and scheduling, as well as node service status monitoring, it is also intended to visualize data through the provision of user interface.  ... 
doi:10.1051/matecconf/202235503036 fatcat:tqwj735o4vcf5ammf3rzzbmx4y

A Stream-Oriented Real-Time Fault Tolerant Strategy for Monitoring Big Data of Power Distribution Network Based on Spark

Zhijian Qu, Xiang Peng, Qunfeng Wang, Ruilin Zhou
2017 DEStech Transactions on Environment Energy and Earth Science  
Aiming at the existing problem of monitoring data volume blowout and lack of valid real-time distributed fault tolerant strategy in power distribution network, a new fault tolerant strategy based on Spark  ...  Depending on the design technique of resilient distributed data set renewal and lineage mechanism, the lineage chain retrospection in fault tolerance of distributed data is realized.  ...  DISTRIBUTION NETWORK MONITORING INFORMATION STREAM Monitoring information stream of distribution network is assigned to the distribution network with the characteristics of the main transmission line,  ... 
doi:10.12783/dteees/icnerr2017/13310 fatcat:bp2ivedja5f4zc7q45uca5j2ry

Processing High-Volume Stream Queries on a Supercomputer

E. Zeitler, T. Risch
2006 22nd International Conference on Data Engineering Workshops (ICDEW'06)  
The high data volume and the need for advanced computations on the streams require substantial hardware resources and scalable stream processing.  ...  We have developed a data stream management system prototype for state-of-the-art parallel hardware.  ...  stream processors of subqueries if needed, iv) communicating with the backend cluster to retrieve input data, and v) monitoring the execution of its stream query.  ... 
doi:10.1109/icdew.2006.118 dblp:conf/icde/ZeitlerR06 fatcat:jrwejkvspfdfrebuj735f6wbwm

Cluster equilibrium scheduling method based on backpressure flow control in railway power supply systems

Zhijian Qu, Hanxin Liu, Hanlin Wang, Xinqiang Chen, Rui Chi, Zixiao Wang, Sakdirat Kaewunruen
2020 PLoS ONE  
By using telemetry data flow as input data sources, the fields of data sources are segmented into different sets by using a distributed model of stream computing parallel processing.  ...  ., increasingly significant processing delay of massive monitoring data and imbalanced tasks in the scheduling and monitoring center for a railway network.  ...  For example, it is supposed that there are four server nodes in the big data monitoring cluster of a railway power distribution network, with distributed weights of {5,1,1,2}.  ... 
doi:10.1371/journal.pone.0243543 pmid:33296425 fatcat:viixheqbw5hnvjiqm5hjosg7zm

Analysis of Plant Breeding on Hadoop and Spark

Shuangxi Chen, Chunming Wu, Yongmao Yu
2016 Advances in Agriculture  
With this cluster, we achieve crop breeding large data analysis tasks in parallel through API provided by Spark.  ...  We propose a crop breeding data analysis platform on Spark. The platform consists of Hadoop distributed file system (HDFS) and cluster based on memory iterative components.  ...  Via cluster management platform, breeding analysis platform can timely monitor nodes in Hadoop cluster, including hot plug nodes, extendable nodes, intelligent nodes, message and e-mail alerts, smart diagnosis  ... 
doi:10.1155/2016/7081491 fatcat:m67r6na6lfc4bloxaf6ywvvvqq

System-Level Resource Monitoring in High-Performance Computing Environments

Sandip Agarwala, Christian Poellabauer, Jiantao Kong, Karsten Schwan, Matthew Wolf
2003 Journal of Grid Computing  
Using dproc, we show that (a) data streams can be customized according to a client's resource availabilities (dynamic stream management), (b) by dynamically varying distributed monitoring (dynamic filtering  ...  Second, to predictably capture and distribute monitoring information, dproc uses a kernel-level group communication facility, termed KECho, which implements events and event channels.  ...  As distributed memory and cluster computers scale to hundreds or thousands of nodes, the perturbation implied by the frequent exchange of monitoring data can be substantial.  ... 
doi:10.1023/b:grid.0000035189.80518.5d fatcat:nfb3v6xvzrfvhegvpsoozw2ioq

Research and Implementation of Multimedia Server Management in Distributed Multimedia Distance Education System

Xinming Han
2016 International Journal of Emerging Technologies in Learning (iJET)  
Firstly, we take the distributed multimedia technology as the key technology and an important means to facilitate the transmission of streaming media data.  ...  This design can effectively reduce the transmission delay of streaming media data.  ...  The polling scheme takes turns distributing the stream data to each node. 2.  ... 
doi:10.3991/ijet.v11i08.6038 fatcat:z5t46624r5b6xox5zoazeb5taa

IoT Blockchain Data Veracity with Data Loss Tolerance

Kwai Cheong Moke, Tan Jung Low, Dodo Khan
2021 Applied Sciences  
A novel IoT blockchain strength monitoring system is also been studied to further enhance data veracity; this is achieved through a capacitance monitoring on the IoT blockchain system.  ...  That is, to have blockchain enhancing the IoT data veracity by leveraging on the features existed in its peer-to-peer network (P2P) and distributed ledger storage technology (DLT).  ...  Hence, IoT blockchain strength monitoring through its capacitance is set up and evaluated.  ... 
doi:10.3390/app11219978 fatcat:lhl7yf4ncnbxxgvk5kmokctpmi

Processing and Analytics of Big Data Streams with Yahoo!S4

Fatos Xhafa, Victor Naranjo, Santi Caballe
2015 2015 IEEE 29th International Conference on Advanced Information Networking and Applications  
Many Internet-based applications generate huge data streams, which are known as Big Data Streams.  ...  Computational Models and Methods for Massive Structured Data.  ...  ACKNOWLEGMENTS This research has been supported by TIN2013-45303-P "ICT-FLAG" Enhancing ICT education through Formative assessment, Learning Analytics and Gamification and TIN2013-46181-C2-1-R "COMMAS"  ... 
doi:10.1109/aina.2015.194 dblp:conf/aina/XhafaNC15 fatcat:s7dppluze5eifcv7z5uojn7x3q

A Software Architecture for the Analysis of Large Sets of Data Streams in Cloud Infrastructures

Mauro Andreolini, Michele Colajanni, Stefania Tosi
2011 2011 IEEE 11th International Conference on Computer and Information Technology  
System management algorithms in private and public cloud infrastructures have to work with literally thousands of data streams generated from resource, application and event monitors.  ...  address in this paper: how to design a software architecture that is able to gather and analyze all information within real-time constraints; how it is possible to reduce the analysis of the huge collected data  ...  The distributed cluster analyzer. At the heart of the distributed cluster analyzer shown in Figure 2c there is a set of analyzer nodes which execute Pig scripts.  ... 
doi:10.1109/cit.2011.62 dblp:conf/IEEEcit/AndreoliniCT11 fatcat:me7yqfwakbfmnpmogq35c6hsjy

A scalable monitoring for the CMS Filter Farm based on elasticsearch

J-M Andre, A Andronidis, U Behrens, J Branson, O Chaze, S Cittolin, G-L Darlea, C Deldicque, M Dobson, A Dupont, S Erhan, D Gigi (+25 others)
2015 Journal of Physics, Conference Series  
The leaf es clusters consist of the very same nodes that form the Filter Farm thus providing natural horizontal scaling.  ...  A flexible monitoring system has been designed for the CMS File-based Filter Farm making use of modern data mining and analytics components.  ...  F 3 general status Individual cluster nodes and appliances are monitored through the F 3 general status page shown in Figure 6 .  ... 
doi:10.1088/1742-6596/664/8/082036 fatcat:jb44w5x2hnhzdi74ndzjm5y7o4
« Previous Showing results 1 — 15 out of 44,784 results