16,954 Hits in 5.6 sec

Advances and Challenges for Scalable Provenance in Stream Processing Systems [chapter]

Archan Misra, Marion Blount, Anastasios Kementsietsidis, Daby Sow, Min Wang
2008 Lecture Notes in Computer Science  
While data provenance is a well-studied topic in both database and workflow systems, its support within stream processing systems presents a new set of challenges.  ...  For example, emerging streaming applications in healthcare or finance call for data provenance, as illustrated in the Century stream processing infrastructure that we are building for supporting online  ...  The relatively limited work on scalable provenance for stream-oriented computing systems includes an efficient process provenance solution in [16] , which focuses on identifying and storing dependencies  ... 
doi:10.1007/978-3-540-89965-5_26 fatcat:znratj32dre5rihd753ctmt6ce

From Big Data to Big Data Mining: Challenges, Issues, and Opportunities [chapter]

Dunren Che, Mejdl Safran, Zhiyong Peng
2013 Lecture Notes in Computer Science  
This paper provides an overview of big data mining and discusses the related challenges and the new opportunities.  ...  , accelerate the processing speed (or velocity) and increase system scalability.  ...  of the closed processing architecture of current database systems [4] ; the speed/velocity request of big data (especially stream data) processing asks for commensurate realtime efficiency which again  ... 
doi:10.1007/978-3-642-40270-8_1 fatcat:e535j3rzszeb3lr4v4qi2duwsy

Security and privacy challenges in big data

2016 International Journal of Latest Trends in Engineering and Technology  
Nowadays, the obtainability of Big Data holds much promise to utilize the power of copious data sets and convert that power into transformations and advances in science, medicine, health care, education  ...  While managing large-scale, distributed data sets, the security and privacy policies throws a major challenge in tracking and monitoring data access and use in a dynamic, decentralized environment.  ...  CONCLUSION Through proper analysis of both streaming and static large data sets, we can make better advances in many scientific and medical disciplines and profitability for many enterprises.  ... 
doi:10.21172/1.73.543 fatcat:jzvaxl2conbnzm3ykzuu265rpa

Visualization and Analysis Tools for Ultrascale Climate Data

Dean N. Williams
2014 EOS  
Acknowledgement The developer team consists of Andrew Bauer, Aashish Chaudhary, and Berk Geveci, Kitware, Inc.; Curtis Canada, Phil Jones, and Boonthanome Nouanesengsy, Los Alamos Na-  ...  data provenance processing and capturing.  ...  The second is loosely coupled integration to provide the fl exibility to use tools such as VisIt, Visualization Streams for Ultimate Scalability (ViSUS), R, MATLAB, and ParCat for data analysis and visualization  ... 
doi:10.1002/2014eo420002 fatcat:66l34te73fclrh5vfxbdtcbhse

Recent advances in harvest clarification for antibodies and related products [chapter]

Akshat Gupta, John P. Amara, Elina Gousseinov, Benjamin Cacace
2020 Approaches to the Purification, Analysis and Characterization of Antibody-Based Therapeutics  
However, because of the large volumes and variability of feed streams in clarification and prefiltration processes, it has not been easy to incorporate today's disposable technologies into depth filtration  ...  N A non-destructive depth filter integrity test based on a challenge of aerosolised salt particles was developed for 100 per cent device testing in manufacture.  ... 
doi:10.1016/b978-0-08-103019-6.00006-0 fatcat:lokr4cckyzemldqp5yednmihcq

Enterprise information extraction

Laura Chiticariu, Yunyao Li, Sriram Raghavan, Frederick R. Reiss
2010 Proceedings of the 2010 international conference on Management of data - SIGMOD '10  
Finally, we outline several open challenges and opportunities for the database community to further advance the state of the art in enterprise IE systems.  ...  A SIGMOD 2006 tutorial [3] outlined challenges and opportunities for the database community to advance the state of the art in information extraction, and posed the following grand challenge: "Can we build  ...  Part 2: Scalable Infrastructure Scalability, both in terms of the amount of data and the complexity of the rule set, is an important challenge for building an enterprise information extraction system.  ... 
doi:10.1145/1807167.1807339 dblp:conf/sigmod/ChiticariuLRR10 fatcat:6blwd4zdebdabhuzyhuhendfey

Video streaming over P2P networks: Challenges and opportunities

Naeem Ramzan, Hyunggon Park, Ebroul Izquierdo
2012 Signal processing. Image communication  
In this paper, we describe and discuss existing video streaming systems over P2P.  ...  Finally, the conclusion is drawn with key challenges and open issues related to video streaming over P2P.  ...  While there have been great advances in the research on video streaming over P2P networks, several technical challenges and open issues still remain.  ... 
doi:10.1016/j.image.2012.02.004 fatcat:aelc73c2nvcxreqta4qazc2ixe

Big Data Analytic based on Scalable PANFIS for RFID Localization [article]

Choiru Za'in and Mahardhika Pratama and Andri Ashfahani and Eric Pardede and Huang Sheng
2018 arXiv   pre-print
Scalable PANFIS can learn big data stream by processing many chunks/partitions of data stream. Scalable PANFIS is also equipped with rule structure merging to eliminate the redundancy among rules.  ...  The rule merging process in Scalable PANFIS shows that there is no significant reduction of accuracy in classification task with 96.67 percent of accuracy in comparison with single PANFIS of 98.71 percent  ...  ACKNOWLEDGMENT This project is fully supported by NTU start up grant and MOE tier 1 research grant.  ... 
arXiv:1804.10166v1 fatcat:aixyxyilzfggdhh2xoezbmjd4a

Towards Self-Organizing Service-Oriented Architectures

Walter Binder, Daniele Bonetta, Cesare Pautasso, Achille Peternier, Diego Milano, Heiko Schuldt, Nenad Stojnic, Boi Faltings, Immanuel Trummer
2011 2011 IEEE World Congress on Services  
Service-oriented architectures (SOAs) provide a successful model for structuring complex distributed software systems, as they reduce the cost of ownership and ease the creation of new applications by  ...  In this paper, we promote self-organizing SOA as a new approach to overcome these limitations. Self-organizing SOA integrates research results in the areas of autonomic and serviceoriented computing.  ...  Solving these challenges will significantly advance the state-of-art in SOAs.  ... 
doi:10.1109/services.2011.44 dblp:conf/services/BinderBPPMSSFT11 fatcat:yz67pjpqtbht3gxbigvicszpaq

Big Data Analytics

Yingxu Wang, Jun Peng
2017 International Journal of Cognitive Informatics and Natural Intelligence  
Some methods and technology progress regarding Big Data analytics for disparate data are presented. Challenges of Big Data analytics in dealing with disparate data are also discussed in this paper.  ...  It is a challenge to integrate disparate data from various sources. Big data is often disparate, dynamic, untrustworthy, and inter-related.  ...  W56HZV-08-C-0236, through a subcontract with Mississippi State University (MSU), and was completed for the Simulation Based Reliability and Safety (SimBRS) research program at MSU.  ... 
doi:10.4018/ijcini.2017040103 fatcat:yc2v7otbajhzzpadbnh3fgosda

Big Data Knowledge Discovery Platforms: A 360 Degree Perspective

2019 International Journal of Engineering and Advanced Technology  
These platforms and architecture are giving a cutting edge to the Big Data Knowledge Discovery process by using Artificial Intelligence, Machine Learning and Expert systems.  ...  Big Datais a buzzword affecting nearly every domain and providing different set new opportunity for the development of knowledge discovery process.  ...  Variety and heterogeneity of data poses problem to effectively gather and assimilate data from contrasting distributed systems with proven scalability.  ... 
doi:10.35940/ijeat.b3901.129219 fatcat:2w7a5tkmsfah7oft4jacfbzdau

Addressing Scalability with Message Queues: Architecture and Use Cases for DIRAC Interware [article]

Wojciech Krzemien, Federico Stagni, Christophe Haen, Zoltan Mathe, Andrew McNab, Milosz Zdybal
2019 arXiv   pre-print
The introduction of MQ as an intermediate component in-between the interacting processes allows to decouple the end-points making the system more flexible and providing high scalability and redundancy.  ...  A Message Queue generic interface has been incorporated into the DIRAC framework to help solving the scalability challenges that must be addressed during LHC Run3, starting in 2021.  ...  Introduction We live in a world of large data streams, which are constantly provided by various sources and need to be processed efficiently.  ... 
arXiv:1902.09645v1 fatcat:glmmnbbor5abjlorcscd57uv7m

Introduction to Data Science and Engineering

Elisa Bertino
2016 Data Science and Engineering  
reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.  ...  This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecomm, which permits unrestricted use, distribution, and  ...  Cui, Jiang, Huang, Xu, Gui, and Zhang in "POS: A High-Level System to Simplify RealTime Stream Application Development on Storm" focus on efficient and high-level techniques for processing streaming data  ... 
doi:10.1007/s41019-016-0005-1 fatcat:towbpjcncjbvvfpny66jnptmc4

A Survey on Content Adaptation Systems towards Energy Consumption Awareness

Mohd Norasri Ismail, Rosziati Ibrahim, Mohd Farhan Md Fudzee
2013 Advances in Multimedia  
In addition, we discuss some energy-related challenges content adaptation systems.  ...  In order for the digital contents to fit the target device, content adaptation is required.  ...  Acknowledgments The authors would like to thank the Universiti Tun Hussein Onn Malaysia (UTHM) and the Malaysian Ministry of Education for providing the research grant for facilitating this research activity  ... 
doi:10.1155/2013/871516 fatcat:2worn3mrwzcwfcfey3cn6ettki

DISTIL: Design and implementation of a scalable synchrophasor data processing system

Michael P Andersen, Sam Kumar, Connor Brooks, Alexandra von Meier, David E. Culler
2015 2015 IEEE International Conference on Smart Grid Communications (SmartGridComm)  
Finally, the system is evaluated in a pilot deployment, archiving more than 216 billion raw datapoints and 515 billion derived datapoints from 13 devices in just 3.9TB.  ...  Capable of sustained writes and reads in excess of 16 million points per second per cluster node, advanced query functionality and highly efficient storage, this database enables novel analysis and visualization  ...  This paper describes an agile, scalable stream processing infrastructure for large networks of μPMUs .  ... 
doi:10.1109/smartgridcomm.2015.7436312 dblp:conf/smartgridcomm/AndersenKBMC15 fatcat:uedgkweknnhgpbcbei67k7jbn4
« Previous Showing results 1 — 15 out of 16,954 results