Filters








122,869 Hits in 7.0 sec

An efficient strategy for the collection and storage of large volumes of data for computation

Uthayanath Suthakar, Luca Magnoni, David Ryan Smith, Akram Khan, Julia Andreeva
2016 Journal of Big Data  
Acknowledgements The work by Uthayanath Suthakar was supported by a Brunel University London College of Engineering, Design and Physical Sciences Thomas Gerald Gray postgraduate research scholarship.  ...  There exists a need for an efficient transfer technique that can move large amounts of data quickly and easily without impacting other users or applications [1] .  ...  The core aims of the presented study were the following: • To propose and design efficient approaches for collecting and storing data for analytics that can also be integrated with other data pipelines  ... 
doi:10.1186/s40537-016-0056-1 fatcat:ytwsb4okzbhtxne3lahcvym3ba

An Intelligent Big Data Analytics System using Enhanced Map Reduce Techniques

2019 VOLUME-8 ISSUE-10, AUGUST 2019, REGULAR ISSUE  
An Intelligent Big Data Analytics System using Enhanced Map Reduce Techniques include a set of Methods, applications and strategy which helps the organization and industry to bring together the data and  ...  on the enhanced the Map Reduce techniques, and then to avoid the unnecessary input and output data, optimize the data storage in order to achieve the best out sourcing of data privacy.  ...  The traditional computing techniques that cannot be processed efficiently for example: face book, you tube ,which contains the large data sets on every day which comes under the concept of Big Data .volume  ... 
doi:10.35940/ijitee.b1105.1292s219 fatcat:a2yfmebjofdslgiobgnhxgujey

Efficient Data Management for Putting Forward Data Centric Sciences [chapter]

Genoveva Vargas-Solar
2017 Communications in Computer and Information Science  
The novel and multidisciplinary data centric and scientific movement promises new and not yet imagined applications that rely on massive amounts of evolving data that need to be cleaned, integrated, and  ...  This paper explores the key challenges and opportunities for data management in this new scientific context, and discusses how data management can best contribute to data centric sciences applications  ...  The large spectrum of data persistence and management solutions are adapted for addressing workloads associated with Big Data volumes; and either simple read write operations or with more complex data  ... 
doi:10.1007/978-3-319-67162-8_25 fatcat:5xzhfomxbbdzdmhvniltnndijy

Survey on Big Data Mining Algorithms

Anushree Raj
2019 International Journal for Research in Applied Science and Engineering Technology  
Big Data mining is the capability of extracting useful information from these large datasets or streams of data, that due to its volume, variability, and velocity, it was not possible before to do it.  ...  The Big Data challenge is becoming one of the most exciting opportunities for the next years.  ...  DATA MINING The explosive growth of available data volume is a result of the computerization of our society and the fast development of powerful data collection and storage tools.  ... 
doi:10.22214/ijraset.2019.6234 fatcat:dgcttowerjdkxhu334janvz5z4

EFFICIENT BIG DATA ANALYSIS USING HADOOP FRAMEWORK FOR SENSOR NETWORK DATA

Sahana v
2018 International Journal of Advanced Research in Computer Science  
So in an order to store, load and process the large scale sensory data we propose a pa per that enable the efficient and effective analysis of the Big Data and also handling the large volumes carried out  ...  The major problem will be the amount of data collected, its storage and processing.  ...  By the usage of the hadoop computing tool, the simultaneous or parallel processing of the data and efficient computation is possible.  ... 
doi:10.26483/ijarcs.v9i3.6105 fatcat:xz2znxn6yjbbrpxb2mlz2ey4ge

Toward a General I/O Layer for Parallel-Visualization Applications

W. Kendall, Jian Huang, T. Peterka, R. Latham, R. Ross
2011 IEEE Computer Graphics and Applications  
We would also like to thank Kwan-Liu Ma for providing the jet dataset and the Argonne Leadership Computing Facility for computing resources and support.  ...  This work is funded primarily through the Institute of Ultra-Scale Visualization (http://www.ultravis.org) under the auspices of the SciDAC program within the U.S. Department of Energy.  ...  The partitioning strategy, which is important for scaling computation, conflicts with physical data storage.  ... 
doi:10.1109/mcg.2011.102 pmid:24808253 fatcat:qr4fzdperfebpjainnh23vxinq

Practical Guide to Storage of Large Amounts of Microscopy Data

Andrey Andreev, Daniel E.S. Koo
2020 Microscopy Today  
This leads to limits on the collection of valuable data and slows data analysis and research progress.  ...  Data storage strategy should be carefully considered and different options compared when designing imaging experiments.  ...  Sandra Gesing, and Tom Morrell for valuable discussions.  ... 
doi:10.1017/s1551929520001091 fatcat:n2dgzodyubfztjf4qngktjv2za

Towards Exascale Distributed Data Management

Giovanni Aloisio, Sandro Fiore
2009 The international journal of high performance computing applications  
The large volume of data and the time needed to locate, access, analyze and visualize data will greatly impact on the scientific productivity of scientists and researchers in several domains.  ...  Collections of data will be stored at different sites and made available to the users for further analysis and studies.  ...  Storage and caching: storage technology is a vital part to manage large volume of data. Both hardware and software components will play a critical role in managing big scientific datasets.  ... 
doi:10.1177/1094342009347702 fatcat:kuaplw62indhhleuzwo3chywyu

An Iteration Aware Multidimensional Data Distribution Prototype for Computing Clusters

Baoqiang Yan, Philip Rhodes
2006 2006 IEEE International Conference on Cluster Computing  
This system reduces both disk and network latency by transforming a large number of small requests into a small number of large requests that fill an ndimensional cache block on the cluster head node.  ...  Disk and network latency must be taken into account when applying parallel computing to large multidimensional datasets because they can hinder performance by reducing the rate at which data can be fed  ...  the dataset is too large to fit in the collective memory of the compute nodes.  ... 
doi:10.1109/clustr.2006.311863 dblp:conf/cluster/YanR06 fatcat:q6kcyfusrzczhdcff2fff2oxt4

A Strategy for Improving the Performance of Small Files in Openstack Swift

Xiaoli Zhang, Chengyu Wen, Zizhen Yuan
2018 International Journal of Computer Applications Technology and Research  
In this paper, we propose an aggregated storage strategy (ASS), and implement it in Swift. ASS comprises two parts which include merge storage and index storage.  ...  This is an effective way to improve the storage access performance of small files in Openstack Swift by adding an aggregate storage module.  ...  This work was supported by Science and Technology Department of Sichuan Province, Fund of Science and Technology Planning (No. 2018JY0290).  ... 
doi:10.7753/ijcatr0708.1006 fatcat:t4s3g3ml7jegxd54wfdgnmswxi

Guest Editors' Introduction: Special Issue on Storage for the Big Data Era

Vlado Stankovski, Radu Prodan
2018 Journal of Grid Computing  
TCGs represent the collected data and their dependencies thus providing efficient possibilities for their storage and management.  ...  Privacy-preservation when dealing with large data sets is an important problem tackled by the study "A Privacy-preserving Compression Storage Method for Large Trajectory Data in Road Network" of Peipei  ... 
doi:10.1007/s10723-018-9439-1 fatcat:gqvgzigve5g2hn7pqpfxvhlqsu

PIDX: Efficient Parallel I/O for Multi-resolution Multi-dimensional Scientific Datasets

Sidharth Kumar, Venkatram Vishwanath, Philip Carns, Brian Summa, Giorgio Scorzelli, Valerio Pascucci, Robert Ross, Jacqueline Chen, Hemanth Kolla, Ray Grout
2011 2011 IEEE International Conference on Cluster Computing  
The IDX data format provides efficient, cache oblivious, and progressive access to large-scale scientific datasets by storing the data in a hierarchical Z (HZ) order.  ...  We then present a data model description and a novel aggregation strategy to enhance the scalability of the PIDX library.  ...  and for efficient access to the underlying data files that make up an IDX dataset.  ... 
doi:10.1109/cluster.2011.19 dblp:conf/cluster/KumarVCSSPRCKG11 fatcat:acjkja3mxzeqljwjd4uir5tskq

Big Data in Capital Markets

Manpreet Singh
2014 International Journal of Computer Applications  
These advancements are as diverse in the areas like collection, storage, aggregation, processing and analysis of financial data.  ...  Moreover, with the advent of the Internet of things," there is an exponential rise in huge volumes of unstructured data from several different sources.  ...  Compute Grids-Compute Grids offer a way of parallelizing processes across multiple servers for handling capacity, efficiency and failure issues.  ... 
doi:10.5120/18751-0008 fatcat:velxslknjzde7i4urlkmslhxby

Cloud big data application for transport

Gavin Kemp, Genoveva Vargas Solar, Catarina Ferreira Da Silva, Parisa Ghodous, Christine Collet, Pedro Pablo Lopez Amalya
2016 International Journal of Agile Systems and Management  
Big data analytics brings new insights and useful correlations of large data collections providing undiscovered knowledge.  ...  A cloud-oriented architecture opens new perspectives for providing efficient and personalised big data management and analytics services to (small) companies.  ...  Acknowledgements The authors would like to thank the region Rhône-Alpes who finances the thesis work of Gavin Kemp by means of the ARC seven programs (http://www.arc7-territoiresmobilites.rhonealpes.fr  ... 
doi:10.1504/ijasm.2016.079940 fatcat:zvabrx65znc6hlrtl5d6vrcgda

NOSQL: A Very Dynamic Approach for Managing Big Data

Assira Amin, Mudasir Ahmed Muttoo, Kirti Bhatia
2018 International Journal of Trend in Scientific Research and Development  
The Big Data is a global system and is considered a data collection that increases so large and can't be managed efficiently by traditional databases management systems.  ...  The present digital world becomes more complex with Big Data means the volume, velocity and variety of data.  ...  The NOSQL is a new open source, distributed data storage concept that is very efficient in terms of handling the huge volume of data.  ... 
doi:10.31142/ijtsrd12962 fatcat:j5zlbdkowffq7b4baysnjopqxa
« Previous Showing results 1 — 15 out of 122,869 results