Filters








4,641 Hits in 4.0 sec

High-performance, massively scalable distributed systems using the MapReduce software framework

Kurt Rohloff, Richard E. Schantz
2010 Programming Support Innovations for Emerging Distributed Applications on - PSI EtA '10  
We provide experimental results generated of an early version of SHARD.  ...  We provide a general approach to construct an information system from the MapReduce software framework that responds to data queries.  ...  EXPERIMENTATION To test the performance of our general design of a scalable information management system based on the MapReduce framework, we developed an early version of SHARD using the Cloudera version  ... 
doi:10.1145/1940747.1940751 dblp:conf/oopsla/RohloffS10 fatcat:p477du7uofavnf6srzd4kwr5pi

Implementation of a Parallel Algorithm Based on a Spark Cloud Computing Platform

Longhui Wang, Yong Wang, Yudong Xie
2015 Algorithms  
The experimental results show that Spark has a very great accelerating effect on the ant colony algorithm when the city scale of TSP or the number of ants is relatively large.  ...  We combine MMAS with Spark MapReduce to execute the path building and the pheromone operation in a distributed computer cluster.  ...  Therefore, it is very important to choose an optimization algorithm which is suitable for the MapReduce framework.  ... 
doi:10.3390/a8030407 fatcat:ravrubj3bzefflbxmvse4eovky

I2mapreduce: Fine-Grain Incremental Processing In Big Data Mining

Miss Mugdha A. Kulkarni, Prof I.R. Shaikh
2016 International Journal Of Engineering And Computer Science  
Experimental results on Amazon EC2 show significant performance improvements of I 2 MapReduce compared to both plain and iterative MapReduce performing re-computation.  ...  I 2 MAPREDUCE: Fine-Grain Incremental Processing in big data mining a novel incremental processing extension to Map Reduce, the most widely used framework for mining big data.  ...  In this paper, i present the programming model and the architecture of Twister an enhanced MapReduce runtime that supports iterative MapReduce computations efficiently.  ... 
doi:10.18535/ijecs/v5i1.3 fatcat:qrtlpc3uvrg3rkovn7efvyzovi

Scalability of Parallel Scientific Applications on the Cloud

Satish Narayana Srirama, Oleg Batrashev, Pelle Jakovits, Eero Vainikko
2011 Scientific Programming  
DOUG is an open source software package for parallel iterative solution of very large sparse systems of linear equations.  ...  We could also observe the limitations of the cloud and its comparison with cluster in terms of performance.  ...  Acknowledgements The research is supported by the European Social Fund through Mobilitas program, the European Regional Development Fund through the Estonian Centre of Excellence in Computer Science and  ... 
doi:10.1155/2011/361854 fatcat:vazhabbturd7dmkk5x45inszbm

Enabling Computational Steering with an Asynchronous-Iterative Computation Framework

Alexandre di Costanzo, Chao Jin, Carlos A. Varela, Rajkumar Buyya
2009 2009 Fifth IEEE International Conference on e-Science  
The framework supports steerable applications by introducing an asynchronous iterative MapReduce programming model that is deployed using Hadoop over a set of virtual machines executing on a multi-cluster  ...  We illustrate and evaluate this framework with a scientific application that aims to fit models of the Milky Way galaxy structure to stars observed by the Sloan Digital Sky Survey.  ...  Acknowledgements This work is partially supported by research grants from the Australian Research Council (ARC) and Australian Department of Innovation, Industry, Science and Research (DIISR).  ... 
doi:10.1109/e-science.2009.43 dblp:conf/eScience/CostanzoJVB09 fatcat:vtp4777lofhtfapgu5jfvpocve

IMPLEMENTATION OF P-PIC ALGORITHM IN MAP REDUCE TO HANDLE BIG DATA

Jayalatchumy D .
2014 International Journal of Research in Engineering and Technology  
The experimental results show that p-PIC can perform well in MR framework for handling big data. It is very fast and scalable.  ...  Google's Mapreduce has attracted a lot of attention for such applications that motivate us to convert sequential algorithm to Mapreduce algorithm.  ...  The graph gives a comparison of PIC, p-PIC and p-PIC in Mapreduce Framework.  ... 
doi:10.15623/ijret.2014.0319022 fatcat:yr7632z6yjhhlmueoob732frzi

Performance Comparison Between Hama and Hadoop

Shuo Li, Baomin Xu
2015 International Journal of Database Theory and Application  
The experimental results show that Hama can achieve much higher performance than Hadoop in our testbed.  ...  We implement Monte Carlo algorithm of Pi in Hama and Hadoop under the same software and hardware environment.  ...  Acknowledgments This work is supported by the National Natural Science Foundation of China (NSFC 61370060).  ... 
doi:10.14257/ijdta.2015.8.3.08 fatcat:xr2yo7uehzhb7kcwde2ixodqdi

D2D Big Data Privacy-Preserving Framework Based on (a, k)-Anonymity Model

Jie Wang, Hongtao Li, Feng Guo, Wenyin Zhang, Yifeng Cui
2019 Mathematical Problems in Engineering  
In this paper, we propose an (a, k)-anonymity privacy-preserving framework for D2D big data deployed on MapReduce.  ...  Firstly, we provide a framework for the D2D big data sharing and analyze the threat model. Then, we propose an (a, k)-anonymity privacy-preserving framework for D2D big data deployed on MapReduce.  ...  MapReduce-Based (a, k)-Anonymity Framework MapReduce-Based (a, k)-Anonymity Algorithm.  ... 
doi:10.1155/2019/2076542 fatcat:sdprvv524rephafzlqjxn4kfi4

An Empirical Comparison of Big Graph Frameworks in the Context of Network Analysis [article]

Jannis Koch, Christian L. Staudt, Maximilian Vogel, Henning Meyerhenke
2016 arXiv   pre-print
The study provides experimental evidence for selecting the appropriate framework depending on the task and data volume.  ...  Out of the distributed frameworks, GraphLab and Apache Giraph generally show the best performance.  ...  Parts of this paper have been published in preliminary form as [Koch et al., 2015] .  ... 
arXiv:1601.00289v1 fatcat:zr3icqvs2fa4vnmjgh2il25e3e

Adapting Skyline Computation to the MapReduce Framework: Algorithms and Experiments [chapter]

Boliang Zhang, Shuigeng Zhou, Jihong Guan
2011 Lecture Notes in Computer Science  
This paper addresses the problem of skyline computation under the MapReduce framework.  ...  As a parallel programming model for data-intensive computing applications, MapReduce runs on a cluster of commercial PCs with the main idea of task decomposition and result reduction.  ...  MR-BNL BNL [1] is an iterative algorithm that repeatedly reads a set of data records for processing.  ... 
doi:10.1007/978-3-642-20244-5_39 fatcat:jktptplihbgcrgjjydux2obp2m

Analysis and Research of K-means Algorithm in Soil Fertility Based on Hadoop Platform [chapter]

Guifen Chen, Yuqin Yang, Hongliang Guo, Xionghui Sun, Hang Chen, Lixia Cai
2015 IFIP Advances in Information and Communication Technology  
The above analysis shows that, K-means algorithm is an effective soil fertility evaluation method;Based on Hadoop platform of parallel K-means algorithm has great realistic meaning to analysis of large  ...  amount of data of soil fertility factors.  ...  Acknowledgements This work was supported by the national " 863 " project (2006AA10A309), National Spark Plan (2008GA661003) and Shi Hang of Jilin province projects (2011-Z20).  ... 
doi:10.1007/978-3-319-19620-6_35 fatcat:xxvtdhebtzh5vaqqbtiktbrl54

Advantages of Giraph over Hadoop in Graph Processing

C. L. Vidal-Silva, E. Madariaga, T. Pham, J. M. Rubio, L. A. Urzua, L. Carter, F. Johnson
2019 Engineering, Technology & Applied Science Research  
This article presents a comparison of the computing performance of the MapReduce tool Hadoop and Giraph on large-scale graphs.  ...  Experimental results show that the use of Giraph for processing large-size graphs reduces the execution time by 25% in comparison with the results obtained using the Hadoop for the same experiments.  ...  Hadoop , an open-source framework based on MapReduce [5, 6] appears as a solution for the mentioned issues.  ... 
doi:10.48084/etasr.2715 fatcat:qlmiru7bunafrnx7he7lh4o32e

YAFIM: A Parallel Frequent Itemset Mining Algorithm with Spark

Hongjian Qiu, Rong Gu, Chunfeng Yuan, Yihua Huang
2014 2014 IEEE International Parallel & Distributed Processing Symposium Workshops  
Experimental results show that, compared with the algorithms implemented with MapReduce, YAFIM achieved u 18 speedup in average for various benchmarks.  ...  However, the existing parallel Apriori algorithms implemented with the MapReduce model are not efficient enough for iterative computation.  ...  Fig. 6 shows the comparison between YAFIM and MRApriori. The x-axis is the pass of the iterations.  ... 
doi:10.1109/ipdpsw.2014.185 dblp:conf/ipps/QiuGYH14 fatcat:qfe2tkx62nayvgh764b3j5x2vi

Performance Analysis of Apriori Algorithm with Different Data Structures on Hadoop Cluster

Sudhakar Singh, Rakhi Garg, P.K. Mishra
2015 International Journal of Computer Applications  
In this paper, we implement three variations of Apriori algorithm using data structures hash tree, trie and hash table trie i.e. trie with hash technique on MapReduce paradigm.  ...  MapReduce is the emerging parallel and distributed technology to process big datasets on Hadoop Cluster.  ...  A driver class is defined with a Mapper class, a Reducer and an optional Combiner class of MapReduce framework.  ... 
doi:10.5120/ijca2015906632 fatcat:bl34g7nlrzfv5gnqktr5cjgvuq

Design and Implementation of Parallelized LDA Topic Model Based on MapReduce

Duan-wu YAN, Tie-jun LI, Xiong-fei YANG, Kun CHEN
2018 DEStech Transactions on Computer Science and Engineering  
on MapReduce framework is feasible.Compared with non-parallel LDA model, the parallel LDA topic model process can obviously improve the analysis efficiency forlarge-scale text datasets.  ...  In order to solve theefficiency bottlenecksof non-parallel LDA topic model while processing large-scale text datasets,a parallel LDA topic model computing framework based on MapReduce is designed and implemented  ...  Acknowledgement This research was financially supported by the Social Science Foundation of Jiangsu Province of China (Project no: 17TQB009).  ... 
doi:10.12783/dtcse/ccnt2018/24712 fatcat:lr5u3o3m55dwdlaoz3f3tdkyr4
« Previous Showing results 1 — 15 out of 4,641 results