Filters








2,753 Hits in 4.7 sec

A Micro-benchmark Suite for Evaluating Hadoop MapReduce on High-Performance Networks [chapter]

Dipti Shankar, Xiaoyi Lu, Md. Wasi-ur-Rahman, Nusrat Islam, Dhabaleswar K. (DK) Panda
<span title="">2014</span> <i title="Springer International Publishing"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/2w3awgokqne6te4nvlofavy5a4" style="color: black;">Lecture Notes in Computer Science</a> </i> &nbsp;
to study the impact of network configuration on the communication patterns of the MapReduce job.  ...  However, a standardized benchmark suite that focuses on helping users evaluate the performance of the stand-alone Hadoop MapReduce component is not available in the current Apache Hadoop community.  ...  For instance, increasing the number of map and reduce tasks improved the performance of the MapReduce job by about 32 % for IPoIB (32 Gbps), while it improved by only 24 % for 10 GigE, for a shuffle data  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1007/978-3-319-13021-7_2">doi:10.1007/978-3-319-13021-7_2</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/aubbemw6knffdg5o5nqwjdpsrq">fatcat:aubbemw6knffdg5o5nqwjdpsrq</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20170814140149/http://www.springer.com/cda/content/document/cda_downloaddocument/9783319130200-c2.pdf?SGWID=0-0-45-1486986-p177071736" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/7d/e1/7de1813fb84453d80de6fb21795dc5bc5f12a8ba.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1007/978-3-319-13021-7_2"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> springer.com </button> </a>

Optimization of Computing and Networking Resources of a Hadoop Cluster Based on Software Defined Network

Ali Khaleel, Hamed Al-Raweshidy
<span title="">2018</span> <i title="Institute of Electrical and Electronics Engineers (IEEE)"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/q7qi7j4ckfac7ehf3mjbso4hne" style="color: black;">IEEE Access</a> </i> &nbsp;
Whilst for the Tera Sort application, the performance of Hadoop MapReduce is improved by 73.39% and 55.93%, compared with the default and Gunther work, respectively.  ...  Moreover, the experimental results in an SDN environment showed the performance of a Hadoop MapReduce job is further improved due to the advantages of the intelligent and centralised management achieved  ...  ACKNOWLEDGMENT The authors would like to thank the Iraqi Ministry of Higher Education and Scientific Research as well as University of Diyala to financially sponsor the current research and study.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/access.2018.2876385">doi:10.1109/access.2018.2876385</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/w2kefnhqrjbhzj6wh2djheqlee">fatcat:w2kefnhqrjbhzj6wh2djheqlee</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20190428173842/https://bura.brunel.ac.uk/bitstream/2438/17072/1/Fulltext.PDF" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/63/04/6304477c6b2068138e8964ac508f41dedc4c0ff8.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/access.2018.2876385"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="unlock alternate icon" style="background-color: #fb971f;"></i> ieee.com </button> </a>

MC2: Map Concurrency Characterization for MapReduce on the Cloud

Mohammad Hammoud, Majd F. Sakr
<span title="">2013</span> <i title="IEEE"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/ftej443yjfel7jenws7pewa4ju" style="color: black;">2013 IEEE Sixth International Conference on Cloud Computing</a> </i> &nbsp;
MapReduce is now a pervasive analytics engine on the cloud. Hadoop is an open source implementation of MapReduce and is currently enjoying wide popularity.  ...  By leveraging the generated predicted information, M C 2 can judiciously guide Map phase configuration and, consequently, improve Hadoop performance.  ...  Indeed, the located sweet spot under the 40 map slot configuration offered Hadoop a performance improvement of 13.1% versus default Hadoop.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/cloud.2013.93">doi:10.1109/cloud.2013.93</a> <a target="_blank" rel="external noopener" href="https://dblp.org/rec/conf/IEEEcloud/HammoudS13.html">dblp:conf/IEEEcloud/HammoudS13</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/wrkzvkx7jvarxls6qhr7yyjh2i">fatcat:wrkzvkx7jvarxls6qhr7yyjh2i</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20131127034627/http://www.qatar.cmu.edu:80/~mhhammou/CLOUD2013-MHH.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/43/d0/43d0de73d84ad3a44145540df2741c07000e2ac2.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/cloud.2013.93"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> ieee.com </button> </a>

Improving MapReduce Performance in a Heterogeneous Cloud: A Measurement Study

Xu Zhao, Ling Liu, Qi Zhang, Xiaoshe Dong
<span title="">2014</span> <i title="IEEE"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/ftej443yjfel7jenws7pewa4ju" style="color: black;">2014 IEEE 7th International Conference on Cloud Computing</a> </i> &nbsp;
Although a fair of research efforts have dedicated to improve MapReduce performance, there still lacks of in-depth understanding of the key factors that affect the performance of MapReduce jobs in heterogeneous  ...  We conclude with five key findings: (1) Early shuffle, though effective for reducing the latency of MapReduce jobs, can impact the performance of map tasks and reduce tasks differently when running on  ...  The first author is also supported by the National Natural Science Foundation of China  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/cloud.2014.61">doi:10.1109/cloud.2014.61</a> <a target="_blank" rel="external noopener" href="https://dblp.org/rec/conf/IEEEcloud/ZhaoLZD14.html">dblp:conf/IEEEcloud/ZhaoLZD14</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/izgexesykndj3admfrpsqqz3a4">fatcat:izgexesykndj3admfrpsqqz3a4</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20161228100250/http://www.istc-cc.cmu.edu/publications/papers/2014/XuZhao-CLOUD2014.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/f8/bc/f8bc50f9575b78af777dc5585168937fe4c5c036.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/cloud.2014.61"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> ieee.com </button> </a>

An Analytical Approach to Evaluation of SSD Effects under MapReduce Workloads

Sungyong Ahn, Sangkyu Park
<span title="2015-10-30">2015</span> <i title="The Institute of Electronics Engineers of Korea"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/pdhhinwb6zdunc5juvdlqkgbdq" style="color: black;">JSTS Journal of Semiconductor Technology and Science</a> </i> &nbsp;
The developed model can precisely predict the execution time of MapReduce jobs with less than 7% difference for most cases.  ...  For this, we propose a MapReduce performance model using queuing network to simulate the execution time of MapReduce job with varying cluster size.  ...  The performance of Hadoop cluster can be improved not only by using SSDs in place of HDDs but also by increasing the number of nodes.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.5573/jsts.2015.15.5.511">doi:10.5573/jsts.2015.15.5.511</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/bno6s6wz2rc3hjntu6sgxlpbk4">fatcat:bno6s6wz2rc3hjntu6sgxlpbk4</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20180720034008/http://ocean.kisti.re.kr/downfile/volume/ieek/E1STAN/2015/v15n5/E1STAN_2015_v15n5_511.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/dd/88/dd8864860facfd59f1bbc6087b0c7cdaec54b7d5.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.5573/jsts.2015.15.5.511"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> Publisher / doi.org </button> </a>

Cross-Phase Optimization in MapReduce [chapter]

Benjamin Heintz, Abhishek Chandra, Jon Weissman
<span title="">2014</span> <i title="Springer New York"> Cloud Computing for Data-Intensive Applications </i> &nbsp;
Using Hadoop, we show that the absence of network and node homogeneity and locality of data lead to poor performance.  ...  The experimental results show the potential of these techniques as performance is improved from 7%-18% depending on the execution environment and application.  ...  ACKNOWLEDGMENT The authors would like to acknowledge NSF Grant IIS-0916425 and NSF Grant CNS-0643505, which supported this research.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1007/978-1-4939-1905-5_12">doi:10.1007/978-1-4939-1905-5_12</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/izaj33b62zhblnxh2vmtno5ivq">fatcat:izaj33b62zhblnxh2vmtno5ivq</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20170922014112/http://www-users.cs.umn.edu/~jon/papers/ben.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/83/d5/83d5ad6aac359c79fdb5ac598cd648add1b47178.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1007/978-1-4939-1905-5_12"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> springer.com </button> </a>

Cross-Phase Optimization in MapReduce

B. Heintz, Chenyu Wang, A. Chandra, J. Weissman
<span title="">2013</span> <i title="IEEE"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/hyryzrmrtje2ljt6ettytqn32y" style="color: black;">2013 IEEE International Conference on Cloud Engineering (IC2E)</a> </i> &nbsp;
Using Hadoop, we show that the absence of network and node homogeneity and locality of data lead to poor performance.  ...  The experimental results show the potential of these techniques as performance is improved from 7%-18% depending on the execution environment and application.  ...  ACKNOWLEDGMENT The authors would like to acknowledge NSF Grant IIS-0916425 and NSF Grant CNS-0643505, which supported this research.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/ic2e.2013.26">doi:10.1109/ic2e.2013.26</a> <a target="_blank" rel="external noopener" href="https://dblp.org/rec/conf/ic2e/HeintzWCW13.html">dblp:conf/ic2e/HeintzWCW13</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/hucgibm4ezb3pd3ek3f3e3aw4y">fatcat:hucgibm4ezb3pd3ek3f3e3aw4y</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20170922014112/http://www-users.cs.umn.edu/~jon/papers/ben.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/83/d5/83d5ad6aac359c79fdb5ac598cd648add1b47178.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/ic2e.2013.26"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> ieee.com </button> </a>

A comprehensive performance analysis of Apache Hadoop and Apache Spark for large scale data sets using HiBench

N. Ahmed, Andre L. C. Barczak, Teo Susnjak, Mohammed A. Rashid
<span title="">2020</span> <i title="Springer Science and Business Media LLC"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/pkhnkszyprhb3orbf6g7tqmgiu" style="color: black;">Journal of Big Data</a> </i> &nbsp;
The advent of distributed computing frameworks such as Hadoop and Spark offers efficient solutions to analyze vast amounts of data.  ...  In this regard, this study investigates the most impacting parameters, under resource utilization, input splits, and shuffle, to compare the performance between Hadoop and Spark, using an implemented cluster  ...  TS and MAR helped to improve the final paper. All authors read and approved the final manuscript. Funding This work was not funded.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1186/s40537-020-00388-5">doi:10.1186/s40537-020-00388-5</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/tm5lu63e7jehzlhknlv3q4wdsa">fatcat:tm5lu63e7jehzlhknlv3q4wdsa</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20210428073746/https://mro.massey.ac.nz/bitstream/handle/10179/16008/A%20comprehensive%20performance%20analysis%20of%20Apache%20Hadoop.pdf;jsessionid=F22BE6B3790166936EFAB9D9F4214A82?sequence=1" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/34/a5/34a5e8756efac71270f702ddbddc6fbbd8811b30.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1186/s40537-020-00388-5"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="unlock alternate icon" style="background-color: #fb971f;"></i> springer.com </button> </a>

Virtual Shuffling for Efficient Data Movement in MapReduce

Weikuan Yu, Yandong Wang, Xinyu Que, Cong Xu
<span title="">2015</span> <i title="Institute of Electrical and Electronics Engineers (IEEE)"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/5jlmyrayyrdazh5awdlsoec77q" style="color: black;">IEEE transactions on computers</a> </i> &nbsp;
Index Terms-Hadoop, MapReduce, virtual shuffling, near-demand merging • The authors are with the  ...  Between these phases, MapReduce requires a shuffling phase to globally exchange the intermediate data generated by the mapping phase.  ...  data in large-scale Hadoop cluster [7] , [8] , few studies have addressed the need of efficient I/O during data shuffling in the Hadoop MapReduce framework.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/tc.2013.216">doi:10.1109/tc.2013.216</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/khh6g7wfwzfjjoxowelxfkljou">fatcat:khh6g7wfwzfjjoxowelxfkljou</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20170722010744/http://www.cs.fsu.edu:80/~yuw/pubs/2015-TC-Yu-virtual-shuffling.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/12/09/12094c89613fb9650e80a933bf0b33be76451948.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/tc.2013.216"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> ieee.com </button> </a>

HPMR: Prefetching and pre-shuffling in shared MapReduce computation environment

Sangwon Seo, Ingook Jang, Kyungchang Woo, Inkyo Kim, Jin-Soo Kim, Seungryoul Maeng
<span title="">2009</span> <i title="IEEE"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/2ade5dt4mjgqzod5n6qf33litm" style="color: black;">2009 IEEE International Conference on Cluster Computing and Workshops</a> </i> &nbsp;
Hadoop is an open-source implementation of the MapReduce programming model. Hadoop is used by many companies including Yahoo!  ...  In this paper, we propose two optimization schemes, prefetching and pre-shuffling, which improve the overall performance under the shared environment while retaining compatibility with the native Hadoop  ...  Thus, it is essential to reduce the shuffling overhead to improve the overall performance of the MapReduce computation.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/clustr.2009.5289171">doi:10.1109/clustr.2009.5289171</a> <a target="_blank" rel="external noopener" href="https://dblp.org/rec/conf/cluster/SeoJWKKM09.html">dblp:conf/cluster/SeoJWKKM09</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/n5dcsszpxrdunfr7eczn25j4my">fatcat:n5dcsszpxrdunfr7eczn25j4my</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20110331175252/http://csl.skku.edu/papers/cluster09.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/be/b2/beb24e1c823d8833a04dba1b8407999127be19a4.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/clustr.2009.5289171"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> ieee.com </button> </a>

Locality-Aware Reduce Task Scheduling for MapReduce

Mohammad Hammoud, Majd F. Sakr
<span title="">2011</span> <i title="IEEE"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/vf7glygozrdhdkhluhlxpb7j4q" style="color: black;">2011 IEEE Third International Conference on Cloud Computing Technology and Science</a> </i> &nbsp;
Hadoop, an open source implementation of MapReduce, schedules map tasks in the vicinity of their inputs in order to diminish network traffic and improve performance.  ...  We implemented LARTS in Hadoop-0.20.2. Evaluation results show that LARTS outperforms the native Hadoop reduce task scheduler by an average of 7%, and up to 11.6%.  ...  We observe that the process of interleaving the execution of map tasks with the shuffling of partitions employed by native Hadoop improves performance but increases network traffic.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/cloudcom.2011.87">doi:10.1109/cloudcom.2011.87</a> <a target="_blank" rel="external noopener" href="https://dblp.org/rec/conf/cloudcom/HammoudS11.html">dblp:conf/cloudcom/HammoudS11</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/iyrlovqosnfoldayb4lq7qoh4q">fatcat:iyrlovqosnfoldayb4lq7qoh4q</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20140712000956/http://www.qatar.cmu.edu:80/~mhhammou/mhh_msakr_LARTS_CloudCom2011_paper.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/72/07/7207036cfb1e5e0aa74756e395dfc9dd94e46af0.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/cloudcom.2011.87"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> ieee.com </button> </a>

CooMR

Xiaobing Li, Yandong Wang, Yizheng Jiao, Cong Xu, Weikuan Yu
<span title="">2013</span> <i title="ACM Press"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/zigbcra6rjdivda6lkzknwuo5q" style="color: black;">Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis on - SC &#39;13</a> </i> &nbsp;
Our evaluation demonstrates that CooMR is able to increase task coordination, improve system resource utilization, and significantly speed up the execution time of MapReduce programs.  ...  Hadoop is a widely adopted open source implementation of MapReduce programming model for big data processing.  ...  Acknowledgments Xiaobing Li and Yandong Wang have contributed equally to the paper. This work is funded in part by National Science Foundation awards 1059376 and 1320016.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1145/2503210.2503276">doi:10.1145/2503210.2503276</a> <a target="_blank" rel="external noopener" href="https://dblp.org/rec/conf/sc/LiWJXY13.html">dblp:conf/sc/LiWJXY13</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/ngietg45fnc2jbehq67od2463m">fatcat:ngietg45fnc2jbehq67od2463m</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20170830064830/http://www.chinacloud.cn/upload/2013-10/13102808076501.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/d6/29/d629fbd25903cfd6f0e7ddf6b2858877c87195f0.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1145/2503210.2503276"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> acm.org </button> </a>

iShuffle: Improving Hadoop Performance with Shuffle-on-Write

Yanfei Guo, Jia Rao, Dazhao Cheng, Xiaobo Zhou
<span title="2017-06-01">2017</span> <i title="Institute of Electrical and Electronics Engineers (IEEE)"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/ll6bfs5o6bahfinh3u5z2cnyoy" style="color: black;">IEEE Transactions on Parallel and Distributed Systems</a> </i> &nbsp;
Hadoop is a popular implementation of the MapReduce framework for running data-intensive jobs on clusters of commodity servers.  ...  We attribute the delay in job completion to the coupling of the shuffle phase and reduce tasks, which leaves the potential parallelism between multiple waves of map and reduce unexploited, fails to address  ...  Some recent work focused on the improvement of shuffle and reduce. MapReduce Online [6] proposed a push-based shuffle mechanism to support the online aggregation and continuous queries.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/tpds.2016.2587645">doi:10.1109/tpds.2016.2587645</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/ljymy24stjd5pczva4akot2qby">fatcat:ljymy24stjd5pczva4akot2qby</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20200320173815/http://0b4af6cdc2f0c5998459-c0245c5c937c5dedcca3f1764ecc9b2f.r43.cf2.rackcdn.com/11733-icac13_guo.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/d9/80/d9807552cd99b6cfd83901febaa8007763f21844.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/tpds.2016.2587645"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> ieee.com </button> </a>

Using Performance Measurements to Improve MapReduce Algorithms

Todd D. Plantenga, Yung Ryn Choe, Ann Yoshimura
<span title="">2012</span> <i title="Elsevier BV"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/cx3f4s3qmfe6bg4qvuy2cxezyu" style="color: black;">Procedia Computer Science</a> </i> &nbsp;
The Hadoop MapReduce software environment is used for parallel processing of distributively stored data.  ...  Although MapReduce simplifies the job of writing parallel algorithms and Hadoop provides a platform-neutral Java implementation, the construction of an algorithm that performs well at scale is still a  ...  The goal is to provide insight for improving the performance of complex MapReduce algorithms.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1016/j.procs.2012.04.210">doi:10.1016/j.procs.2012.04.210</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/xncit5lhk5a67exd3trrkpbcje">fatcat:xncit5lhk5a67exd3trrkpbcje</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20190418130604/https://core.ac.uk/download/pdf/82147671.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/f1/55/f155e5388dc5dbe785ac5b75055f112aad894230.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1016/j.procs.2012.04.210"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="unlock alternate icon" style="background-color: #fb971f;"></i> elsevier.com </button> </a>

Survey of MapReduce frame operation in bioinformatics

Q. Zou, X.-B. Li, W.-R. Jiang, Z.-Y. Lin, G.-L. Li, K. Chen
<span title="2013-02-07">2013</span> <i title="Oxford University Press (OUP)"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/op7ztx4fhvairowgqifu7dnvsi" style="color: black;">Briefings in Bioinformatics</a> </i> &nbsp;
The open source Apache Hadoop project, which adopts the MapReduce framework and a distributed file system, has recently given bioinformatics researchers an opportunity to achieve scalable, efficient and  ...  In this article, we present MapReduce frame-based applications that can be employed in the next-generation sequencing and other biological domains.  ...  We introduce the flow of shuffle in MapReduce, which can help the bioinformatics researchers to understand the mechanism of MapReduce.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1093/bib/bbs088">doi:10.1093/bib/bbs088</a> <a target="_blank" rel="external noopener" href="https://www.ncbi.nlm.nih.gov/pubmed/23396756">pmid:23396756</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/sro4pk6aobcotoeozbbtpo6x5q">fatcat:sro4pk6aobcotoeozbbtpo6x5q</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20180729035043/https://watermark.silverchair.com/bbs088.pdf?token=AQECAHi208BE49Ooan9kkhW_Ercy7Dm3ZL_9Cf3qfKAc485ysgAAAZwwggGYBgkqhkiG9w0BBwagggGJMIIBhQIBADCCAX4GCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQMnSmiWZijz2ZW5ZpRAgEQgIIBT4170qRIexst4HtkV7jRh_jN-v_02NRpt6NVaTIqAF4pJXtuYZqBzhGWW7wv3XLIxpoNYEzMpIIq5WvYRvJUOdh9KZPvYsZT3s-Z4rzbTbaPVgDJQSsZi9wH1Qoeh-8Ol7lha6FulloxKPs4ZuaubNqFB3-ehdxFtkw3w1P5HRol3YrznVbO7WJNwca3l9iaDJL2yy4QNDF_KVzx9AMZHC8chOiptYxG0TLGtM9AL41lgfreTf1KS23cthEVX7I1I9G8s_NYYHE0VkRU7t-nUKomYbXMI89MH_6GB9Npk-w_wFZvtUlbQcmnkX5Qb_r1mYabJXFMek0TCQp-J5rnkeEjCXmfH-67KRM_1IbTfg7i0_5SarRSEIQaxsr31hrcS4ZWskmOkR5VC3DgCPz6pIzFZLNLbGgySZbQY8zdBV65D9V5SNpLbbK6V3ishL6h" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/eb/85/eb854571196bddb05b358c0983c2e1b5fb55426d.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1093/bib/bbs088"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> oup.com </button> </a>
&laquo; Previous Showing results 1 &mdash; 15 out of 2,753 results