Filters








622 Hits in 6.9 sec

Big Data at HPC Wales [article]

Sidharth N. Kashyap, Ade J. Fewings, Jay Davies, Ian Morris, Andrew Thomas Thomas Green, Martyn F. Guest
<span title="2015-06-30">2015</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
We describe a solution that dynamically creates a unified cluster based on YARN in an HPC Environment, without the need to configure and allocate a dedicated Hadoop cluster.  ...  This coupling of environments creates a platform for applications to utilize the native HPC solutions along with the Big Data Frameworks.  ...  We use Lustre version 2.1.3 on Sandy Bridge machines with dual processor EP nodes (16 Cores), 414G of local storage and 64G memory per node.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1506.08907v1">arXiv:1506.08907v1</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/d3sna2txtbcnlnyktedkknpyxe">fatcat:d3sna2txtbcnlnyktedkknpyxe</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20200929201902/https://arxiv.org/ftp/arxiv/papers/1506/1506.08907.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/a4/2f/a42f55021c0c9d19bb88377966b34d95ec6b52d3.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1506.08907v1" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>

Computational storage: an efficient and scalable platform for big data and HPC applications

Mahdi Torabzadehkashi, Siavash Rezaei, Ali HeydariGorji, Hosein Bobarshad, Vladimir Alves, Nader Bagherzadeh
<span title="2019-11-15">2019</span> <i title="Springer Science and Business Media LLC"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/pkhnkszyprhb3orbf6g7tqmgiu" style="color: black;">Journal of Big Data</a> </i> &nbsp;
MapReduce and HPC applications in-place without any modifications on the underlying distributed processing framework.  ...  For the proof of concept, we build a fully functional Catalina prototype and a CSD-equipped platform using 16 Catalina CSDs to run Intel HiBench Hadoop and HPC benchmarks to investigate the benefits of  ...  Acknowledgements Not applicable.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1186/s40537-019-0265-5">doi:10.1186/s40537-019-0265-5</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/wnhnkgsvwbcp7ebcohrjyysbe4">fatcat:wnhnkgsvwbcp7ebcohrjyysbe4</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20191217092640/https://journalofbigdata.springeropen.com/track/pdf/10.1186/s40537-019-0265-5" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/31/7c/317c62040880e4e0ee0c6d03e698d8d1cb0c5e9c.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1186/s40537-019-0265-5"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="unlock alternate icon" style="background-color: #fb971f;"></i> springer.com </button> </a>

A quantitative assessment of the Hadoop framework for analyzing massively parallel DNA sequencing data

Alexey Siretskiy, Tore Sundqvist, Mikhail Voznesenskiy, Ola Spjuth
<span title="2015-06-04">2015</span> <i title="Oxford University Press (OUP)"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/apde6vbe7vfoxkfibp7n4mzonm" style="color: black;">GigaScience</a> </i> &nbsp;
Recently, the Hadoop platform has emerged as an interesting option to address the challenges of increasingly large datasets with distributed storage, distributed processing, built-in data locality, fault  ...  The most common e-infrastructure for analyzing this data consists of batch systems that are based on high-performance computing resources; however, the bioinformatics software that is built on this platform  ...  Acknowledgements We thank system experts Pontus Freyhult, and Peter Ankerstål at UPPMAX for valuable discussions on effective storage and network usage.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1186/s13742-015-0058-5">doi:10.1186/s13742-015-0058-5</a> <a target="_blank" rel="external noopener" href="https://www.ncbi.nlm.nih.gov/pubmed/26045962">pmid:26045962</a> <a target="_blank" rel="external noopener" href="https://pubmed.ncbi.nlm.nih.gov/PMC4455317/">pmcid:PMC4455317</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/g7h67xeysjhrtgdsuerveo2jjm">fatcat:g7h67xeysjhrtgdsuerveo2jjm</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20200208132413/http://europepmc.org/backend/ptpmcrender.fcgi?accid=PMC4455317&amp;blobtype=pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/4e/7c/4e7c664adc0bfc1800d70d6593515c78df1509db.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1186/s13742-015-0058-5"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="unlock alternate icon" style="background-color: #fb971f;"></i> Publisher / doi.org </button> </a> <a target="_blank" rel="external noopener" href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4455317" title="pubmed link"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> pubmed.gov </button> </a>

Scalable parallel computing on clouds using Twister4Azure iterative MapReduce

Thilina Gunarathne, Bingjing Zhang, Tak-Lon Wu, Judy Qiu
<span title="">2013</span> <i title="Elsevier BV"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/idqeix2ukbd5fbgjjdkjmaj3ka" style="color: black;">Future generations computer systems</a> </i> &nbsp;
Performance measurements show comparable or a factor of 2 to 4 better results than the traditional MapReduce runtimes deployed on up to 256 instances and for jobs with tens of thousands of tasks.  ...  The utility computing model introduced by cloud computing, combined with the rich set of cloud infrastructure and storage services, offers a very attractive environment in which scientists can perform  ...  These applications running on the Azure Cloud exhibited performance comparable to the Apache Hadoop on a dedicated local cluster.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1016/j.future.2012.05.027">doi:10.1016/j.future.2012.05.027</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/7555zbco7rggvhvpvswuvsnteu">fatcat:7555zbco7rggvhvpvswuvsnteu</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20121114195044/http://grids.ucs.indiana.edu/ptliupages/publications/Scalable_Parallel_Computing_on_Clouds_Using_Twister4Azure_Iterative_MapReduce_cr_submit.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/a2/a6/a2a64dd247410dd2a040b59a52a69447acb29440.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1016/j.future.2012.05.027"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> elsevier.com </button> </a>

MARIANE: MApReduce Implementation Adapted for HPC Environments

Zacharia Fadika, Elif Dede, Madhusudhan Govindaraju, Lavanya Ramakrishnan
<span title="">2011</span> <i title="IEEE"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/7sf5a2agbvfunpi7fyrezz2t5y" style="color: black;">2011 IEEE/ACM 12th International Conference on Grid Computing</a> </i> &nbsp;
The most popular open source implementation of MapReduce, Hadoop, is based on the Hadoop Distributed File System (HDFS).  ...  This paper shows the applicability and high performance of the MapReduce paradigm through MARIANE, an implementation designed for clustered and shared-disk file systems and as such not dedicated to a specific  ...  These measures to storage management cast both Twister and Dryad, as Hadoop and DELMA, with the inability as of yet to flourish on traditional HPC frameworks. VIII.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/grid.2011.20">doi:10.1109/grid.2011.20</a> <a target="_blank" rel="external noopener" href="https://dblp.org/rec/conf/grid/FadikaDGR11.html">dblp:conf/grid/FadikaDGR11</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/nv4q5i44sffzfkfg7qtypy6z6y">fatcat:nv4q5i44sffzfkfg7qtypy6z6y</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20170922005728/http://www.cs.binghamton.edu/~mgovinda/papers/MARIANE-GRID-2011-cameraready.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/b4/e7/b4e719e6f459bad46c7993ac6eaef6fbe756d53c.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/grid.2011.20"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> ieee.com </button> </a>

Hadoop on HPC: Integrating Hadoop and Pilot-based Dynamic Resource Management [article]

Andre Luckow and Ioannis Paraskevakos and George Chantzialexiou and Shantenu Jha
<span title="2016-01-31">2016</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
The Apache Hadoop ecosystem has evolved to meet the requirements of data processing applications and has addressed many of the limitations of HPC platforms.  ...  For example, the scientific domains of bio-molecular dynamics, genomics and network science need to couple traditional computing with Hadoop/Spark based analysis.  ...  We acknowledge financial support of NSF 1440677 and NSF 1443054. This work has also been made possible thanks to computer resources provided by XRAC award TG-MCB090174 and an Amazon Computing Award.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1602.00345v1">arXiv:1602.00345v1</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/iyrqqxv4xjdwbdcwil2v23rfom">fatcat:iyrqqxv4xjdwbdcwil2v23rfom</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20200914092401/https://arxiv.org/pdf/1602.00345v1.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/ae/22/ae22b458118c599001bc39973decdb429b9433da.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1602.00345v1" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>

Pilot-Abstraction: A Valid Abstraction for Data-Intensive Applications on HPC, Hadoop and Cloud Infrastructures? [article]

Andre Luckow, Pradeep Mantha, Shantenu Jha
<span title="2015-01-21">2015</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
HPC environments have traditionally been designed to meet the compute demand of scientific applications and data has only been a second order concern.  ...  Further, there is a lack of abstractions that unify access to increasingly heterogeneous infrastructure (HPC, Hadoop, clouds) and allow reasoning about performance trade-offs in this complex environment  ...  While the Hadoop platform has proven its value in scientific applications, challenges remain in deploying ABDS applications on HPC infrastructure and when integrating these with HPC applications, e. g.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1501.05041v1">arXiv:1501.05041v1</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/eiu3inxk7bblrcoh7orjkrimjq">fatcat:eiu3inxk7bblrcoh7orjkrimjq</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20200929165352/https://arxiv.org/pdf/1501.05041v1.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/ee/03/ee0359e0918cf2eae653a1382fda480b9260e023.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1501.05041v1" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>

Building a Private HPC Cloud for Compute and Data-Intensive Applications

Moussa Taifi, Abdallah Khreishah, Justin Y. Shi
<span title="2013-04-30">2013</span> <i title="Academy and Industry Research Collaboration Center (AIRCC)"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/vly56cfljjfflmkyub2mvqs63m" style="color: black;">International Journal on Cloud Computing Services and Architecture</a> </i> &nbsp;
The orderly batch-oriented HPC cluster offers maximal potential for performance per application, but limits resource efficiency and user flexibility.  ...  We report our evaluation criteria using Open Source software, and performance studies for compute-intensive and data-intensive applications.  ...  ACKNOWLEDGEMENTS The authors would like Thomas Stauffer, AakashPradeep and Dr.Jie Wu for useful feedback and suggestions, as well operational support during the design and deployment phases of the private  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.5121/ijccsa.2013.3201">doi:10.5121/ijccsa.2013.3201</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/73pwlzqdgnavlp6rjhs2yjtycq">fatcat:73pwlzqdgnavlp6rjhs2yjtycq</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20180603065239/http://www.airccse.org/journal/ijccsa/papers/3213ijccsa01.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/a7/ba/a7ba401b762c707a73daf222c44e4f61383d094b.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.5121/ijccsa.2013.3201"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> Publisher / doi.org </button> </a>

Mission possible: Unify HPC and Big Data stacks towards application-defined blobs at the storage layer

Pierre Matri, Yevhen Alforov, Álvaro Brandon, María S. Pérez, Alexandru Costan, Gabriel Antoniu, Michael Kuhn, Philip Carns, Thomas Ludwig
<span title="">2018</span> <i title="Elsevier BV"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/idqeix2ukbd5fbgjjdkjmaj3ka" style="color: black;">Future generations computer systems</a> </i> &nbsp;
However, on HPC platforms this is made difficult by the centralized storage architecture using file-based storage.  ...  In this paper we advocate that the growing trend of equipping HPC compute nodes with local storage redistributes the cards by enabling object storage to be deployed alongside the application on the compute  ...  Acknowledgments This work is part of the "BigStorage: Storagebased Convergence between HPC and Cloud to handle Big Data" project, H2020-MSCA-ITN-2014-642963, funded by the European Commission within the  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1016/j.future.2018.07.035">doi:10.1016/j.future.2018.07.035</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/2lr4a4t34nau3e3pkgrpmfcfu4">fatcat:2lr4a4t34nau3e3pkgrpmfcfu4</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20190503112251/https://hal.archives-ouvertes.fr/hal-01892682/file/HPC_BD_Convergence__Elsevier_-2.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/67/fb/67fb89d5980957cd46d12a5272e77a15d93e6137.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1016/j.future.2018.07.035"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> elsevier.com </button> </a>

Recommendations on e-infrastructures for next-generation sequencing

Ola Spjuth, Erik Bongcam-Rudloff, Johan Dahlberg, Martin Dahlö, Aleksi Kallio, Luca Pireddu, Francesco Vezzi, Eija Korpelainen
<span title="2016-06-07">2016</span> <i title="Oxford University Press (OUP)"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/apde6vbe7vfoxkfibp7n4mzonm" style="color: black;">GigaScience</a> </i> &nbsp;
We cover demands on computational and storage resources, networks, software stacks, automation of analysis, education, and also discuss emerging trends in the field.  ...  In this work, we provide recommendations based on the collective experiences from participants in the EU COST Action SeqAhead for the tasks of data preprocessing, upstream processing, data delivery, and  ...  Acknowledgements The authors would like to thank the members of the COST Action BM1006 Next Generation Sequencing Data Analysis Network SeqAhead for studies, presentations, workshops, and discussions leading  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1186/s13742-016-0132-7">doi:10.1186/s13742-016-0132-7</a> <a target="_blank" rel="external noopener" href="https://www.ncbi.nlm.nih.gov/pubmed/27267963">pmid:27267963</a> <a target="_blank" rel="external noopener" href="https://pubmed.ncbi.nlm.nih.gov/PMC4897895/">pmcid:PMC4897895</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/vwt5p5p6q5gpdkzt5ezdsyebsy">fatcat:vwt5p5p6q5gpdkzt5ezdsyebsy</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20171016033705/http://publisher-connector.core.ac.uk/resourcesync/data/Springer-OA/pdf/915/aHR0cDovL2xpbmsuc3ByaW5nZXIuY29tLzEwLjExODYvczEzNzQyLTAxNi0wMTMyLTcucGRm.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/08/bf/08bf0d0a1de6af11610dfc7f086be703276c5fc6.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1186/s13742-016-0132-7"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="unlock alternate icon" style="background-color: #fb971f;"></i> Publisher / doi.org </button> </a> <a target="_blank" rel="external noopener" href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4897895" title="pubmed link"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> pubmed.gov </button> </a>

Big Data Techniques, Systems, Applications, and Platforms: Case Studies from Academia

Atanas Radenski, Todor Gurov, Kalinka Kaloyanova, Nikolay Kirov, Maria Nisheva, Peter Stanchev, Eugenia Stoimenova
<span title="2016-10-02">2016</span> <i title="IEEE"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/zbznsfsymfchdlmdsmo2bglth4" style="color: black;">Proceedings of the 2016 Federated Conference on Computer Science and Information Systems</a> </i> &nbsp;
Big data is a broad term with numerous dimensions, most notably: big data characteristics, techniques, software systems, application domains, computing platforms, and big data milieu (industry, government  ...  In this paper we briefly introduce fundamental big data characteristics and then present seven case studies of big data techniques, systems, applications, and platforms, as seen from academic perspective  ...  To do so, the authors measured the performance of a basic Spark codon count algorithm and compared it to a couple of Hadoop MR algorithms: a basic Hadoop MR codon count algorithm and an optimized "local  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.15439/2016f91">doi:10.15439/2016f91</a> <a target="_blank" rel="external noopener" href="https://dblp.org/rec/conf/fedcsis/RadenskiGKKNSS16.html">dblp:conf/fedcsis/RadenskiGKKNSS16</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/22ekovjsajd5hhilzczvzoay7m">fatcat:22ekovjsajd5hhilzczvzoay7m</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20180424084245/https://annals-csis.org/Volume_8/pliks/91.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/49/c7/49c7b4a57547eaf081d0ff80bea45c52e68f9c85.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.15439/2016f91"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> Publisher / doi.org </button> </a>

Will They Blend?: Exploring Big Data Computation Atop Traditional HPC NAS Storage

Ellis H. Wilson, Mahmut T. Kandemir, Garth Gibson
<span title="">2014</span> <i title="IEEE"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/s77bfh66ena2znlnuhuruo464a" style="color: black;">2014 IEEE 34th International Conference on Distributed Computing Systems</a> </i> &nbsp;
In our exploration, we unearth subtle pitfalls resultant from this mashup of new-era Big Data computation on conventional HPC storage and share the clever architectural configurations that allow us to  ...  within these traditional High Performance Computing (HPC) architectures.  ...  Because HPC computing systems are of comparable scale to Big Data clusters, it is particularly interesting to be able to support both types of applications using existing HPC storage for convenience and  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/icdcs.2014.60">doi:10.1109/icdcs.2014.60</a> <a target="_blank" rel="external noopener" href="https://dblp.org/rec/conf/icdcs/WilsonKG14.html">dblp:conf/icdcs/WilsonKG14</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/zn6tchq5czg35frhqhts3oykxu">fatcat:zn6tchq5czg35frhqhts3oykxu</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20170921203100/http://repository.cmu.edu/cgi/viewcontent.cgi?article=3605&amp;context=compsci" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/dc/dd/dcdd350c20c532b06d555deaa5e29a7d3af22686.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/icdcs.2014.60"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> ieee.com </button> </a>

On the Performance of Spark on HPC Systems: Towards a Complete Picture [chapter]

Orcun Yildiz, Shadi Ibrahim
<span title="">2018</span> <i title="Springer International Publishing"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/2w3awgokqne6te4nvlofavy5a4" style="color: black;">Lecture Notes in Computer Science</a> </i> &nbsp;
However, with the growing needs of users and size of data, commodity-based infrastructure will strain under the heavy weight of Big Data.  ...  As first steps toward Big Data processing on HPC systems, several research efforts have been devoted to understanding the performance of Big Data applications on these systems.  ...  RENATER and other contributing partners (see http://www.grid5000.fr/ for details).  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1007/978-3-319-69953-0_5">doi:10.1007/978-3-319-69953-0_5</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/5wv5scvzkzcz3ddcvlhfaod7cy">fatcat:5wv5scvzkzcz3ddcvlhfaod7cy</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20210718193812/https://hal.archives-ouvertes.fr/hal-01742016/document" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/0b/7f/0b7ff2056ae60d27156501c4c2d78aa4c3fddfc0.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1007/978-3-319-69953-0_5"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> springer.com </button> </a>

Building a research data science platform from industrial machines

Fang Cherry Liu, Fu Shen, Duen Horng Chau, Neil Bright, Mehmet Belgin
<span title="">2016</span> <i title="IEEE"> <a target="_blank" rel="noopener" href="https://fatcat.wiki/container/faqqmambavbalpofpx3p6nntua" style="color: black;">2016 IEEE International Conference on Big Data (Big Data)</a> </i> &nbsp;
Using the Hadoop platform and tools built on top of it drew immense interest from academia after it gained success in industry.  ...  While traditional HPC offers tools on leveraging existing technologies with data processing needs, the large volume of data and the speed of data generation pose significant challenges.  ...  We chose RAID 1 for service node on a mirrored 1TB hard drive and chose to not use RAID for slave nodes with one 500GB hard drive dedicated to the OS while another 500GB hard drive dedicated to data storage  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/bigdata.2016.7840859">doi:10.1109/bigdata.2016.7840859</a> <a target="_blank" rel="external noopener" href="https://dblp.org/rec/conf/bigdataconf/LiuSCBB16.html">dblp:conf/bigdataconf/LiuSCBB16</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/z2pp57xhsvdkfjdqlutvnjuita">fatcat:z2pp57xhsvdkfjdqlutvnjuita</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20170516225542/http://www.cc.gatech.edu/~dchau/papers/16-bigdata-platform.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/31/69/316917c4ca1d12e570e660165759646cb700d20e.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/bigdata.2016.7840859"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> ieee.com </button> </a>

Magellan

Lavanya Ramakrishnan, Anping Liu, Piotr T. Zbiegel, Scott Campbell, Rick Bradshaw, Richard Shane Canon, Susan Coghlan, Iwona Sakrejda, Narayan Desai, Tina Declerck
<span title="">2011</span> <i title="ACM Press"> Proceedings of the 2nd international workshop on Scientific cloud computing - ScienceCloud &#39;11 </i> &nbsp;
However, there is a limited understanding on how to operate and use clouds for scientific applications.  ...  Cloud resources promise to be an avenue to address new categories of scientific applications including data-intensive science applications, on-demand/surge computing, and applications that require customized  ...  The performance across a HPC machine, virtual machines, and a Hadoop cluster was found to be comparable (within 10%), making this a feasible application for clouds as well.  ... 
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1145/1996109.1996119">doi:10.1145/1996109.1996119</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/d2qjc7ckcbb6fegpoklmquvkpq">fatcat:d2qjc7ckcbb6fegpoklmquvkpq</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20160518235230/http://homepages.cs.ncl.ac.uk:80/paolo.missier/doc/p07.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/16/db/16dbf6858cf894b0f0acd7eb7b072e3199d76fa2.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1145/1996109.1996119"> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> acm.org </button> </a>
&laquo; Previous Showing results 1 &mdash; 15 out of 622 results