67 Hits in 7.2 sec

Effective Management of ReRAM-based Hybrid SSD for Multiple Node HDFS

Nayoung Park, Byungjun Lee, Kyung Tae Kim, Hee Yong Youn
2015 International Journal of Networked and Distributed Computing (IJNDC)  
Most existing researches of Hybrid SSD are based on a single storage, while the management of multiple nodes like HDFS is still immature.  ...  Recently, the research of Hybrid ReRAM/MLC NAND SSD is rapidly expanding into the storage areas.  ...  In order to satisfy the strict requirements on the future storage system of higher speed, reliability and energy-efficiency, SSD is preferred.  ... 
doi:10.2991/ijndc.2015.3.3.4 fatcat:hsjumdql45aqhc66lpnr4nz3gi

Eco-Storage: A Hybrid Storage System with Energy-Efficient Informed Prefetching

Maen M. Al Assaf, Xunfei Jiang, Mohamed Riduan Abid, Xiao Qin
2013 Journal of Signal Processing Systems  
We show that these two steps can be handled in parallel to decreases the system's power consumption. Our Eco-Storage technique differs from existing energy-aware prefetching schemes in two ways.  ...  In this paper, we present an energy-aware informed prefetching technique called Eco-Storage that makes use of the application-disclosed access patterns to group the informed prefetching process in a hybrid  ...  For example, HDFS (Hadoop Distributed File System) data block size is 64 MB [21] .  ... 
doi:10.1007/s11265-013-0784-9 fatcat:ijzqnds53vdpjjtoj5ry4wl4zm

Analysis and evaluation of MapReduce solutions on an HPC cluster

Jorge Veiga, Roberto R. Expósito, Guillermo L. Taboada, Juan Touriño
2016 Computers & electrical engineering  
The results have shown that new frameworks like DataMPI can outperform Hadoop, although using IP over InfiniBand also provides significant benefits without code modifications.  ...  This work aims to establish a taxonomy of these frameworks together with a thorough evaluation, which has been carried out in terms of performance and energy efficiency metrics.  ...  Acknowledgements This work was supported by the Ministry of Economy and Competitiveness of Spain and FEDER funds of the EU (Project TIN2013-42148-P).  ... 
doi:10.1016/j.compeleceng.2015.11.021 fatcat:d3n3fkxit5hypdb5mlwh2j7wiy

Predictive Prefetching for Parallel Hybrid Storage Systems

Maen M. Al Assaf
2015 International Journal of Communications, Network and System Sciences  
The fundamental concept of our approach is to invoke parallel hybrid storage system's parallelism and prefetch data among multiple storage levels (e.g. solid state disks, and hard disk drives) in parallel  ...  Our results show that our PPHSS can improve system performance by 4% across real-world I/O traces without the need of using large size caches.  ...  For example, the data block size in HDFS (Hadoop Distributed File System) is 64 MB [39] . In reference [40] , HDFS block size is increased to improve system performance.  ... 
doi:10.4236/ijcns.2015.85018 fatcat:ehw2wtoshzdo7l6b42pgswzqlu

Evaluation of Distributed Databases in Hybrid Clouds and Edge Computing: Energy, Bandwidth, and Storage Consumption [article]

Yaser Mansouri, Victor Prokhorenko, Faheem Ullah, M. Ali Babar
2021 arXiv   pre-print
To address these research gaps, in this paper, we investigate energy, bandwidth and storage consumption of the most used and common distributed databases.  ...  While most of the state-of-the-art studies have investigated only response time and scalability of distributed databases, focusing on other various metrics (e.g., energy, bandwidth, and storage consumption  ...  As the databases are moved into the edge server node and hybrid cloud, this percentage of energy consumption for CPU decreases and for REST increases.  ... 
arXiv:2109.07260v2 fatcat:uyysj7ilyrf7rfke3ks7e3ylwy

A comprehensive view of Hadoop research—A systematic literature review

Ivanilton Polato, Reginaldo Ré, Alfredo Goldman, Fabio Kon
2014 Journal of Network and Computer Applications  
Our objective was to identify gaps, providing motivation for new research, and outline collaborations to Apache Hadoop and its ecosystem, classifying and quantifying the main topics addressed in the literature  ...  Recently, the number of publications in journals and conferences about Hadoop has increased consistently, which makes it difficult for researchers to comprehend the full body of research and areas that  ...  Table A1 and A2 Table A1 Studies with implementation and/or experiments (MapReduce and data storage & manipulation categories). Appendix A.  ... 
doi:10.1016/j.jnca.2014.07.022 fatcat:4xjveqy6mrctzjc4ou7llyy4u4

Human-Computer Interaction of Networked Vehicles Based on Big Data and Hybrid Intelligent Algorithm

Jianfeng Shang, Huiying Liu, Weidong Li, Mohammad Farukh Hashmi
2022 Wireless Communications and Mobile Computing  
Based on the proposed big data platform, the parallel programming framework of MapReduce and HDFS distributed storage system are used to process the real-time vehicle dynamic information in parallel, and  ...  the output result is used as the input of running genetic algorithm simulated annealing (GA-SA) for parallel calculation.  ...  Acknowledgments This work was supported in part by the National Key R&D Program of China (Grant No. 2018YFB1402600) and the National Science Foundation of China (Grant No. 61772190).  ... 
doi:10.1155/2022/5281132 fatcat:vd7yenlb3ffnbbrjnrrii6gc5q

A Survey of Big Data Machine Learning Applications Optimization in Cloud Data Centers and Networks [article]

Sanaa Hamid Mohamed, Taisir E.H. El-Gorashi, Jaafar M.H. Elmirghani
2019 arXiv   pre-print
The MapReduce programming model and its widely-used open-source platform; Hadoop, are enabling the development of a large number of cloud-based services and big data applications.  ...  and power consumption.  ...  This work was supported by the Engineering and Physical Sciences Research Council, INTERNET (EP/H040536/1), STAR (EP/K016873/1) and TOWS (EP/S016570/1) projects.  ... 
arXiv:1910.00731v1 fatcat:kvi3br4iwzg3bi7fifpgyly7m4

Using Hadoop Technology to Overcome Big Data Problems by Choosing Proposed Cost-efficient Scheduler Algorithm for Heterogeneous Hadoop System (BD3)

Abou_el_ela Abdou Hussein
2020 Journal of Scientific Research and Reports  
We highlight the challenges that face big data processing and how to overcome these challenges using Hadoop and its use in processing big data sets as a solution for resolving various problems in a distributed  ...  Also we institutes absolute description of Hadoop Pros and cons and improvements to face hadoop problems by choosing proposed Cost-efficient Scheduler Algorithm for heterogeneous Hadoop system.  ...  This hybrid method can efficiently raise data transfer speed.  ... 
doi:10.9734/jsrr/2020/v26i930310 fatcat:tph6busczrbntgzpdidvcqb454

Introduction to Big Data Technology [chapter]

Bilal Abu-Salih, Pornpit Wongthongtham, Dengya Zhu, Kit Yan Chan, Amit Rudra
2021 Social Big Data Analytics  
This chapter will first have historical review of big data; followed by discussion of characteristics of big data, i.e. from the 3V's to up 10V's of big data.  ...  Big data is no more "all just hype" but widely applied in nearly all aspects of our business, governments, and organizations with the technology stack of AI.  ...  In particular various currently incorporated technologies, tools, APIs, and approaches are discussed that are used from infrastructure/platform/ecosystem to constructional units and components.  ... 
doi:10.1007/978-981-33-6652-7_2 fatcat:dog5ym666famdedniwyewwswiq

Big Data Methodologies, Tools And Infrastructures

Kim Hee, Todor Ivanov, Roberto V. Zicari, Rut Waldenfels, Hevin Özmen, Naveed Mushtaq, Minsung Hong, Tharsis Teoh, Rajendra Akerkar
2018 Zenodo  
This report, which is a follow up of Deliverable 1.1, offers an in-depth introduction to relevant technologies for Big Data Analytics and Big Data Management.  ...  The goal is to create value out of this amount of data, by providing a comprehensive picture of what's happening, using business analytics, leveraging big data tools and predictive analytics, to help transportation  ...  The main challenge is using significantly improved technologies and methods to gather and understand the data in order for business decisions to be informed by better insights.  ... 
doi:10.5281/zenodo.1465539 fatcat:mkad5yu2tnfw7fdi3xqcermac4

A time–energy performance analysis of MapReduce on heterogeneous systems with GPUs

Dumitrel Loghin, Lavanya Ramapantulu, Oana Barbu, Yong Meng Teo
2015 Performance evaluation (Print)  
We evaluate the time and energy performance of three MapReduce applications with diverse resource demands on a Hadoop-CUDA framework.  ...  For compute-intensive workloads, the brawny heterogeneous system achieves speedups of up to 2.3 and reduces the energy usage by almost half compared to the brawny homogeneous system.  ...  Acknowledgements We are grateful to Nvidia for providing us with four Jetson TK1 boards.  ... 
doi:10.1016/j.peva.2015.06.015 fatcat:h2e3dk2dwjawdlgc4t3g7g3lme

A Survey on Large Scale Metadata Server for Big Data Storage [article]

Ripon Patgiri, Sabuzima Nayak
2020 arXiv   pre-print
MDS, and g) Tree-based MDS.  ...  Thus, MDS is categorized in various ways depending on the underlying architecture and design methodology. The article surveys on the various kinds of MDS architecture, designs, and methodologies.  ...  Also, DDcache improves execution speed which reduces the coherence latency, which also reduces the energy consumption.  ... 
arXiv:2005.06963v1 fatcat:6i2qvakfqzbjtfe5rofd2z7e6u

SupMR: Circumventing Disk and Memory Bandwidth Bottlenecks for Scale-up MapReduce

Michael Sevilla, Ike Nassi, Kleoni Ioannidou, Scott Brandt, Carlos Maltzahn
2014 2014 IEEE International Parallel & Distributed Processing Symposium Workshops  
In this paper, we mitigate the ingest and merge bottlenecks by leveraging the scale-up MapReduce model.  ...  Our techniques are based on well-known algorithms and scale-out MapReduce optimizations, but applying them to a scale-up computation framework to mitigate the ingest and merge bottlenecks is novel.  ...  We also identify utilization and energy consumption as significant factors in comparing this approach to an "equivalent" scale-out implementation.  ... 
doi:10.1109/ipdpsw.2014.168 dblp:conf/ipps/SevillaNIBM14 fatcat:r6ensxkeifae5ove7i7kbd3bje

Power and Performance Evaluation of Memory-Intensive Applications

Kaiqiang Zhang, Dongyang Ou, Congfeng Jiang, Yeliang Qiu, Longchuan Yan
2021 Energies  
The findings we present in this paper provide useful insights and guidance for system designers and data center operators to help them in energy-efficiency-aware job scheduling and energy conservation.  ...  In terms of power and energy consumption, DRAMs play a key role in a modern server system as well as processors.  ...  For example, the hybrid memory cube (HMC) [35] has promised to enhance bandwidth and density and decrease power consumption for the next-generation main memory systems.  ... 
doi:10.3390/en14144089 fatcat:zqsxdvo4yvd6pc7qfmoo5lcuki
« Previous Showing results 1 — 15 out of 67 results