203 Hits in 4.3 sec

Compaction management in distributed key-value datastores

Muhammad Yousuf Ahmad, Bettina Kemme
2015 Proceedings of the VLDB Endowment  
Storage infrastructure behind Facebook Messages: Using HBase at scale. IEEE Data Eng.  ...  After execution: degrade read latency severely -Input files removed, evicted en masse from cache ► Cache misses COMPACT RAM UPDATES C O M P A C T W A R M U P Application HBase Client HBase Master  ... 
doi:10.14778/2757807.2757810 fatcat:g476nttln5h57cx2bd6izh5dde

Apache hadoop goes realtime at Facebook

Dhruba Borthakur, Samuel Rash, Rodrigo Schmidt, Amitanand Aiyer, Jonathan Gray, Joydeep Sen Sarma, Kannan Muthukkaruppan, Nicolas Spiegelberg, Hairong Kuang, Karthik Ranganathan, Dmytro Molkov, Aravind Menon
2011 Proceedings of the 2011 international conference on Management of data - SIGMOD '11  
in other applications at Facebook and many other web-scale companies.  ...  Apache HBase is a database-like layer built on Hadoop designed to support billions of messages per day.  ...  We have Acknowledgements are also due to Patrick Kling for implementing a test suite for HDFS HA as part of his internship at Facebook.  ... 
doi:10.1145/1989323.1989438 dblp:conf/sigmod/BorthakurGSMSKRMMRSA11 fatcat:gnicex2fwzbotcpmlaqhz7k2mm

On the energy efficiency of client-centric data consistency management under random read/write access to Big Data with Apache HBase [article]

Álvaro García-Recuero
2016 arXiv   pre-print
of the system as well as lessons learned to underpin further design of energy-efficient cluster scale storage systems.  ...  On the other hand, computational developments have started to lag behind storage advancements, therein becoming a future bottleneck for the ongoing data growth which already approaches Exascale levels.  ...  with the messages application at Facebook, and balanced in order to see the effect of a mixed workload (50%-50%).  ... 
arXiv:1509.02640v7 fatcat:w4sewclujrd73bh4yqdcbmchru

On the Application of Cloud Computing in Educational Informatization

Jian-yong FAN
2017 DEStech Transactions on Economics Business and Management  
According to incomplete statistics, the current number of Facebook users has exceeded 2.2 billion people. The number of messages are sent out every day on the Facebook platform as high as 12 billion.  ...  HBase is different from the general relational database, it is a suitable for unstructured data storage distributed, column-oriented database. Hive is a data warehouse infrastructure built on Hadoop.  ... 
doi:10.12783/dtem/eced2017/9838 fatcat:w7jhka376zbctfwc2mh7elcura

An overview of the Hadoop/MapReduce/HBase framework and its current applications in bioinformatics

Ronald C Taylor
2010 BMC Bioinformatics  
The concepts behind Hadoop and the associated HBase project are defined, and current bioinformatics software that employ Hadoop is described.  ...  Bioinformatics researchers are now confronted with analysis of ultra large-scale data sets, a problem that will only increase at an alarming rate in coming years.  ...  Acknowledgements RCT thanks the Bioinformatics Open Source Conference (BOSC) for the opportunity to present a talk on this subject at the July 2010 BOSC meeting, of which this article is an expansion.  ... 
doi:10.1186/1471-2105-11-s12-s1 pmid:21210976 pmcid:PMC3040523 fatcat:r74rokyv6fc45dfid2fd6qgk64

Hadoop Ecosystem: An Introduction

2016 International Journal of Science and Research (IJSR)  
But, No one uses kernel alone. "Hadoop" is taken to be a combination of HDFS and MapReduce.  ...  To complement the Hadoop modules there are also a variety of other projects that provide specialized services and are broadly used to make Hadoop laymen accessible and more usable, collectively known as  ...  At the present time, Pig's infrastructure layer consists of a compiler that produces sequences of Map-Reduce programs, for which large-scale parallel implementations already exist (e.g., the Hadoop subproject  ... 
doi:10.21275/v5i6.nov164121 fatcat:fep6wrhjdfcknjghm3nnvdqmnq

Consensus in the Cloud: Paxos Systems Demystified

Ailidani Ailijiang, Aleksey Charapko, Murat Demirbas
2016 2016 25th International Conference on Computer Communication and Networks (ICCCN)  
We also categorize the coordination use-patterns in cloud, and examine Google and Facebook infrastructures, as well as Apache toplevel projects to investigate how they use Paxos protocols and systems.  ...  In order to elucidate the correct use of distributed coordination systems, we compare and contrast popular Paxos protocols and Paxos systems and present advantages and disadvantages for each.  ...  MySQL is used as a relational backbone at Facebook with many higher level storage systems, such as Haystack and f4, interacting with it.  ... 
doi:10.1109/icccn.2016.7568499 dblp:conf/icccn/AilijiangCD16 fatcat:5ywmioihzffddiajsa3tv4qhei

Big Data Storage Tools Using NoSQL Databases and Their Applications in Various Domains: A Systematic Review

Amen Faridoon, Muhammad Imran
2021 Computing and informatics  
Facebook creates 4 petabytes of data. The 5 billion searches are made and 65 billion messages are sent on WhatsApp. The 4 petabytes of data are created from each connected car.  ...  Initially, Cassandra was developed at Facebook to power the inbox search feature. Now it has become an Apache incubator project.  ... 
doi:10.31577/cai_2021_3_489 fatcat:mh6jl6vtznf5zmayq6l3yxaaga

Clouder: a flexible large scale decentralized object store

Ricardo Vilaça, Rui Oliveira
2009 Proceedings of the Third Workshop on Dependable Distributed Data Management - WDDM '09  
large sets of related data at once.  ...  This thesis aims at reducing the gap between traditional RDBMS and large scale data stores, by seeking mechanisms to provide additional consistency guarantees and higher level data processing primitives  ...  Cassandra (Lakshman and Malik 2010 ) is a distributed storage engine initially developed by Facebook to be used at the Facebook social network site and is now an Apache open source project 6 .  ... 
doi:10.1145/1518691.1518699 fatcat:bmegtojlkngnrecectx3o5d5zm

Enhancing Query Support in HBase via an Extended Coprocessors Framework [chapter]

Himanshu Vashishtha, Eleni Stroulia
2011 Lecture Notes in Computer Science  
In pioneer days they used oxen for heavy pulling, and when one ox could not budge a log, they did not try to grow a larger ox.  ...  Instead, infrastructure is increasingly provided as a service (IaaS) with specialized providers offering large-scale computational resources (computing power, storage and network bandwidth) at economical  ...  In this new model of computing infrastructure, one can lease storage and virtual machines, configured in a manner that meets the computing task at hand.  ... 
doi:10.1007/978-3-642-24755-2_7 fatcat:wfyafv7hcrex7fygsdkljgzxj4

Graph Aware Caching Policy for Distributed Graph Stores

Hidayet Aksu, Mustafa Canim, Yuan-Chi Chang, Ibrahim Korpeoglu, Ozgur Ulusoy
2015 2015 IEEE International Conference on Cloud Engineering  
We implemented graph aware caching on a distributed data store Apache HBase in the Hadoop family.  ...  practice, applications ranging from social networks, knowledge representations to Internet of things benefit from graph data stores built on a combination of relational and non-relational technologies aimed at  ...  It can be invoked by an HBase client to execute at one or multiple target regions in parallel.  ... 
doi:10.1109/ic2e.2015.39 dblp:conf/ic2e/AksuCCKU15 fatcat:boumt4wnqnb7jpgsjtqcjn6wae

A Comprehensive Study of HBase Storage Architecture—A Systematic Literature Review

Muhammad Umair Hassan, Irfan Yaqoob, Sidra Zulfiqar, Ibrahim A. Hameed
2021 Symmetry  
There is a fight to use and store this tremendous information effectively. HBase is the top option for storing huge data.  ...  We perform a systematic literature review on a number of published works proposed for HBase storage architecture.  ...  The first time Facebook selected HBase to implement its new messaging platform was in November 2010 [10] .  ... 
doi:10.3390/sym13010109 fatcat:6jqnicyw55fgrdjxrjxsl3bozm

A comprehensive social media data processing and analytics architecture by using big data platforms: a case study of twitter flood-risk messages

Michal Podhoranyi
2021 Earth Science Informatics  
The secondary objective, trying to describe the flood alert situation by using only Tweet messages and exploring the informative potential of such data is demonstrated as well.  ...  Based on the flood tweet analysis, it was observed that Twitter messages with some considerations are informative enough to be used to estimate general flood alert situations in particular regions.  ...  Such functionality delivers distributed frameworks for large-scale data processing.  ... 
doi:10.1007/s12145-021-00601-w pmid:33727982 pmcid:PMC7951942 fatcat:bxsqn2dbb5ee3crzdsmhifuuyy

Empirical Analysis of Recent Advances, Characteristics and Challenges of Big Data

Burhanullah Khattak, Aurangzeb Khan, Khairullah Khan, Wahab Khan, Muhammad Kamran, Muhammad Fahad
2018 EAI Endorsed Transactions on Scalable Information Systems  
Hbase is incorporated in several big data solutions and data-driven websites like Platform of Facebook for Messaging.  ...  Large data presenters like Google, Yahoo Flickr, and Facebook, run a hyper-scale computing environment.  ... 
doi:10.4108/eai.13-7-2018.159621 fatcat:wukpkji5cfgqlp47ucc6ugubdy

Using Hadoop Technology to Overcome Big Data Problems by Choosing Proposed Cost-efficient Scheduler Algorithm for Heterogeneous Hadoop System (BD3)

Abou_el_ela Abdou Hussein
2020 Journal of Scientific Research and Reports  
This paper mainly focuses on different components of hadoop like Hive, Pig, and Hbase, etc.  ...  We highlight the challenges that face big data processing and how to overcome these challenges using Hadoop and its use in processing big data sets as a solution for resolving various problems in a distributed  ...  HDFS is data storage infrastructure without losing large chunks of failure, massive amounts of information stored in the scale are progressively viable.  ... 
doi:10.9734/jsrr/2020/v26i930310 fatcat:tph6busczrbntgzpdidvcqb454
« Previous Showing results 1 — 15 out of 203 results