237 Hits in 7.3 sec

Re-enabling high-speed caching for LSM-trees [article]

Lei Guo, Dejun Teng, Rubao Lee, Feng Chen, Siyuan Ma, Xiaodong Zhang
2016 arXiv   pre-print
We propose dLSM in order to re-enable high-speed caching during intensive writes. dLSM is an LSM-tree with a compaction buffer on the disk, working as a cushion to minimize the cache invalidation caused  ...  With the help of a small on-disk compaction buffer, dLSM achieves a high query performance by enabling effective caching, while retaining all merits of LSM-tree for write-intensive data processing.  ...  Our solution and contributions In this paper, we propose dLSM-tree (dLSM in short), in order to enable high speed caching for write intensive data.  ... 
arXiv:1606.02015v1 fatcat:xtasfdttnvftpkfmsci7c7ro6m

Storage Management with Multi-Version Partitioned B-Trees [article]

Christian Riegger, Ilia Petrov
2022 arXiv   pre-print
Secondly, we compare MV-PBT against LSM-Trees.  ...  MV-PBT offers an up to 2x increased steady throughput in comparison to LSM-Trees and several orders of magnitude in comparison to B+ Trees in a YCSB workload.  ...  are optimized for high update rates and obtain a sequential write pattern, since modifications are buffered in tree-based LSM components in main memory.  ... 
arXiv:2209.09726v1 fatcat:5ozspjvmsnaeln4ljveu3qhr2i

pLSM: A Highly Efficient LSM-Tree Index Supporting Real-Time Big Data Analysis

Jin Wang, Yong Zhang, Yang Gao, Chunxiao Xing
2013 2013 IEEE 37th Annual Computer Software and Applications Conference  
We designed a new plug-in system PuntStore with pLSM (Punt Log Structured Merge Tree) index engine. To improve read performance, Cache Oblivious Look-ahead Array (COLA) is adopted in our design.  ...  NoSQL databases are provided to deal with the new challenges brought by Big Data because of its high performance, storage, scalability and availability.  ...  Figure 1 .Figure 2 . 12 A general structure of LSM-Tree An example of Cache Oblivious Look-ahead Array Figure 3 . 3 The Insert Algorithm for COLA Bloom Filter is a random data structure with high space  ... 
doi:10.1109/compsac.2013.40 dblp:conf/compsac/WangZGX13 fatcat:cebzjsca4nc6fpfnv37qsrelb4

LSbM-tree: Re-Enabling Buffer Caching in Data Management for Mixed Reads and Writes

Dejun Teng, Lei Guo, Rubao Lee, Feng Chen, Siyuan Ma, Yanfeng Zhang, Xiaodong Zhang
2017 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS)  
In order to re-enable high-speed buffer caching during intensive writes, we propose Log-Structured buffered-Merge tree (simplified as LSbM-tree) by adding a compaction buffer on disks, to minimize the  ...  With the help of a small on-disk compaction buffer, LSbM achieves a high query performance by enabling effective buffer caching, while retaining all the merits of LSM-tree for write-intensive data processing  ...  We present an effective and low cost variant of LSM-tree to accomplish our goal. LSM-tree [1] was originally designed for high throughput transaction systems.  ... 
doi:10.1109/icdcs.2017.70 dblp:conf/icdcs/TengGLCMZZ17 fatcat:d66qnmusq5bdvfs4ryyg4f53r4

SpanDB: A Fast, Cost-Effective LSM-tree Based KV Store on Hybrid Storage

Hao Chen, Chaoyi Ruan, Cheng Li, Xiaosong Ma, Yinlong Xu
2021 USENIX Conference on File and Storage Technologies  
As a compromise, we propose SpanDB, an LSM-tree-based KV store that adapts the popular RocksDB system to utilize selective deployment of high-speed SSDs.  ...  To better utilize this fast disk, SpanDB provides high-speed, parallel WAL writes via SPDK, and enables asynchronous request processing to mitigate inter-thread synchronization overhead and work efficiently  ...  Noh for helpful discussions during his visit to QCRI. We also thank Sen Zheng of Zhongjia IT, for his valuable technical support during the COVID-19 lockdown.  ... 
dblp:conf/fast/ChenRLMX21 fatcat:qu2lmqp56fcgfliutfayre3oia

HashKV: Enabling Efficient Updates in KV Storage via Hashing

Helen H. W. Chan, Yongkun Li, Patrick P. C. Lee, Yinlong Xu
2018 USENIX Annual Technical Conference  
Persistent key-value (KV) stores mostly build on the Log-Structured Merge (LSM) tree for high write performance, yet the LSM-tree suffers from the inherently high I/O amplification.  ...  We propose HashKV, which aims for high update performance atop KV separation under updateintensive workloads.  ...  It can also return the KV pairs from the write cache for reads. If the write cache is full, HashKV flushes all the cached KV pairs to the LSM-tree and the value store.  ... 
dblp:conf/usenix/ChanLLX18 fatcat:wkoa6sgperhpxcvsugw6xuxtzy

Efficient LSM-Tree Key-Value Data Management on Hybrid SSD/HDD Zoned Storage [article]

Jinhong Li and Qiuping Wang and Patrick P. C. Lee
2022 arXiv   pre-print
We make a case for key-value (KV) stores based on log-structured merge trees (LSM-trees) as host-level applications, and present HHZS, a middleware system that bridges an LSM-tree KV store with hybrid  ...  HHZS leverages hints issued by the flushing, compaction, and caching operations of the LSM-tree KV store to manage KV objects in placement, migration, and caching in hybrid ZNS SSD and HM-SMR HDD zoned  ...  When HHZS re- For example, referring to Figure 4 , suppose that the LSM-tree KV store issues a read to data block b1 with the SST ID 4 and offset offset4 ( 3 ), while b1 is not cached by the in-memory  ... 
arXiv:2205.11753v1 fatcat:fefuu3zh3jgt3d46feo4vqffda

IndexFS: Scaling File System Metadata Performance with Stateless Caching and Bulk Insertion

Kai Ren, Qing Zheng, Swapnil Patil, Garth Gibson
2014 SC14: International Conference for High Performance Computing, Networking, Storage and Analysis  
We also propose two client-based stormfree caching techniques: bulk namespace insertion for creation intensive workloads such as N-N checkpointing; and stateless consistent metadata caching for hot spot  ...  In this paper, we introduce a middleware design called IndexFS that adds support to existing file systems such as PVFS, Lustre, and HDFS for scalable high-performance operations on metadata and small files  ...  We especially thank Los Alamos National Laboratory for running our software on one of their HPC clusters (Smog), Panasas for providing a storage cluster and LinkedIn for giving us a trace of its HDFS metadata  ... 
doi:10.1109/sc.2014.25 dblp:conf/sc/RenZPG14 fatcat:g443hulj4jhjtjzpaqumutjmia

Enabling Efficient Updates in KV Storage via Hashing: Design and Performance Evaluation [article]

Yongkun Li, Helen H. W. Chan, Patrick P. C. Lee, Yinlong Xu
2019 arXiv   pre-print
Persistent key-value (KV) stores mostly build on the Log-Structured Merge (LSM) tree for high write performance, yet the LSM-tree suffers from the inherently high I/O amplification.  ...  We propose HashKV, which aims for high update performance atop KV separation under update-intensive workloads.  ...  For example, under 20% of reserved space, the throughput of HashKV increases by 29.1% and the total write size reduces by 16.3% when the write cache size increases from 4 MiB to 64 MiB.  ... 
arXiv:1811.10000v2 fatcat:fmdmbxmojfcr7eoqbydcu4afha

SLSM - A Scalable Log Structured Merge Tree with Bloom Filters for Low Latency Analytics

P.A. Riyaz, Surekha Mariam Varghese
2016 Procedia Technology - Elsevier  
HBase is a NoSQL database suitable for random, real-time read/write access to Big Data. LSM tree used in HBase helps to achieve this high performance. Commodity hardware have moderate RAM size.  ...  SLSM, an optimized Log Structured Merge Tree which dramatically reduces the read amplification and write amplification is proposed in this paper for commodity hardware.  ...  Related Works LSM Trees Many of the NoSQL database uses LSM trees as a data structure for high performance.  ... 
doi:10.1016/j.protcy.2016.05.075 fatcat:szkfkhh35jeydj47i4iek22rxy

Closing the B-tree vs. LSM-tree Write Amplification Gap on Modern Storage Hardware with Built-in Transparent Compression [article]

Yifan Qiao, Xubin Chen, Ning Zheng, Jiangpeng Li, Yang Liu, Tong Zhang
2021 arXiv   pre-print
Recent years have witnessed significant interest in applying log-structured merge tree (LSM-tree) as an alternative to B-tree.  ...  The current consensus is that, compared with B-tree, LSM-tree has distinct advantages in terms of storage space efficiency and write amplification.  ...  INTRODUCTION This paper presents a B-tree design solution optimized for a growing family of commercial data storage hardware that internally carry out high-speed hardware-based lossless data compression  ... 
arXiv:2107.13987v1 fatcat:yux6dmpsena3tce5xrvwlcdgby

Efficient Data Ingestion and Query Processing for LSM-Based Storage Systems [article]

Chen Luo, Michael J. Carey
2019 arXiv   pre-print
In recent years, the Log Structured Merge (LSM) tree has been widely adopted by NoSQL and NewSQL systems for its superior write performance.  ...  Despite its popularity, however, most existing work has focused on LSM-based key-value stores with only a primary LSM-tree index; auxiliary structures, which are critical for supporting ad-hoc queries,  ...  A filter, such as a Bloom filter [13] or a range filter [11] on secondary keys, is directly built into LSM-trees to enable data skipping for faster scans.  ... 
arXiv:1808.08896v2 fatcat:l676rpnkmnhz3baq6x5j3wwilm

SplinterDB: Closing the Bandwidth Gap for NVMe Key-Value Stores

Alexander Conway, Abhishek Gupta, Vijay Chidambaram, Martin Farach-Colton, Richard P. Spillane, Amy Tai, Rob Johnson
2020 USENIX Annual Technical Conference  
The SplinterDB memtable and cache are designed to be highly concurrent and to reduce cache misses.  ...  This paper presents SplinterDB, a new keyvalue store explicitly designed for NVMe solid-state-drives.  ...  Acknowledgements We would like to thank Ittai Abraham for his insight and contribution to this project.  ... 
dblp:conf/usenix/ConwayGCFSTJ20 fatcat:jpjiwxgdqbecbcs4qjohzpzgw4

Leaper: A Learned Prefetcher for Cache Invalidation in LSM-tree based Storage Engines

Lei Yang, Hong Wu, Tieying Zhang, Xuntao Cheng, Feifei Li, Lei Zou, Yujie Wang, Rongyao Chen, Jianying Wang, Gui Huang
2020 Proceedings of the VLDB Endowment  
Frequency-based cache replacement policies that work well on page-based database storage engines are no longer sufficient for the emerging LSM-tree (Log-Structure Merge-tree) based storage engines.  ...  As a side-effect, such operations invalidate the corresponding entries in the cache for each involved record, causing sudden drops on the cache hit rates and spikes on access latency.  ...  Since data in LSM-tree based storage engines are commonly represented as key-value pairs for high performance, the query information (i.e., the structures and semantic information of query strings [23  ... 
dblp:journals/pvldb/YangWZCLZWCWH20 fatcat:yrlrnn27jbgzzp3m56n62afd3q


Russell Sears, Mark Callaghan, Eric Brewer
2008 Proceedings of the VLDB Endowment  
Rose 1 is a database storage engine for high-throughput replication.  ...  LSM-tree lookups have performance comparable to B-tree lookups. If Rose read each value that it updated then its write throughput would also be comparable to a B-tree.  ...  ACKNOWLEDGEMENTS We would like to thank Petros Maniatis, Tyson Condie, Jens Dittrich and the anonymous reviewers for their feedback. Portions of this work were performed at Intel Research, Berkeley.  ... 
doi:10.14778/1453856.1453914 fatcat:phivdc64ifd2ngui7ptn6r4v2m
« Previous Showing results 1 — 15 out of 237 results