581 Hits in 8.8 sec

Efficient Data Ingestion and Query Processing for LSM-Based Storage Systems [article]

Chen Luo, Michael J. Carey
2019 arXiv   pre-print
We then present several new and efficient maintenance strategies for LSM-based storage systems.  ...  Despite its popularity, however, most existing work has focused on LSM-based key-value stores with only a primary LSM-tree index; auxiliary structures, which are critical for supporting ad-hoc queries,  ...  This work is supported by NSF awards CNS-1305430, IIS-1447720, and IIS-1838248 along with industrial support from Amazon, Google, and Microsoft and support from the Donald Bren Foundation (via a Bren Chair  ... 
arXiv:1808.08896v2 fatcat:l676rpnkmnhz3baq6x5j3wwilm

Efficient data ingestion and query processing for LSM-based storage systems

Chen Luo, Michael J. Carey
2019 Proceedings of the VLDB Endowment  
We then present several new and efficient maintenance strategies for LSM-based storage systems.  ...  In this paper, we focus on efficient data ingestion and query processing for generalpurpose LSM-based storage systems.  ...  This work is supported by NSF awards CNS-1305430, IIS-1447720, and IIS-1838248 along with industrial support from Amazon, Google, and Microsoft and support from the Donald Bren Foundation (via a Bren Chair  ... 
doi:10.14778/3303753.3303759 fatcat:qv4s5o7rsnfqxikvwrtgf7nw6y

Tree indexing on solid state drives

Yinan Li, Bingsheng He, Robin Jun Yang, Qiong Luo, Ke Yi
2010 Proceedings of the VLDB Endowment  
To address this asymmetry of read-write speeds in tree indexing on the flash disk, we propose FD-tree, a tree index designed with the logarithmic method and fractional cascading techniques.  ...  With the logarithmic method, an FD-tree consists of the head tree -a small B+-tree on the top, and a few levels of sorted runs of increasing sizes at the bottom.  ...  [14] used a similar idea to design a stepped tree index and the hash index for data warehouses. Our FD-tree follows the idea of logarithmic method.  ... 
doi:10.14778/1920841.1920990 fatcat:3r66ggvttfcqbdqwbqslxuqxg4

A practical concurrent index for solid-state drives

Risi Thonangi, Shivnath Babu, Jun Yang
2012 Proceedings of the 21st ACM international conference on Information and knowledge management - CIKM '12  
We design, implement, and evaluate an index structure called FD+tree and an associated concurrency control scheme called FD+FC.  ...  Good concurrency control is especially critical to indexes on solidstate drives, because they typically rely on batch updates, which may take long and block concurrent index accesses.  ...  FD+DS as index size varies, for a GR workload with W l = 0 Figure 9 : 9 (a) Benefits of memory sharing between L new 0 and L old 0 . (b, c) Performance of FD+FC vs.  ... 
doi:10.1145/2396761.2398437 dblp:conf/cikm/ThonangiB012 fatcat:q6habe2h6bhtpmnprg62t6rdsa

LSM-based Storage Techniques: A Survey [article]

Chen Luo, Michael J. Carey
2019 arXiv   pre-print
In this paper, we provide a survey of recent research efforts on LSM-trees so that readers can learn the state-of-the-art in LSM-based storage techniques.  ...  We further survey several representative LSM-based open-source NoSQL systems and discuss some potential future research directions resulting from the survey.  ...  This work was supported by NSF awards CNS-1305430, IIS-1447720, and IIS-1838248 along with industrial support from Amazon, Google, and Microsoft and support from The Donald Bren Foundation (via a Bren  ... 
arXiv:1812.07527v2 fatcat:cr6ancmllfdwrkxbowqtxy27gm

Index maintenance for time-travel text search

Avishek Anand, Srikanta Bedathur, Klaus Berberich, Ralf Schenkel
2012 Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval - SIGIR '12  
In this work, we describe a novel index structure that efficiently supports time-travel text search and can be maintained incrementally as new document versions are added to the web archive.  ...  Different index structures have been proposed to efficiently support time-travel text search.  ...  Another benefit with an append operation is the avoidance of the expensive decompression and compression cycles in building the new updated index.  ... 
doi:10.1145/2348283.2348318 dblp:conf/sigir/AnandBBS12 fatcat:ed4q4tgtmvhzzh3vz3eetzdgye

Efficient, Modular Metadata Management with Loris

Richard van Heuven van Staereling, Raja Appuswamy, David C. van Moolenbroek, Andrew S. Tanenbaum
2011 2011 IEEE Sixth International Conference on Networking, Architecture, and Storage  
1) LSM-tree-based metadata storage, 2) an indexing infrastructure that uses LSMtrees for maintaining real-time attribute indices, and 3) scalable metadata querying using an attribute-based query language  ...  Such systems provide domainspecific storage formats for performance-optimized metadata storage, search-based access interfaces for enabling declarative queries, and type-specific indexing structures for  ...  In doing so, we present our Lorisbased metadata management system that provides 1) LSMtree-based metadata storage, 2) an indexing infrastructure that uses LSM-trees for maintaining real-time attribute  ... 
doi:10.1109/nas.2011.32 dblp:conf/nas/StaerelingAMT11 fatcat:ofxqgic7r5fpflfhc5e3aurelm


Jihang Liu, Shimin Chen, Lujun Wang
2020 Proceedings of the VLDB Endowment  
Moreover, we study the benefits of LB + -Tree in two real-world systems: X-Engine, a commercial OLTP storage engine, and Memcached, an open source key-value store.  ...  obtaining similar search and deletion performance.  ...  X-Engine Experiments X-Engine is a storage engine for supporting OLTP transactions on Alibaba's backbone e-commerce platform [12] . It implements an LSM-tree design with a tiered storage.  ... 
doi:10.14778/3384345.3384355 fatcat:pq5gbuoukjfdvpwzgy3dbhzuhu

Parallel Index-based Stream Join on a Multicore CPU [article]

Amirhesam Shahvarani, Hans-Arno Jacobsen
2019 arXiv   pre-print
To complement the index, we design an algorithm to realize a parallel index-based stream join that exploits the computational power of multicore processors.  ...  Our experiments using an octa-core processor show that our parallel stream join achieves up to 5.5 times higher throughput than a single-threaded approach.  ...  Components in LSM-Tree are configured to be used in different storage media and LSM-Tree applies delta update to alleviate the cost of write operations in low bandwidth storage media.  ... 
arXiv:1903.00452v1 fatcat:xqf7zbvkeffo5bg277anze3roa


Jingtian Zhang, Sai Wu, Zeyuan Tan, Gang Chen, Zhushi Cheng, Wei Cao, Yusong Gao, Xiaojie Feng
2019 Proceedings of the VLDB Endowment  
However, those new indexes cannot be easily intergrated with key-value systems, because most of them do not consider how the data can be efficiently flushed to disk.  ...  Many new memory indexing structures have been proposed and outperform current in-memory skip-list index adopted by LevelDB, RocksDB and other key-value systems.  ...  The authors would like to thank the anonymous reviewers for their valuable suggestions and opinions.  ... 
doi:10.14778/3352063.3352134 fatcat:ogm3tagvwzfoxbamk5thfxmkoq

Key-Value Stores on Flash Storage Devices: A Survey [article]

Krijn Doekemeijer, Animesh Trivedi
2022 arXiv   pre-print
With the increasing need for timely data analysis, performance becomes more and more critical. In the past, these stores were frequently optimised to run on HDD and DRAM devices.  ...  Key-value stores (KV) have become one of the main components of the modern storage and data processing system stack.  ...  These immutable tables should eventually be written to storage. At the same time, a new mutable memtable is created or an old memtable is reused.  ... 
arXiv:2205.07975v1 fatcat:5sv2j2m3l5hrlnpjohlcxsh574

Reducing the Storage Overhead of Main-Memory OLTP Databases with Hybrid Indexes

Huanchen Zhang, David G. Andersen, Andrew Pavlo, Michael Kaminsky, Lin Ma, Rui Shen
2016 Proceedings of the 2016 International Conference on Management of Data - SIGMOD '16  
For a main-memory database, however, these indexes consume a large fraction of the total memory available and are thus a major source of storage overhead of in-memory databases.  ...  Our first contribution is hybrid index, a dual-stage index architecture that achieves both space efficiency and high performance.  ...  National Science Foundation under awards IIS-1409802, CNS-1345305, and CCF-1535821, as well as Intel via the Intel Science and Technology Center for Cloud Computing (ISTC-CC).  ... 
doi:10.1145/2882903.2915222 dblp:conf/sigmod/ZhangAPKMS16 fatcat:rh4qexnquvfolddctr3fqory44

NVKVS: Non-Volatile Memory Optimized Key-Value Separated LSM-Tree

Ryan Nathanael Soenjoto Widodo, Hiroki Ohtsuji, Erika Hayashi, Eiji Yoshida, Hirotake Abe, Kazuhiko Kato
2022 Journal of Information Processing  
LSM-tree-based key-value stores (KVSs) with KV separation minimizes the write latency and write amplification of LSM-trees by writing the KV-pairs sequentially into a log file and indexing these KV pairs  ...  in an LSM-tree.  ...  Acknowledgments The authors would like to thank editors, reviewers, and readers who have given their time and valuable comments.  ... 
doi:10.2197/ipsjjip.30.332 fatcat:zl7wg6aym5duvg4ff4kv562hfu


Russell Sears, Mark Callaghan, Eric Brewer
2008 Proceedings of the VLDB Endowment  
Also, LSM-trees cannot become fragmented and provide fast, predictable index scans. Rose's write performance relies on replicas' ability to perform writes without looking up old values.  ...  It increases replication throughput by reducing sequential I/O, and enables efficient tree lookups by supporting small page sizes and doubling as an index of the values it stores.  ...  ACKNOWLEDGEMENTS We would like to thank Petros Maniatis, Tyson Condie, Jens Dittrich and the anonymous reviewers for their feedback. Portions of this work were performed at Intel Research, Berkeley.  ... 
doi:10.14778/1453856.1453914 fatcat:phivdc64ifd2ngui7ptn6r4v2m

Online Adaptation and Energy Minimization for Hardware Recurrent Spiking Neural Networks

Yu Liu, Yingyezhe Jin, Peng Li
2018 ACM Journal on Emerging Technologies in Computing Systems  
dissipation by up to 29% compared to a baseline LSM with little extra hardware overhead on a Xilinx Virtex-6 FPGA.  ...  In this article, we systematically present three techniques for optimizing energy efficiency while maintaining good performance of the proposed LSM neural processors from both an algorithmic and hardware  ...  This LUT is indexed by Δt and w d old , where the spike timing difference Δt has been already discretized by the properly chosen emulation timestep.  ... 
doi:10.1145/3145479 fatcat:uinhhedbkbhlzedgsqiebcoqrq
« Previous Showing results 1 — 15 out of 581 results