Filters








14,388 Hits in 6.4 sec

Design of Object Storage Using OpenNVM for High-performance Distributed File System

Fuyumasa Takatsu, Kohei Hiraga, Osamu Tatebe
2016 Journal of Information Processing  
The current trend for high-performance distributed file systems is object-based architecture that uses local object storage to store the file data.  ...  The IO performance of such systems depends on the local object storage that manages the underlying low-level storage, such as Fusion IO ioDrive, a flash device connected through PCI express.  ...  Acknowledgments This work is supported by JST CREST, "System Software for Post Petascale Data Intensive Science" and "EBD: Extreme Big Data -Convergence of Big Data and HPC for Yottabyte Processing".  ... 
doi:10.2197/ipsjjip.24.824 fatcat:5oevpt7agrd6nlhs5ovoolymce

PPFS: A Scale-out Distributed File System for Post-petascale Systems

Fuyumasa Takatsu, Kohei Hiraga, Osamu Tatebe
2017 Journal of Information Processing  
The fusion of the research field of high-performance computing (HPC) with that of big data, which has become known as the field of extreme big data, is problematic in that file creation in storage systems  ...  such as distributed file systems is not optimized.  ...  Acknowledgments This work is supported by JST CREST, "System Software for Post Petascale Data Intensive Science", "EBD: Extreme Big Data-Convergence of Big Data and HPC for Yottabyte Processing" and "Statistical  ... 
doi:10.2197/ipsjjip.25.438 fatcat:verikdn7c5a73asuql4fjujsf4

Online monitoring and visualisation of database structural deterioration

Takashi Hoshino, Kazuo Goda, Masaru Kitsuregawa
2010 International Journal of Autonomic Computing  
Database reorganisation, which removes structural deterioration by relocating data in the secondary storage, is an administrator's headache; it is difficult to schedule reorganisation efficiently to keep  ...  deterioration distribution, with a novel structural deterioration model, a storage performance model and an incremental update technique.  ...  Acknowledgements This work has been supported in part by Leading Project 'e-Society' Advance Storage and Next-generation IT Program Development of Out-of-Order Database Engine, which both are joint research  ... 
doi:10.1504/ijac.2010.033011 fatcat:ikn2ogbjljg2fnznolxu6j5ms4

MDupl: A Replica Strategy of Cloud Storage system

Hongtao Yu, Sijie Liu, Zehua Fan
2021 Procedia Computer Science  
This paper proposes a Qos-oriented replica strategy named MDupl which can confirm data replica number and distribution in cloud storage system.  ...  This paper proposes a Qos-oriented replica strategy named MDupl which can confirm data replica number and distribution in cloud storage system.  ...  An IO request packet corresponds to one time of access operation in storage system, Table 2 . 2 Data categories of digital resource platform Digital content Data size File quantity Resources type  ... 
doi:10.1016/j.procs.2021.05.047 fatcat:2djtoynrezcihoizcrvzsluvsa

Blockchain Towards Prioritization-Based Distributed Storage of Big Data for Internet of Vehicles

2022 Advances in Machine Learning & Artificial Intelligence  
Designing a scalable, high-performance big data distributed storage system for IoV, an advanced data-processing system for car services.  ...  This paper proposes a prioritization-based distributed storage of big data processing application in Internet of Vehicle (IoV) system.  ...  As a distributed key-value storage, HBase has the scalability with data volume. Memcached and Redis are in-memory caching platforms. They provide high-performance key-value storage.  ... 
doi:10.33140/amlai.03.01.04 fatcat:yjg7wkzt2jegfctvocnbpfnnzy

A Strategy for Improving the Performance of Small Files in Openstack Swift

Xiaoli Zhang, Chengyu Wen, Zizhen Yuan
2018 International Journal of Computer Applications Technology and Research  
This is an effective way to improve the storage access performance of small files in Openstack Swift by adding an aggregate storage module.  ...  During the short encounter time, the object-to-volume mapping information is stored in Key-Value store at the second stage.  ...  Finally, storage nodes register objects to the KV server, which is to add new entries to the key-value store.  ... 
doi:10.7753/ijcatr0708.1006 fatcat:t4s3g3ml7jegxd54wfdgnmswxi

Exploring the Behavior of Coherent Accelerator Processor Interface (CAPI) on IBM Power8+ Architecture and FlashSystem 900 [article]

Kaushik Velusamy, Smriti Prathapan, Milton Halem
2019 arXiv   pre-print
This library provides the application, a direct access to the underlying flash storage through user space APIs, to manage and access the data in flash.  ...  This removes the overhead and complexity of the IO subsystem and allows the accelerator to operate as part of an application.  ...  Acknowledgement We would like to thank Mike Vageline of IBM Cognitive Systems and Software Development for his support on the CAPIFlash -The IBM Data Engine for NoSQL library.  ... 
arXiv:1909.07166v1 fatcat:evjkrajbfneqvmi34d3fs6h6ny

Mapping Datasets to Object Storage System

Xiaowei Aaron Chu, Jeff LeFevre, Aldrin Montana, Dana Robinson, Quincey Koziol, Peter Alvaro, Carlos Maltzahn, C. Doglioni, D. Kim, G.A. Stewart, L. Silvestris, P. Jackson (+1 others)
2020 EPJ Web of Conferences  
For example, storage servers might include local key/value stores combined with chunk stores that require different optimizations than a local file system.  ...  of access libraries and storage systems and 3) fully leveraging of the existing load balancing, elasticity, and failure management of distributed storage systems like Ceph.  ...  structure of the data or take an active role in optimizing data access.  ... 
doi:10.1051/epjconf/202024504037 fatcat:gtjjgsoknbgttg53o5mnbsqlsi

Mapping Datasets to Object Storage System [article]

Xiaowei Chu, Jeff LeFevre, Aldrin Montana, Dana Robinson, Quincey Koziol, Peter Alvaro, Carlos Maltzahn
2020 arXiv   pre-print
For example, storage servers might include local key/value stores combined with chunk stores that require different optimizations than a local file system.  ...  of access libraries and storage systems and 3) fully leveraging of the existing load balancing, elasticity, and failure management of distributed storage systems like Ceph.  ...  structure of the data or take an active role in optimizing data access.  ... 
arXiv:2007.01789v1 fatcat:5rpiypf27jbanextsiscbnn6zm

Pattern-Direct and Layout-Aware Replication Scheme for Parallel I/O Systems

Yanlong Yin, Jibing Li, Jun He, Xian-He Sun, Rajeev Thakur
2013 2013 IEEE 27th International Symposium on Parallel and Distributed Processing  
Experimental results show that PDLA is effective in improving data access performance of parallel I/O systems.  ...  The basic idea of PDLA is replicating identified data access pattern, and saving these reorganized replications with optimized data layouts based on access cost analysis.  ...  Each record in the Berkeley DB hash table is a key-value pair; the key is the patternID and the value contains the data access pattern and the runtime information.  ... 
doi:10.1109/ipdps.2013.114 dblp:conf/ipps/YinLHST13 fatcat:xrkrzb2tdjd4dfwcf5edzwij2e

Next-Generation Information Technology Systems for Fast Detectors in Electron Microscopy [chapter]

Dieter Weber, Alexander Clausen, Rafal E. Dunin-Borkowski
2020 Handbook on Big Data and Machine Learning in the Physical Sciences  
Starting from 2009, the data rate of TEM cameras has outpaced the development of network, mass storage and memory bandwidth by almost two orders of magnitude.  ...  Similar developments have occurred for advanced X-ray sources such as the European XFEL, requiring special information technology (IT) systems for data handling (Sauter, Hattne, Grosse-Kunstleve, & Echols  ...  The use of raw binary data blocks that can be distributed across a distributed storage solution and accessed with the full range of optimized file reading methods that the operating system provides on  ... 
doi:10.1142/9789811204579_0005 fatcat:gi4bt7vbovc2hfzzzlyj6n56pu

Dynamic load balancing algorithm for large data flow in distributed complex networks

Zhuo Zhang
2018 Open Physics  
At the same time, the algorithm can quickly find the sub-optimal nodes when the optimal nodes have been occupied, so it is very suitable for load balancing in highly concurrent systems.  ...  The most commonly used system to store and process large amounts of data is the NoSQL (Not only Structured Query Language) database.  ...  A key-value storage database is a database where data is stored in memory and disk as keys and values.  ... 
doi:10.1515/phys-2018-0089 fatcat:uze6lojkbfdxdj7hr3wolmyy5e

Rethinking Key-Value Store for Parallel I/O Optimization

Yanlong Yin, Antonios Kougkas, Kun Feng, Hassan Eslami, Yin Lu, Xian-He Sun, Rajeev Thakur, William Gropp
2014 2014 International Workshop on Data Intensive Scalable Computing Systems  
Key-Value Stores (KVStore) are being widely used as the storage system for large-scale Internet services and cloud storage systems.  ...  However, they are rarely used in HPC systems, where parallel file systems (PFS) are the dominant storage systems.  ...  This gives us an opportunity to use Key-Value Store to optimize the performance for some specific workloads. III.  ... 
doi:10.1109/discs.2014.11 dblp:conf/sc/YinKFELSTG14 fatcat:vue5xxtn45avfblxip3m5pgbxi

ONE: A Predictable and Scalable DW Model [chapter]

João Pedro Costa, José Cecílio, Pedro Martins, Pedro Furtado
2011 Lecture Notes in Computer Science  
The star schema model has been widely used as the facto DW storage organization on relational database management systems (RDBMS).  ...  In this paper we evaluate if the underlying premises of the star schema model storage organization still upholds.  ...  Query optimizers have to evaluate and assess which combination and orchestration of access methods, joining algorithms and joining order in order to determine the query execution plan with minimum costs  ... 
doi:10.1007/978-3-642-23544-3_1 fatcat:f33zzllgozbxhg23wmvqf7gatu

Automatic Storage Structure Selection for hybrid Workload [article]

Hongzhi Wang, Yan Wei, Hao Yan
2020 arXiv   pre-print
In the system, we introduce a machine learning method to build a cost model for the storage engine, and a column-oriented data layout generation algorithm.  ...  Motivated by this, we propose an automatic storage structure selection system based on learning cost, which is used to dynamically select the optimal storage structure of the database under hybrid workloads  ...  When considering the table schema, we treat all fields in primary key as key fields, and others as value fields.  ... 
arXiv:2008.06640v1 fatcat:cwj7oy2svfb7re5cx5t4loaaqq
« Previous Showing results 1 — 15 out of 14,388 results