774 Hits in 3.8 sec

Performance and scalability evaluation of the Ceph parallel file system

Feiyi Wang, Mark Nelson, Sarp Oral, Scott Atchley, Sage Weil, Bradley W. Settlemyer, Blake Caldwell, Jason Hill
2013 Proceedings of the 8th Parallel Data Storage Workshop on - PDSW '13  
Ceph is an emerging open-source parallel distributed file and storage system.  ...  This paper presents our file and block I/O performance and scalability evaluation of Ceph for scientific high-performance computing (HPC) environments. Our work makes two unique contributions.  ...  BACKGROUND Ceph [5] is a distributed storage system designed for scalability, reliability, and performance.  ... 
doi:10.1145/2538542.2538562 dblp:conf/sc/WangNOAWSCH13 fatcat:36xaoflsencilpodqls5ak3m7e

Current Status of the Ceph Based Storage Systems at the RACF

A. Zaytsev, H. Ito, C. Hollowell, T. Wong, T. Rao
2015 Journal of Physics, Conference Series  
This contribution contains a thorough review of various functionality, performance and stability tests performed with all three (object storage, block storage and file system) levels of Ceph by using the  ...  With the current status of Ceph project, both object storage and block storage (RBD) layers are production ready on a large scale, and the Ceph file system storage layer (CephFS) is rapidly getting to  ...  Finally, the Ceph file system (CephFS) [6] allows us to study the use case of a resilient and scalable distributed file system for possible applications in our HEP specific data storage environment.  ... 
doi:10.1088/1742-6596/664/4/042027 fatcat:gbkjkydgo5crzhsvghcv7ycwbq

Building an organic block storage service at CERN with Ceph

Daniel van der Ster, Arne Wiebalck
2014 Journal of Physics, Conference Series  
Beyond the object storage use-cases, Ceph implements a block storage service, RBD, and a general file system, CephFS.  ...  A high performance tape buffer for CASTOR CASTOR requires a high performance tape buffer in order to supply the O(100) tape drives with data at 250MB/s each.  ... 
doi:10.1088/1742-6596/513/4/042047 fatcat:ukx2y2wbvfa33lpabcfphsmxeq

Supporting SLA via Adaptive Mapping and Heterogeneous Storage Devices in Ceph

Sopanhapich Chum, Heekwon Park, Jongmoo Choi
2021 Electronics  
This paper proposes a new resource management scheme that supports SLA (Service-Level Agreement) in a bigdata distributed storage system.  ...  It examines two use cases and observes that separating data and metadata into different devices gives a positive impact on the performance per cost ratio.  ...  Ceph Architecture Ceph is a scalable, reliable, and open-source distributed storage solution [14, 31] .  ... 
doi:10.3390/electronics10070847 fatcat:pandgnaq3vg2lohaye3pv3rto4

Impact of Single Parameter Changes on Ceph Cloud Storage Performance

Stefan Meyer, John P. Morrison
2016 Scalable Computing : Practice and Experience  
This paper uses the Ceph distributed file system, and in particular its global parameters, to show how a single changed parameter can effect the performance for a range of access patterns when tested with  ...  In a general purpose cloud system efficiencies are yet to be had from supporting diverse applications and their requirements within a storage system used for a private cloud.  ...  The back-ends for Cinder cover a great variety of systems [4] , including proprietary storage systems such as Dell EqualLogic or EMC VNX Direct, distributed file systems, such as Ceph or GlusterFS, and  ... 
doi:10.12694/scpe.v17i4.1201 fatcat:r7idxywsuveo3not6rna7rcaue

Formalization and Analysis of Ceph Using Process Algebra

Ran LI, Huibiao ZHU, Jiaqi YIN
2021 IEICE transactions on information and systems  
Ceph is an object-based parallel distributed file system that provides excellent performance, reliability, and scalability.  ...  Additionally, Ceph provides its Cephx authentication system to authenticate users, so that it can identify users and realize authentication.  ...  Among them, Ceph is an emerging open-source distributed file system. Excellent performance, reliability and scalability make Ceph stand out from numerous distributed file systems [5] .  ... 
doi:10.1587/transinf.2021edp7070 fatcat:apw6yjd65rdbpdivja2kgbvxuq

Testing of several distributed file-systems (HDFS, Ceph and GlusterFS) for supporting the HEP experiments analysis

Giacinto Donvito, Giovanni Marzulli, Domenico Diacono
2014 Journal of Physics, Conference Series  
Using Hadoop File System and MapReduce in a small/medium Grid site H Riahi, G Donvito, L Fanò et al.  ...  In particular this work will cover storage solutions that provide both standard POSIX storage access and cloud technologies; we focused our attention and our test on HDFS, Ceph, and GlusterFS.  ...  Ceph is a quite young file-system that has been designed in order to guarantee great scalability, performance and very good high availability features.  ... 
doi:10.1088/1742-6596/513/4/042014 fatcat:lso7wehosbcpli7d5jcwha2y24

Understanding the Performance of Ceph Block Storage for Hyper-Converged Cloud with All Flash Storage [article]

Moo-Ryong Ra
2018 arXiv   pre-print
The problem becomes more onerous if we consider a complex distributed system, such as Ceph, for the cloud with the proliferation of SSD drives.  ...  In this paper, we aim to answer some of these questions based on comprehensive microbenchmarks, and consequently better understand the behavior of Ceph in a hyper-converged cloud with all-flash storage  ...  The base layer of Ceph is an object store (RADOS). Block (RBD) and file system (CephFS) APIs are built on top of the RADOS layer.  ... 
arXiv:1802.08102v1 fatcat:q52g4ka3z5gipctn42lx4oiayq

NetFlow Monitoring and Cyberattack Detection Using Deep Learning with Ceph

Chao-Tung Yang, Jung-Chun Liu, Endah Kristiani, Ming-Lun Liu, Ilsun You, Giovanni Pau
2020 IEEE Access  
Ceph is used as an open-source distributed storage platform that offers high efficiency, high reliability, scalability, and preliminary preprocessing of raw data with Python, removing redundant areas and  ...  This paper presents a detailed architecture in which the collected log data of the network can be processed and analyzed.  ...  To store these materials, it is critical to creating a high-performance, reliable, and scalable storage environment [3] .  ... 
doi:10.1109/access.2019.2963716 fatcat:ap4qg6zodrbo5lnqdqmuugrkyy

Leveraging OpenStack and Ceph for a Controlled-Access Data Cloud

Evan F. Bollig, Graham T. Allan, Benjamin J. Lynch, Yectli A. Huerta, Mathew Mix, Edward A. Munsell, Raychel M. Benson, Brent Swartz
2018 Proceedings of the Practice and Experience on Advanced Research Computing - PEARC '18  
Many also seek safe harbors to operate on or store sensitive and/or controlled-access data in a high capacity environment.  ...  While traditional HPC has and continues to satisfy most workflows, a new generation of researchers has emerged looking for sophisticated, scalable, on-demand, and self-service control of compute infrastructure  ...  high performance computing (HPC) systems.  ... 
doi:10.1145/3219104.3219165 dblp:conf/xsede/BolligALHMMBS18 fatcat:ltd5lktmvfcstpbxkfx2ercpvu

Modeling Replication and Erasure Coding in Large Scale Distributed Storage Systems Based on CEPH [chapter]

Daniele Manini, Marco Gribaudo, Mauro Iacono
2016 Lecture Notes in Information Systems and Organisation  
and erasure coding in large scale distributed storage systems based on CEPH Abstract.  ...  Today's most of high performance computing applications use parallel programming paradigms to reach the desired efficiency objectives.  ...  Introduction High performance computing applications usually rely on parallel execution to divide a large problem in small elements that can be considered concurrently.  ... 
doi:10.1007/978-3-319-40265-9_20 fatcat:653nbbverbd3zjnfjlg7i42zwm

High-Performance Storage Support for Scientific Applications on the Cloud

Dongfang Zhao, Xu Yang, Iman Sadooghi, Gabriele Garzoglio, Steven Timm, Ioan Raicu
2015 Proceedings of the 6th Workshop on Scientific Cloud Computing - ScienceCloud '15  
We analyze and evaluate four representative file systems (S3FS, HDFS, Ceph, and FusionFS) on three platforms (Kodiak cluster, Amazon EC2 and FermiCloud) with a variety of benchmarks to explore how well  ...  We believe that when executing scientific applications in the cloud, a node-local distributed storage architecture is a key approach to overcome the challenges from the conventional shared/parallel storage  ...  The scalable throughput of Ceph is attributed by its distributed metadata and data movement. The metadata in Ceph is organized in such a way that high availability and scalability are guaranteed.  ... 
doi:10.1145/2755644.2755648 dblp:conf/hpdc/ZhaoYSGTR15 fatcat:3h2ay4o6ajd4rjtzyzckollfwe

Distributed Metadata Management of Mass Storage System in High Energy Physics

Qiulan Huang, Ran Du, YaoDong Cheng, Jingyan Shi, Gang Chen, Wenxiao Kan
2017 Journal of Physics, Conference Series  
In this contribution, we design and implement a dynamic and scalable distributed metadata management system(StarFS) to High Energy Physics (HEP) mass storage system.  ...  A system with DULA could provide uniform data distribution and efficient data positioning.  ...  Metadata performance would produce a big effect on the scalability, availability and high performance of the massive storage system.  ... 
doi:10.1088/1742-6596/898/6/062003 fatcat:kkiinu26mzggtfqyd3d5gg4y2q

Cider: a Rapid Docker Container Deployment System through Sharing Network Storage

Lian Du, Tianyu Wo, Renyu Yang, Chunming Hu
2017 2017 IEEE 19th International Conference on High Performance Computing and Communications; IEEE 15th International Conference on Smart City; IEEE 3rd International Conference on Data Science and Systems (HPCC/SmartCity/DSS)  
To this end, we propose Cider -a novel deployment system to enable rapid container deployment in a high concurrent and scalable manner at scale.  ...  In fact, it is observable that the image pulling operation is mainly responsible for the degraded performance.  ...  Using Ceph As the Underlying Storage Ceph is a scalable, high-performance distributed storage system [14] and is widely-used in cloud computing.  ... 
doi:10.1109/hpcc-smartcity-dss.2017.44 dblp:conf/hpcc/DuWYH17 fatcat:i6i6gkelrbdi3n3u76555mdela

Performance Evaluations of Distributed File Systems for Scientific Big Data in FUSE Environment

Jun-Yeong Lee, Moon-Hyun Kim, Syed Asif Raza Raza Shah, Sang-Un Ahn, Heejun Yoon, Seo-Young Noh
2021 Electronics  
In this study, we investigated and benchmarked various distributed file systems, such as Ceph, GlusterFS, Lustre and EOS for data-intensive environments.  ...  In our experiment, we configured the distributed file systems under a Reliable Array of Independent Nodes (RAIN) structure and a Filesystem in Userspace (FUSE) environment.  ...  The distributed file system provides horizontal scalability compared to RAID, which uses vertical scalability.  ... 
doi:10.3390/electronics10121471 fatcat:bpukewy7ircc5nqrskn64vygga
« Previous Showing results 1 — 15 out of 774 results