2,463 Hits in 4.0 sec

Data Chunks Placement Optimization for Hybrid Storage Systems

Agil Yolchuyev, Janos Levendovszky
2021 Future Internet  
In this paper, we propose a new framework for optimizing the data uploading process that is currently used with hybrid cloud storage systems.  ...  The optimization is concerned with spreading the data over the multiple storages in the HCS system according to some predefined objective functions.  ...  Heuristics for Finding an Optimal Data Chunk Placement In this section, we propose an algorithm for finding the optimal data chunk placement.  ... 
doi:10.3390/fi13070181 fatcat:w45gf6w6djfxhg6eohlykypnzq

VeloC: Towards High Performance Adaptive Asynchronous Checkpointing at Large Scale

Bogdan Nicolae, Adam Moody, Elsa Gonsiorowski, Kathryn Mohror, Franck Cappello
2019 2019 IEEE International Parallel and Distributed Processing Symposium (IPDPS)  
Global checkpointing to external storage (e.g., a parallel file system) is a common I/O pattern of many HPC applications.  ...  Extensive experiments on a pre-Exascale supercomputing system show significant benefits. Index Terms-parallel I/O; checkpoint-restart; immutable data; adaptive multilevel asynchronous I/O  ...  the Exascale Computing Project (ECP), Project Number: 17-SC-20-SC, a collaborative effort of two DOE organizations -the Office of Science and the National Nuclear Security Administration, responsible for  ... 
doi:10.1109/ipdps.2019.00099 dblp:conf/ipps/NicolaeMGMC19 fatcat:6anbo4rezvedleejttnowfmnpe

Even data placement for load balance in reliable distributed deduplication storage systems

Min Xu, Yunfeng Zhu, Patrick P. C. Lee, Yinlong Xu
2015 2015 IEEE 23rd International Symposium on Quality of Service (IWQoS)  
This paper studies the load balance problem in the setting of a reliable distributed deduplication storage system, which deploys deduplication for storage efficiency and erasure coding for reliability.  ...  To this end, we formulate a combinatorial optimization problem, and propose a greedy, polynomial-time Even Data Placement (EDP) algorithm, which identifies a data placement that effectively achieves read  ...  PROBLEM We formulate a combinatorial optimization problem that searches for a data placement policy that maximizes read balance, while preserving storage balance.  ... 
doi:10.1109/iwqos.2015.7404754 dblp:conf/iwqos/XuZLX15 fatcat:lxfankgerbb6fhzh2v6yotvhz4

Mosaic: A Budget-Conscious Storage Engine for Relational Database Systems

Lukas Vogel, Alexander van Renen, Satoshi Imamura, Viktor Leis, Thomas Neumann, Alfons Kemper
2020 Proceedings of the VLDB Endowment  
Our approach uses device models and linear optimization to find a data placement solution that maximizes I/O throughput for the workload.  ...  Relational database systems are purpose-built for a specific storage device class (e.g., HDD, SSD, or DRAM).  ...  We evaluate Mosaic on two data sets to show the advantage of Mosaic's column-granular data placement over existing approaches of RDBMS and big data query engines.  ... 
dblp:journals/pvldb/0001RIL0K20 fatcat:eol5dx6ka5f6nobhv4rpgv2mau

A cost-aware region-level data placement scheme for hybrid parallel I/O systems

Shuibing He, Xian-He Sun, Bo Feng, Xin Huang, Kun Feng
2013 2013 IEEE International Conference on Cluster Computing (CLUSTER)  
In this paper, we propose a cost-aware regionlevel (CARL) data placement scheme for hybrid parallel I/O systems.  ...  The performance of hybrid SSD-HDD architecture depends on the utilization of the SSD and scheduling of data placement.  ...  focused on the data placement policy in an SSD-based hybrid storage system [12] - [15] .  ... 
doi:10.1109/cluster.2013.6702615 dblp:conf/cluster/HeSFHF13 fatcat:2aggvfi3znbxtjnacve7w4x6xe

Cost-optimized redundant data storage in the cloud

Philipp Waibel, Johannes Matt, Christoph Hochreiner, Olena Skarlat, Ronny Hans, Stefan Schulte
2017 Service Oriented Computing and Applications  
The use of cloud-based storage systems for storing data is a popular alternative to local storage systems.  ...  Furthermore, we present a heuristic optimization approach for the global optimization.  ...  System model For the data object placement optimization, we provide a mixed-integer linear programming (MILP)-based local and a global data placement approach and a heuristic approach.  ... 
doi:10.1007/s11761-017-0218-9 fatcat:fb5qk7xs4farlat65ycdkw2wcm

Scalia: An adaptive scheme for efficient multi-cloud storage

Thanasis G. Papaioannou, Nicolas Bonvin, Karl Aberer
2012 2012 International Conference for High Performance Computing, Networking, Storage and Analysis  
A growing amount of data is produced daily resulting in a growing demand for storage solutions.  ...  In this paper 1 , we introduce Scalia, a cloud storage brokerage solution that continuously adapts the placement of data based on its access pattern and subject to optimization objectives, such as storage  ...  In this paper, we introduce Scalia, a system that continuously adapts the placement of data among several storage providers subject to optimization objectives, such cost minimization.  ... 
doi:10.1109/sc.2012.101 dblp:conf/sc/PapaioannouBA12 fatcat:hekn3cqrzrajncvj5afye6lbiq

Healthcare Cloud Integration using Distributed Cloud Storage and Hybrid Image Compression

Sherif E.Hussein, Sherif M. Badr
2013 International Journal of Computer Applications  
Today, clouds are mainly used for handling highly intensive computing workloads and for providing very large data storage facilities.  ...  Therefore, the theory of data compression became more significant for reducing data redundancy to save more hardware space and transmission bandwidth.  ...  The engine also tries to maintain the optimality of the chunk placement of an object, by periodically computing the best provider set using the data access statistics.  ... 
doi:10.5120/13839-1268 fatcat:4tud3s3fbredlckms6aiiyalqi

Exploring heterogeneity of unreliable machines for P2P backup

Piotr Skowron, Krzysztof Rzadca
2013 2013 International Conference on High Performance Computing & Simulation (HPCS)  
In contrast to standard p2p storage systems using directly a DHT, the contracts allow our system to optimize replicas' placement depending on a specific optimization strategy, and so to take advantage  ...  We present the architecture of a p2p backup system that uses pairwise replication contracts between a data owner and a replicator.  ...  The authors thank Sonja Buchegger and Gunnar Kreitz for their helpful remarks.  ... 
doi:10.1109/hpcsim.2013.6641398 dblp:conf/ieeehpcs/SkowronR13 fatcat:nr6ia5nkjvcklk7neg63nm42jq

Flexible replica placement for optimized P2P backup on heterogeneous, unreliable machines

Piotr Skowron, Krzysztof Rzadca
2015 Concurrency and Computation  
Because every data chunk is associated with a list of storage contracts, each chunk can be placed at any location (the location depends on the placement strategy).  ...  Our main contribution is the following: (1) We present an architecture of a storage system that uses pairwise (bilateral) replication contracts for storing data and therefore enables to optimize placement  ...  In our system, each peer keeps information about replica placement of its data chunks in an index structure called DataCatalog.  ... 
doi:10.1002/cpe.3491 fatcat:2xpwma4udvgn7mc2xyxlyckhha

Exploit the known or explore the unknown?

Raffaele Chiocchetti, Dario Rossi, Giuseppe Rossini, Giovanna Carofiglio, Diego Perino
2012 Proceedings of the second edition of the ICN workshop on Information-centric networking - ICN '12  
Most Information Centric Networking designs propose the usage of widely distributed in-network storage.  ...  By means of packet-level simulations, we investigate the performance trade-offs of exploitation/exploration approaches, and introduce an hybrid solution.  ...  Acknowledgements This work was carried out at LINCS and funded by the ANR Project CONNECT Content-Oriented Networking: a New Experience for Content Transfer (CONNECT).  ... 
doi:10.1145/2342488.2342491 dblp:conf/acmicn/ChiocchettiRRCP12 fatcat:xohaj3ycq5h4tmc2imfzqlyxay

Survey on Caching and Replication Algorithm for Content Distribution in Peer to Peer Networks

C SriguruLakshmi, G Sivakumar, V Venkatachalam
2013 International Journal of Computer Science Engineering and Information Technology  
In this paper, we focuses on caching and replication algorithm for content distribution in peer to peer networks.  ...  Caching and replication is a key strategy for improve the reliability, availability, and performance in peer to peer networks.  ...  The objective is to design a hybrid peer-to-peer system for distributed data sharing. Consistency maintenance is propagating the updates from a primary file to its replica.  ... 
doi:10.5121/ijcseit.2013.3503 fatcat:mecdixx4hvbt3ajtnmeqpwtwui

Dynamic scaling of VoD services into hybrid clouds with cost minimization and QoS guarantee

Xuanjia Qiu, Hongxing Li, Chuan Wu, Zongpeng Li, Francis C.M. Lau
2012 2012 19th International Packet Video Workshop (PV)  
In this paper, we present a dynamic algorithm that optimally makes decisions on video replication and user request dispatching in a hybrid cloud of on-premise servers and geo-distributed cloud data centers  ...  , based on the Lyapunov optimization framework.  ...  uploading chunks to users from the on-premise servers and the cloud data centers, (2) the storage cost for replicated videos at the data centers, (3) the migration cost for copying videos from the on-premise  ... 
doi:10.1109/pv.2012.6229726 dblp:conf/pv/QiuLWLL12 fatcat:6imhypthe5hojanyxbnqc2uonu

Taming Tail Latency for Erasure-coded, Distributed Storage Systems [article]

Vaneet Aggarwal, Abubakr O. Al-Abbasi, Jingxian Fan, Tian Lan
2017 arXiv   pre-print
To this end, we propose a framework for quantifying and optimizing tail latency in erasure-coded storage systems.  ...  Numerical results show significant reduction of tail latency for erasure-coded storage systems with a realistic workload.  ...  CONCLUSIONS This paper provides bounds on latency tail probabilities for distributed storage systems using erasure coding.  ... 
arXiv:1703.08337v1 fatcat:ib7xdibt35gpbjrovjvue6u5gq

On the duality of data-intensive file system design

Wittawat Tantisiriroj, Seung Woo Son, Swapnil Patil, Samuel J. Lang, Garth Gibson, Robert B. Ross
2011 Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis on - SC '11  
In both categories, the underlying file system is a key component for scalable application performance.  ...  In this paper, we explore the similarities and differences between PVFS, a parallel file system used in HPC at large scale, and HDFS, the primary storage system used in cloud computing with Hadoop.  ...  helped diagnose HDFS behavior, Philip Carns from ANL helped debug the flush() issues in PVFS, and Bin Fu and Brendan Meeder helped us with their scientific applications and data-sets for evaluation.  ... 
doi:10.1145/2063384.2063474 dblp:conf/sc/TantisirirojSPLGR11 fatcat:nldy2cthxjd2zdmuvl6ghl6jni
« Previous Showing results 1 — 15 out of 2,463 results