Filters








118,056 Hits in 6.0 sec

Large-scale File System Design and Architecture

V. Dynda, P. Rydlo
2002 Acta Polytechnica  
, the paper surveys general challenges and requirements for large-scale file systems, and then the design of particular elementary parts of the proposed file system is presented.  ...  and will be referred further in the text under this name or its abbreviation GFS (Gaston File System).  ...  The environment of a large-scale distributed file system, as wide as the Internet, contains many individual machines.  ... 
doaj:25aaf578e751450aa40c8b546fc91f64 fatcat:fuyj64sz5bdfnpbkhlzfawrccq

Cluster-based file replication in large-scale distributed systems

Harjinder S. Sandhu, Songnian Zhou
1992 Proceedings of the 1992 ACM SIGMETRICS joint international conference on Measurement and modeling of computer systems - SIGMETRICS '92/PERFORMANCE '92  
reduce file access delays and server and backbone network utilizations in large-scale distributed systems over a wide range of workload conditions.  ...  The increasing need for data sharing in large-scale ing a statistical workload model based upon measurement data and real workload characteristics show that clusterbased file replication can significantly  ...  Harjinder Sandhu was supported in part by an Ontario Graduate Scholarship.  ... 
doi:10.1145/133057.133092 dblp:conf/sigmetrics/SandhuZ92 fatcat:3ox5grr6kbha3julrtjy3rkkwi

Cluster-based file replication in large-scale distributed systems

Harjinder S. Sandhu, Songnian Zhou
1992 Performance Evaluation Review  
reduce file access delays and server and backbone network utilizations in large-scale distributed systems over a wide range of workload conditions.  ...  The increasing need for data sharing in large-scale ing a statistical workload model based upon measurement data and real workload characteristics show that clusterbased file replication can significantly  ...  Harjinder Sandhu was supported in part by an Ontario Graduate Scholarship.  ... 
doi:10.1145/149439.133092 fatcat:2i7fk4ytlnghxojp6ipmxyevpa

ShardFS vs. IndexFS

Lin Xiao, Kai Ren, Qing Zheng, Garth A. Gibson
2015 Proceedings of the Sixth ACM Symposium on Cloud Computing - SoCC '15  
distributed transactions.  ...  While few commercial file systems offer anything better than federating individually non-scalable namespace servers, a recent academic file system, IndexFS, demonstrates scalable namespace processing based  ...  Acknowledgments This research was supported in part by the National Science Foundation under awards CNS-1042537 and CNS-1042543 (PRObE, www.nmc-probe.org), the DOE and Los Alamos National Laboratory, under  ... 
doi:10.1145/2806777.2806844 dblp:conf/cloud/XiaoRZG15 fatcat:nrveu5s2nraalovprkfxuhuqaq

Design and Implementation of a Metadata Management Scheme for Large Distributed File Systems

Jong Hyeon YUN, Yong Hun PARK, Dong Min SEO, Seok Jae LEE, Jae Soo YOO
2009 IEICE transactions on information and systems  
Most large-scale distributed file systems decouple a metadata operation from read and write operations for a file.  ...  In the distributed file systems, a certain server named a metadata server (MDS) is responsible for maintaining the metadata information of the file systems.  ...  In this paper, we propose a new metadata management scheme for a cluster of MDSs to efficiently distribute metadata in the large-scale file systems.  ... 
doi:10.1587/transinf.e92.d.1475 fatcat:4rbqxpresbd5bcaietunjcdmi4

Scalable Storage for Data-Intensive Computing [chapter]

Abhishek Verma, Shivaram Venkataraman, Matthew Caesar, Roy H. Campbell
2011 Handbook of Data Intensive Computing  
Then, we describe how metadata management can be improved for a file system built to support large scale data-intensive applications.  ...  We implement Ring File System (RFS), that uses a single hop Distributed Hash Table, to manage file metadata and a traditional client-server model for managing the actual data.  ...  We presented and evaluated RFS, a scalable, fault-tolerant and high throughput file system that is well suited for large scale data-intensive applications.  ... 
doi:10.1007/978-1-4614-1415-5_4 fatcat:7wvjnynljvdwhp5qwrosfgr4bq

Distributed Metadata Management Scheme in HDFS

Xiaojun LiuChunxia Tu
2012 Advances in Information Sciences and Service Sciences  
Metadata management is critical to distributed file system. In HDFS architecture, a single master server manages all metadata, while a number of data servers store file data.  ...  A Hadoop Distributed File System (HDFS) is designed to store very large data sets reliably and to stream those data sets at high bandwidth to user applications.  ...  Distributed File System (HDFS) [2] are the most common file system deployed in large scale distributed systems such as Face book, Google and Yahoo today.  ... 
doi:10.4156/aiss.vol4.issue22.56 fatcat:w62tatvl35adrnpejesk7d3wgm

Data Management in Large-Scale P2P Systems [chapter]

Patrick Valduriez, Esther Pacitti
2005 Lecture Notes in Computer Science  
In this paper, we discuss the main issues related to data management in large-scale P2P systems. we first recall the main principles behind data management in distributed systems and the basic techniques  ...  In this paper, we discuss the issues of providing high-level data management services (schema, queries, replication, availability, etc.) in a P2P system.  ...  Martins for their many inputs and fruitful discussions in the context of the APPA project, and S. Abiteboul and I. Manolescu for fruitful discussions in the context of the MDP2P project.  ... 
doi:10.1007/11403937_9 fatcat:nwjttulmdffx7l5s3lw25qd4ta

A Comprehensive Survey on Taxonomy and Challenges of Distributed File Systems

D Sathian, R. Ilamathi, R. Praveen Kumar, J. Amudhavel, K. Rathinapriya
2016 Indian Journal of Science and Technology  
Findings: In this research, an overview about various evolution of file system and how distributed file system is being used effectively in wide variety of network applications.  ...  Background/Objectives: To analyze, find and interpret the current challenges in Distributed File Systems for research.  ...  Centralized architecture requires more administration while scaling up in the distributed file system. Decentralized architecture scaling can be managed by an administrator itself 20,21 .  ... 
doi:10.17485/ijst/2016/v9i11/89268 fatcat:n2ii66rlmjdqji3fa4whvksg6m

Managing Very-Large Distributed Datasets [chapter]

Miguel Branco, Ed Zaluska, David de Roure, Pedro Salgado, Vincent Garonne, Mario Lassnig, Ricardo Rocha
2008 Lecture Notes in Computer Science  
We describe DQ2's design and implementation, which builds upon previous work on distributed file systems, peer-to-peer systems and Data Grids.  ...  In this paper, we introduce a system for handling very large datasets, which need to be stored across multiple computing sites.  ...  We would like to acknowledge the many contributions to the design by Torre Wenaus and David Cameron and the help of David and Benjamin Gaidioz in implementing DQ2.  ... 
doi:10.1007/978-3-540-88871-0_54 fatcat:nbx7bqkiajgmnnvwqeeh2liuve

Mahasen: Distributed Storage Resource Broker [chapter]

K. D. A. K. S. Perera, T. Kishanthan, H. A. S. Perera, D. T. H. V. Madola, Malaka Walpola, Srinath Perera
2013 Lecture Notes in Computer Science  
Mahasen (Distributed Storage Resource Broker) is a Data Grid Management System (DGMS) that can manage a large volume of distributed data. It targets high volume data intensive applications.  ...  OODT provides a product service and profile service which manage data and metadata respectively. Storage/Replication. OODT stores data product in a file-based storage in a distributed manner.  ... 
doi:10.1007/978-3-642-40820-5_32 fatcat:tog2jyw7hzhunc5kh6bvkxnywy

Grid Data Management: Open Problems and New Issues

Esther Pacitti, Patrick Valduriez, Marta Mattoso
2007 Journal of Grid Computing  
We first recall the main principles behind data management in distributed systems and the basic techniques. Then we make precise the requirements for Grid data management.  ...  This makes data management critical since the techniques must scale up while addressing the autonomy, dynamicity and heterogeneity of the data sources.  ...  In this paper, we consider the general problem of Grid data management, with many applications and data sources distributed in a large-scale computer network.  ... 
doi:10.1007/s10723-007-9081-9 fatcat:6mitt2g7p5hklk323lvzdfrb2y

Comparative Analysis of Andrew Files System and Hadoop Distributed File System

Monali Mavani
2013 Lecture Notes on Software Engineering  
Parameters which are taken for comparison are Design Goals, Processes, File management, Scalability, Protection, Security, cache management replication etc.  ...  Index Terms-Andrew file system, Google file system, Hadoop distributed file system.  ...  HDFS is used to store files which are used in large scale distributed data processing where AFS is used to provide abstraction of local unix like file system to distributed users connected in client server  ... 
doi:10.7763/lnse.2013.v1.27 fatcat:lubat3fpqffcph5vfvenvggh2u

CDRM: A Cost-Effective Dynamic Replication Management Scheme for Cloud Storage Cluster

Qingsong Wei, Bharadwaj Veeravalli, Bozhao Gong, Lingfang Zeng, Dan Feng
2010 2010 IEEE International Conference on Cluster Computing  
We implemented CDRM in Hadoop Distributed File System (HDFS) and experiment results conclusively demonstrate that our CDRM is cost effective and outperforms default replication management of HDFS in terms  ...  Data replication has been widely used as a mean of increasing the data availability of large-scale cloud storage systems where failures are normal.  ...  CONCLUSION In this paper, we design a cost-effective dynamic replication management scheme for large-scale cloud storage system refereed to as CDRM.  ... 
doi:10.1109/cluster.2010.24 dblp:conf/cluster/WeiVGZF10 fatcat:l5xojeakmfesxo6tsxtunwiwcq

Distributed Data Management and Distributed File Systems

Maria Girone
2015 Journal of Physics, Conference Series  
Two of the most challenges services are the distributed file systems and the distributed data management systems.  ...  In this paper I will discuss how we have evolved from local site services to more globally independent services in the areas of distributed file systems and data management and how these capabilities may  ...  Large-scale software distribution is an ideal application for a caching system like CVMFS because between two software releases in high energy physics only about 10% of the files change, which allows caching  ... 
doi:10.1088/1742-6596/664/4/042022 fatcat:qxudd42aw5chzcfioflqhqlhf4
« Previous Showing results 1 — 15 out of 118,056 results