Filters








13 Hits in 4.2 sec

Building A High Performance Parallel File System Using Grid Datafarm and ROOT I/O [article]

Y. Morita, S. Sekiguchi
2003 arXiv   pre-print
Event parallelism of the HENP data analysis enables us to take maximum advantage of the high performance cluster computing and networking when we keep the parallelism both in the data processing phase,  ...  The framework is designed to work naturally with the parallel file system of Grid Datafarm (Gfarm).  ...  Acknowledgments The authors wish to thank NII, the National Institute of Informatics, and the KEK Network Group for their support on high speed network connectivity in Japanese academic institutions and  ... 
arXiv:cs/0306092v1 fatcat:ncsslwjlgzhp5hgcbmcxkvay54

A Taxonomy of Data Grids for Distributed Data Sharing, Management and Processing [article]

Srikumar Venugopal, Rajkumar Buyya, Kotagiri Ramamohanarao
2005 arXiv   pre-print
We then provide comprehensive taxonomies that cover various aspects of architecture, data transportation, data replication and resource allocation and scheduling.  ...  They combine high-end computing technologies with high-performance networking and wide-area storage management techniques.  ...  We thank our colleagues at the University of Melbourne -Krishna Nadiminti, Tianchi Ma and Sushant Goel -for their comments on this paper.  ... 
arXiv:cs/0506034v1 fatcat:com2we7confwvil3zz6kqubg5u

A taxonomy of Data Grids for distributed data sharing, management, and processing

Srikumar Venugopal, Rajkumar Buyya, Kotagiri Ramamohanarao
2006 ACM Computing Surveys  
We then provide comprehensive taxonomies that cover various aspects of architecture, data transportation, data replication and resource allocation and scheduling.  ...  They combine high-end computing technologies with high-performance networking and wide-area storage management techniques.  ...  We thank our colleagues at the University of Melbourne -Krishna Nadiminti, Tianchi Ma, Sushant Goel and Chee Shin Yeo-for their comments on this paper.  ... 
doi:10.1145/1132952.1132955 fatcat:hwrpfyfwmrhn5merrqnfgt3mme

A Taxonomy of Data Management Models in Distributed and Grid Environments

Farrukh Nadeem
2016 International Journal of Information Technology and Computer Science  
To meet the specific needs of these environments for data organization, replication, transfer, scheduling etc. the data management systems implement different data management models.  ...  The distributed environments vary largely in their architectures, from tightly coupled cluster environment to loosely coupled Grid environment and completely uncoupled peer-to-peer environment, and thus  ...  Usually, the Grid projects from scientific domains are data intensive. These projects are more often from domain of high energy physics, astronomy, bioinformatics and earth sciences etc.  ... 
doi:10.5815/ijitcs.2016.03.03 fatcat:lfunjcv2tzdg7ew5ljftykafri

Grids and Grid technologies for wide-area distributed computing

Mark Baker, Rajkumar Buyya, Domenico Laforenza
2002 Software, Practice & Experience  
for running coarse-grained distributed and parallel applications.  ...  In fact, many applications can benefit from the Grid infrastructure, including collaborative engineering, data exploration, high-throughput computing, and of course distributed supercomputing.  ...  We have had intellectual communication and exchanged views on this upcoming technology with David Abramson (Monash), Fran Berman (UCSD), David C.  ... 
doi:10.1002/spe.488 fatcat:qcgblfolg5aori5m4fncgqdhsy

Cloud Data Management for Scientific Workflows: Research Issues, Methodologies, and State-of-the-Art

Dong Yuan, Lizhen Cui, Xiao Liu
2014 2014 10th International Conference on Semantics, Knowledge and Grids  
With this continuing data explosion, high performance computing systems are needed to store and process data efficiently, and workflow technologies are facilitated to automate these scientific applications  ...  Running scientific workflow applications usually need not only high performance computing resources but also massive storage.  ...  It also has a corresponding file system, process scheduler and parallel I/O APIs. GDMP [16] mainly focuses on replication in the grid environment, which has been utilised in High Energy Physics.  ... 
doi:10.1109/skg.2014.37 dblp:conf/skg/YuanCL14 fatcat:suphvnfsujgc3ps2tbvsj6zaya

Wide area data replication for scientific collaborations

A. Chervenak, R. Schuler, C. Kesselman, S. Koranda, B. Moe
2005 The 6th IEEE/ACM International Workshop on Grid Computing, 2005.  
We present the design and implementation of a Data Replication Service (DRS), one of a planned set of higher-level data management services for Grids.  ...  The capabilities of the DRS are based on the publication capability of the Lightweight Data Replicator (LDR) system developed for the LIGO Scientific Collaboration.  ...  Acknowledgements We are grateful to Mats Rynge for his help in setting up the DRS service in the GT4.0 testbed; to Ravi Madduri and Bill Allcock for help with RFT testing and configuration and for providing  ... 
doi:10.1109/grid.2005.1542717 dblp:conf/grid/ChervenakSKKM05 fatcat:aq67trbdeffyhb2upn6efsgsha

Distributed Data Mining on Grids: Services, Tools, and Applications

M. Cannataro, A. Congiusta, A. Pugliese, D. Talia, P. Trunfio
2004 IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics)  
Data mining algorithms are widely used today for the analysis of large corporate and scientific datasets stored in databases and data archives.  ...  For the development of data mining applications on grids we designed a system called KNOWLEDGE GRID.  ...  ACKNOWLEDGMENT The authors wish to thank the organizations involved in the MIUR SP3 project that offered their machines for running the experiments discussed in this paper.  ... 
doi:10.1109/tsmcb.2004.836890 pmid:15619945 fatcat:rk5sqabry5h5va4bymk6cyv4ou

Wide area data replication for scientific collaborations

Ann Chervenak, Robert Schuler, Carl Kesselman, Scott Koranda, Brian Moe
2008 International Journal of High Performance Computing and Networking  
We present the design and implementation of a Data Replication Service (DRS), one of a planned set of higher-level data management services for Grids.  ...  The capabilities of the DRS are based on the publication capability of the Lightweight Data Replicator (LDR) system developed for the LIGO Scientific Collaboration.  ...  Acknowledgements We are grateful to Mats Rynge for his help in setting up the DRS service in the GT4.0 testbed; to Ravi Madduri and Bill Allcock for help with RFT testing and configuration and for providing  ... 
doi:10.1504/ijhpcn.2008.020857 fatcat:oekp4fi3s5gcjar2isrnvz26ni

Comparative Analysis of Distributed and Parallel File Systems' Internal Techniques [article]

Viacheslav Dubeyko
2019 arXiv   pre-print
However, evolution of physical technologies of persistent data storage requires significant changing of concepts and approaches of file systems' internal techniques.  ...  As a result, problem of improving performance of data processing treats as a problem of file system performance optimization.  ...  , task in data-intensive computing such as high energy physics, astronomy, space exploration, and human genome analysis.  ... 
arXiv:1904.03997v1 fatcat:j7ko6enq7zaflf5ajs5ci6ghwy

Gfarm v2: A Grid file system that supports high-performance distributed and parallel data computing [article]

Y Morita, O Tatebe, N Soda, S Sekiguchi, S Matsuoka
2005
Grid Datafarm architecture is designed for facilitating reliable file sharing and high-performance distributed and parallel data computing in a Grid across administrative domains by providing a global  ...  This paper discusses the design and implementation of Gfarm v2 that provides a secure, robust, scalable and high-performance global virtual file system.  ...  We also thank the members of Grid Technology Research Center, AIST, for their cooperation in this work.  ... 
doi:10.5170/cern-2005-002.1172 fatcat:arkf63irwvd73boeo3baeia3u4

Global Data For Global Science

Multiple Authors
2015 Zenodo  
The Proceedings of the 1st ICSU World Data System Conference, held 3–6 September 2011 at Kyoto University, Japan, contains the papers submitted to the Data Science Journal.  ...  Watanabe and other members of WDS-SC for their arrangement in publishing a special issue on CODATA Data Science Journal, as a proceeding ACKNOWLEDGEMENTS The present work was done by using resources of  ...  Second, the Director would like to recognize NOAA's National Climatic Data Center for hosting the WDC and for making the GOSIC utility possible on an operational basis.  ... 
doi:10.5281/zenodo.34188 fatcat:dhoudwjt2bdjnmnq2e3jubmysa

Grid Services for Measurements on Digital Wireless Communications Systems

Aniello Napolitano
2009
The attention is focused on the implementation of GRID services for performance assessing of digital wireless communication systems.  ...  in GRID architecture, such as PBS scheduler, for dealing with the instruments discovery issue.  ...  , high-energy physics data analysis, climatology, large-scale remote instrument operation, and so forth.  ... 
doi:10.6092/unina/fedoa/3901 fatcat:a53cdddd7ne6tdshx5lu4jidzm