INTELLIGENT SUPPORT FOR HIGH I/O REQUIREMENTS OF LEADING EDGE SCIENTIFIC CODES ON HIGH-END COMPUTING SYSTEMS: THE ESTEDI PROJECT

KERSTIN KLEESE, PETER BAUMANN
2001 Developments in Teracomputing  
The most valuable assets in every scientific community are the expert work force and the research results/data produced. The last decade has seen new experimental and computational techniques developing at an ever faster pace, encouraging the production of ever larger quantities of data in ever shorter time spans. Concurrently the traditional scientific working environment has changed beyond recognition. Today scientists can use a wide spectrum of experimental, computational and analytical
more » ... ities, often widely distributed over the UK and Europe. In this environment new challenges are posed for the Management of Data, foremost in the areas of data archival, organisation of data in a heterogeneous, distributed environment, data accessibility and data exploration. In mid 1998 CLRC -Daresbury Laboratory started a Data Management Project (DAMP) which is looking specifically at the requirements of computational scientists in a High Performance Computing Environment. As part of this work surveys have been carried out, workshops were organised (Data Management 2000) and a number of follow on projects and collaborations were initiated. The European ESTEDI collaboration is such a project, it focuses on a particular aspect of data management, the support of high I/O requirements of codes on high-end computing facilities. This paper will give a short introduction into the aims and realm of the DAMP project and reports on the work of the ESTEDI co-operation.
doi:10.1142/9789812799685_0014 fatcat:ol3twqflzzalxncykzyndwt6kq