2,262 Hits in 3.9 sec

Experiences in Exascale Scientific Data Management

Mario Lassnig, Martin Barisits, Dimitrios Christidis
2020 IEEE Data Engineering Bulletin  
Scientific data management has become an increasingly difficult challenge.  ...  This article reports on our experiences in this field: we illustrate challenges in operating the exascale dataflows of the high energy physics experiment ATLAS, we detail the concepts and architecture  ...  In this article we address four topics relevant to these challenges: in Section 2 we describe first-hand experiences developing and operating an exascale data management system for high energy physics,  ... 
dblp:journals/debu/LassnigBC20 fatcat:kjuftpsohrbpvbts47kt3owbfm

Towards Exascale Distributed Data Management

Giovanni Aloisio, Sandro Fiore
2009 The international journal of high performance computing applications  
Significant improvements in the data management field will increase research productivity in solving complex scientific problems.  ...  Distribution: the exascale scenario will involve a lot of distributed data available at an international level across several countries (data coming from simulations, observations, experiments, etc.).  ...  Scientific Discovery of data: in a exascale environment search & discovery of data will continue to play a critical role.  ... 
doi:10.1177/1094342009347702 fatcat:kuaplw62indhhleuzwo3chywyu

Architectures and methodologies for future deployment of multi-site Zettabyte-Exascale data handling platforms

V Acín, I Bird, T Boccali, G Cancio, I P Collier, D Corney, B Delaunay, M Delfino, L dell'Agnello, J Flix, P Fuhrmann, M Gasthuber (+12 others)
2015 Journal of Physics, Conference Series  
large European scientific data centres on architectures and methodologies that should be studied by building proof-of-concept systems, in order to prepare the way for building reliable and economic Zettabyte-Exascale  ...  It also addresses the challenges associated with the need to preserve scientific data for many decades.  ...  data of experiments. • The overall increase in scale.  ... 
doi:10.1088/1742-6596/664/4/042009 fatcat:jy3xnl5r5zd3hmt3g2kac6h5sy

How a Community Software Ecosystem Perspective Helps to Advance Science Goals in the Exascale Computing Project [article]

Lois Curfman McInnes
Exascale Computing Project (ECP) are working toward scientific advances on forthcoming exascale platforms, across a diverse suite of applications in chemistry, materials, energy, Earth and space science  ...  Scientific Software Stack (E4S).  ...  More info about the impact of ECP software technologies • Let's Talk Exascale Podcast: -Supporting Scientific Discovery and Data Analysis in the Exascale Era -ECP Leadership Discusses Project  ... 
doi:10.6084/m9.figshare.14120930.v2 fatcat:erhhkvll55gxjddlsolwhvjrki

The Landscape of Exascale Research

Stijn Heldens, Pieter Hijma, Ben Van Werkhoven, Jason Maassen, Adam S. Z. Belloum, Rob V. Van Nieuwpoort
2020 ACM Computing Surveys  
We use a three-stage approach in which we (1) discuss various exascale landmark studies, (2) use data-driven techniques to analyze the large collection of related literature, and (3) discuss eight research  ...  In this work, we present an overview of these efforts and provide insight into the important trends, developments, and exciting research opportunities in exascale computing.  ...  ., data management, visualization, scientific discovery).  ... 
doi:10.1145/3372390 fatcat:jhtwt7pxd5c5darhz75hiqgsnq

The International Exascale Software Project roadmap

Jack Dongarra, Pete Beckman, Terry Moore, Patrick Aerts, Giovanni Aloisio, Jean-Claude Andre, David Barkai, Jean-Yves Berthou, Taisuke Boku, Bertrand Braunschweig, Franck Cappello, Barbara Chapman (+53 others)
2011 The international journal of high performance computing applications  
This report describes the work of the community to prepare for the challenges of exascale computing, ultimately combing their efforts in a coordinated International Exascale Software Project. and the Tokyo  ...  Encourage and facilitate collaboration in education and training: The magnitude of the changes in programming models and software infrastructure and tools brought about by the transition to peta/exascale  ...  Likewise, the areas that have less software maturity (e.g., health and energy) have more Xs in the programming, languages, and debugging columns. Science and  ... 
doi:10.1177/1094342010391989 fatcat:twdszcjfxraijpsdcdacvpp6vm

Beyond Super: Powering the exascale ERA [article]

April Neoh
2020 Figshare  
technologies to deliver a scientific outcome.  ...  Listen to how HPE and Cray are combining unique IP to deliver the capabilities you will need in the exascale era with systems that perform like a supercomputer and run like a cloud.Together, we are redefining  ...  That ultimately manifests itself in some sort of HPC cluster and in the future an Exascale supercomputer.  ... 
doi:10.6084/m9.figshare.11929641.v1 fatcat:4nnoxewkqzbupp6e54f24f72mq

How a Community Software Ecosystem Perspective Helps to Advance Science Goals in the Exascale Computing Project [article]

Lois Curfman McInnes
Exascale Computing Project (ECP) are working toward scientific advances on forthcoming exascale platforms, across a diverse suite of applications in chemistry, materials, energy, Earth and space science  ...  Scientific Software Stack (E4S).  ...  compression methods to manage big science data volumes -ALPINE project tests novel algorithm for in situ exascale data analysis -Workflow technologies impact SC20 Gordon Bell COVID-19 award winner and  ... 
doi:10.6084/m9.figshare.14120930.v1 fatcat:gnfherkpivcdhefriq6yv6jyje

Exascale computing and big data

Daniel A. Reed, Jack Dongarra
2015 Communications of the ACM  
Acknowledgments We are grateful for insights and perspectives we received from the DARPA and DOE exascale hardware, software, and application study groups.  ...  Commercial cloud providers have already faced many of these problems, and their experience in large-scale resource management has much to offer the scientific computing ecosystem.  ...  Advances today are also shaped by data-analysis pipelines, data architectures, and machine learning tools that manage large volumes of scientific and engineering data.  ... 
doi:10.1145/2699414 fatcat:d3lyhp4hyveefoq4y3ganonqiu

EXA2PRO programming environment

Dimitrios Soudris, Raymond Namyst, Dirk Pleiter, Georgi Gaydadjiev, Tobias Becker, Matthieu Haefele, Lazaros Papadopoulos, Christoph W. Kessler, Dionysios D. Kehagias, Athanasios Papadopoulos, Panos Seferlis, Alexander Chatzigeorgiou (+2 others)
2018 Proceedings of the 18th International Conference on Embedded Computer Systems Architectures, Modeling, and Simulation - SAMOS '18  
The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.  ...  HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not.  ...  In applications deployed in exascale architectures memory management techniques can assist in performance improvements in several ways.  ... 
doi:10.1145/3229631.3239369 dblp:conf/samos/SoudrisPKKPSCAT18 fatcat:sbqs5qyhgvgtdghb5sf2no6y3q

Making a case for distributed file systems at Exascale

Ioan Raicu, Ian T. Foster, Pete Beckman
2011 Proceedings of the third international workshop on Large-scale system and application performance - LSAP '11  
We propose that future high-end computing systems be designed with non-volatile memory on every compute node, allowing every compute node to actively participate in the metadata and data management and  ...  Exascale computers will enable the unraveling of significant scientific mysteries.  ...  ACKNOWLEDGMENTS This work was supported in part by the U.S. Dept. of Energy under Contract DE-AC02-06CH11357, as well as the National Science Foundation grant NSF-0937060 CIF-72 and NSF-1054974.  ... 
doi:10.1145/1996029.1996034 fatcat:bon3bizokzckhl5ajrzsqt7tqq

Evolution of the ATLAS PanDA workload management system for exascale computational science

T Maeno, K De, A Klimentov, P Nilsson, D Oleynik, S Panitkin, A Petrosyan, J Schovancova, A Vaniachine, T Wenaus, D Yu
2014 Journal of Physics, Conference Series  
scientific communities -To take the core components of PanDA and package them in an experiment neutral package • General components + customizable layers • The experiment specific layers as plug-ins  ...  of processing and data management, for HEP and other data- intensive sciences, and a wider exascale community  3 FTE for 3 years from 2012  Three dimensions to evolution - Making PanDA available  ...  many areas : networking, VO independent PanDA instance, cloud computing, HPC  Strong interest in the project from several experiments and scientific centers to have a collaborative project  ... 
doi:10.1088/1742-6596/513/3/032062 fatcat:fvhg3qtvmrcrjgotac62gffpju

Message from the Workshop Chairs

2020 2020 IEEE/ACM 2nd Annual Workshop on Extreme-scale Experiment-in-the-Loop Computing (XLOOP)  
Additionally, computational and data management infrastructure is moving into the exascale, with the capacity to ingest and process ever larger datasets.  ...  This is an exciting time for large-scale experimental science-physics experiments are being scaled up in energy and resolution; new and higher quality images are available for the observation of earth  ...  High performance data management and transfer techniques must be developed to manage and manipulate simulated and observed data sets.  ... 
doi:10.1109/xloop51963.2020.00004 fatcat:4ntdnzlwlzdtvkwshcs7i3iv6u

ESiWACE2 Sustainability Plan

Bryan Lawrence, Joachim Biercamp
2022 Zenodo  
of society to exploit exascale (and post-exascale) computing to provide the weather and climate information that our society urgently needs.  ...  The European Centre of Excellence in Weather and Climate in Europe (ESiWACE) is an activity that has been funded by the European Commission since 2016 under the HPC Centre of Excellence (CoE) programme  ...  But efficiently managing exabytes of data which is at rest requires complex information systems, optimised for handling weather and climate data.  ... 
doi:10.5281/zenodo.6302900 fatcat:rwores4gwzfyjbx6xhzkguwdr4

Architecture-aware Algorithms and Software for Peta and Exascale Computing

Jack Dongarra
2011 2011 IEEE International Parallel & Distributed Processing Symposium  
Ø Efficient in situ data analysis will require restructuring of scientific workflows and applications. Ø Techniques for data coordinating and mining Top 10 Challenges to Exascale 1.  ...  Data management: Ø Creating data management software that can handle the volume, velocity and diversity of data that is anticipated.  ...  Interconnect technology: § Increasing the performance and energy efficiency of data movement.  ... 
doi:10.1109/ipdps.2011.419 dblp:conf/ipps/Dongarra11 fatcat:3minwqg5ojbaxnhgi5c3rv53g4
« Previous Showing results 1 — 15 out of 2,262 results