Filters








54 Hits in 0.75 sec

Data Analysis using ALICE Run3 Framework

Giulio Eulisse, Jan Fiete Grosse-Oetringhaus, Peter Hristov, Gian Michele Innocenti, Naghmeh Mohammadi
2019 Zenodo  
ALICE Experiment is currently undergoing a major upgrade program, both in terms of hardware and software, to prepare for the LHC Run 3. A new Software Framework is being developed in collaboration with the FAIR experiments at GSI to cope with the 100 fold increase in collected collisions. We present our progress to adapt such a framework for the end user physics data analysis. In particular, we will highlight the design and technology choices. We will show how we adopt Apache Arrow as a
more » ... for our in memory analysis data layout. We will illustrate the benefits of this solution, such as: efficient and parallel data processing, interoperability with a large number of analysis tools and ecosystems, integration with the modern ROOT declarative analysis framework RDataFrame.
doi:10.5281/zenodo.3599423 fatcat:dej2agqvnfa77olowx66qoh2vu

IGUANA Architecture, Framework and Toolkit for Interactive Graphics [article]

George Alverson, Giulio Eulisse, Shahzad Muzaffar, Ianna Osborne, Lassi A. Tuura, Lucas Taylor
2003 arXiv   pre-print
IGUANA is a generic interactive visualisation framework based on a C++ component model. It provides powerful user interface and visualisation primitives in a way that is not tied to any particular physics experiment or detector design. The article describes interactive visualisation tools built using IGUANA for the CMS and D0 experiments, as well as generic GEANT4 and GEANT3 applications. It covers features of the graphical user interfaces, 3D and 2D graphics, high-quality vector graphics
more » ... for print media, various textual, tabular and hierarchical data views, and integration with the application through control panels, a command line and different multi-threading models.
arXiv:cs/0306042v1 fatcat:ulqzp5smxvbfzn7dj67tv4ucnu

Running synchronous detector reconstruction in ALICE using declarative workflows

Matthias Richter, Giulio Eulisse, Sandro Christian Wenzel, David Rohr, Ruben Shahoyan
2019 Zenodo  
The ALICE experiment at the Large Hadron Collider (LHC) at CERN will deploy a combined online-offline facility for detector readout and reconstruction, as well as data compression. This system is designed to allow the inspection of all collisions at rates of 50 kHz in the case of Pb-Pb and 400 kHz for pp collisions in order to give access to rare physics signals. The input data rate of up to of 3.4 TByte/s requires that a large part of the detector reconstruction will be realized online in the
more » ... ynchronous stage of the system. The data processing is based on individual algorithms which will be executed in parallel processes on multiple compute nodes. Data and workload will be distributed among the nodes and processes using message queue communication provided by the FairMQ package of the ALFA software framework. As the ALICE specific layer, a message-passing aware data model and annotation allows to efficiently describe data and routing. Finally, the Data Processing Layer introduces the description of the reconstruction in a data-flow oriented approach, and makes the complicated nature of a distributed system transparent to users and developers. So-called workflows are defined in a declarative language as sequences of processes with inputs, the algorithm, and outputs as the three descriptive properties. With this layered structure of the ALICE software, development of specific stages of the reconstruction can be done in a flexible way in the domain of the specified processes without the need of boiler-plate adjustments and taking into account details of the distributed and parallel system. The Data Processing Layer framework takes care of generating the workflow with the required connections and synchronization, and interfaces to the backend deploying the workflow on computing resources. For the development it is completely transparent whether to run a workflow on a laptop or a computer cluster. The modular software framework is the basis for splitting the data processing into manageable pieces and helps to distri [...]
doi:10.5281/zenodo.3599461 fatcat:bantuq6tdvat7gt7rwtpyprczu

Heterogeneous High Throughput Scientific Computing with APM X-Gene and Intel Xeon Phi [article]

David Abdurachmanov, Brian Bockelman, Peter Elmer, Giulio Eulisse, Robert Knight, Shahzad Muzaffar
2014 arXiv   pre-print
Electrical power requirements will be a constraint on the future growth of Distributed High Throughput Computing (DHTC) as used by High Energy Physics. Performance-per-watt is a critical metric for the evaluation of computer architectures for cost- efficient computing. Additionally, future performance growth will come from heterogeneous, many-core, and high computing density platforms with specialized processors. In this paper, we examine the Intel Xeon Phi Many Integrated Cores (MIC)
more » ... or and Applied Micro X-Gene ARMv8 64-bit low-power server system-on-a-chip (SoC) solutions for scientific computing applications. We report our experience on software porting, performance and energy efficiency and evaluate the potential for use of such technologies in the context of distributed computing systems such as the Worldwide LHC Computing Grid (WLCG).
arXiv:1410.3441v1 fatcat:glzfbbezwfbijl7j4an3akobzm

Techniques and tools for measuring energy efficiency of scientific software applications [article]

David Abdurachmanov, Peter Elmer, Giulio Eulisse, Robert Knight, Tapio Niemi, Jukka K. Nurminen, Filip Nyback, Goncalo Pestana, Zhonghong Ou, Kashif Khan
2014 arXiv   pre-print
The scale of scientific High Performance Computing (HPC) and High Throughput Computing (HTC) has increased significantly in recent years, and is becoming sensitive to total energy use and cost. Energy-efficiency has thus become an important concern in scientific fields such as High Energy Physics (HEP). There has been a growing interest in utilizing alternate architectures, such as low power ARM processors, to replace traditional Intel x86 architectures. Nevertheless, even though such solutions
more » ... have been successfully used in mobile applications with low I/O and memory demands, it is unclear if they are suitable and more energy-efficient in the scientific computing environment. Furthermore, there is a lack of tools and experience to derive and compare power consumption between the architectures for various workloads, and eventually to support software optimizations for energy efficiency. To that end, we have performed several physical and software-based measurements of workloads from HEP applications running on ARM and Intel architectures, and compare their power consumption and performance. We leverage several profiling tools (both in hardware and software) to extract different characteristics of the power use. We report the results of these measurements and the experience gained in developing a set of measurement techniques and profiling tools to accurately assess the power consumption for scientific workloads.
arXiv:1410.3440v1 fatcat:uvmaqmw7gvbc3db3gse2uwmnj4

Data Handling In The Alice O2 Event Processing

Matthias Richter, Mikolaj Krzewicki, Giulio Eulisse, A. Forti, L. Betev, M. Litmaath, O. Smirnova, P. Hristov
2019 EPJ Web of Conferences  
The ALICE experiment at the Large Hadron Collider (LHC) at CERN is planned to be operated in a continuous data-taking mode in Run 3. This will allow to inspect data from all Pb-Pb collisions at a rate of 50 kHz, giving access to rare physics signals embedded in a large background. Based on experience with real-time reconstruction of particle trajectories and event properties in the ALICE High Level Trigger, the ALICE O2 facility is currently designed and developed to support processing of a
more » ... inuous, triggerless stream of data segmented into entities referred to as timeframes. Both raw data input into the ALICE O2 system and the actual processing of aggregated timeframes are distributed among multiple processes on a manynode cluster. Process communication is based on the asynchronous message passing paradigm. This paper presents the basic concept for identification of data in the distributed system together with prototype implementations and performance measurements.
doi:10.1051/epjconf/201921401035 fatcat:do6cram4p5fm3fukawhenkrdfy

ALICE Run 3 Analysis Framework

Anton Alkin, Giulio Eulisse, Jan Fiete Grosse-Oetringhaus, Peter Hristov, Maja Kabus, C. Biscarat, S. Campana, B. Hegner, S. Roiser, C.I. Rovelli, G.A. Stewart
2021 EPJ Web of Conferences  
In LHC Run 3 the ALICE Collaboration will have to cope with an increase of lead-lead collision data of two orders of magnitude compared to the Run 1 and 2 data-taking periods. The Online-Offline (O2) software framework has been developed to allow for distributed and efficient processing of this unprecedented amount of data. Its design, which is based on a message-passing back end, required the development of a dedicated Analysis Framework that uses the columnar data format provided by Apache
more » ... ow. The O2 Analysis Framework provides a user-friendly high-level interface and hides the complexity of the underlying distributed framework. It allows the users to access and manipulate the data in the new format both in the traditional "event loop" and a declarative approach using bulk processing operations based on Arrow's Gandiva sub-project. Building on the well-tested system of analysis trains developed by ALICE in Run 1 and 2, the AliHyperloop infrastructure is being developed. It provides a fast and intuitive user interface for running demanding analysis workflows in the GRID environment and on the dedicated Analysis Facility. In this document, we report on the current state and ongoing developments of the Analysis Framework and of AliHyperloop, highlighting the design choices and the benefits of the new system.
doi:10.1051/epjconf/202125103063 fatcat:cdwzxb3kijfuzgebogvwyefkva

Power-aware applications for scientific cluster and distributed computing [article]

David Abdurachmanov, Peter Elmer, Giulio Eulisse, Paola Grosso, Curtis Hillegas, Burt Holzman, Ruben L. Janssen, Sander Klous, Robert Knight, Shahzad Muzaffar
2014 arXiv   pre-print
The aggregate power use of computing hardware is an important cost factor in scientific cluster and distributed computing systems. The Worldwide LHC Computing Grid (WLCG) is a major example of such a distributed computing system, used primarily for high throughput computing (HTC) applications. It has a computing capacity and power consumption rivaling that of the largest supercomputers. The computing capacity required from this system is also expected to grow over the next decade. Optimizing
more » ... power utilization and cost of such systems is thus of great interest. A number of trends currently underway will provide new opportunities for power-aware optimizations. We discuss how power-aware software applications and scheduling might be used to reduce power consumption, both as autonomous entities and as part of a (globally) distributed system. As concrete examples of computing centers we provide information on the large HEP-focused Tier-1 at FNAL, and the Tigress High Performance Computing Center at Princeton University, which provides HPC resources in a university context.
arXiv:1404.6929v2 fatcat:op74jfffafhyjbxgu42u3xnzai

Evolution of the ALICE Software Framework for Run 3

Giulio Eulisse, Piotr Konopka, Mikolaj Krzewicki, Matthias Richter, David Rohr, Sandro Wenzel, A. Forti, L. Betev, M. Litmaath, O. Smirnova, P. Hristov
2019 EPJ Web of Conferences  
ALICE is one of the four major LHC experiments at CERN. When the accelerator enters the Run 3 data-taking period, starting in 2021, ALICE expects almost 100 times more Pb-Pb central collisions than now, resulting in a large increase of data throughput. In order to cope with this new challenge, the collaboration had to extensively rethink the whole data processing chain, with a tighter integration between Online and Offline computing worlds. Such a system, code-named ALICE O2, is being developed
more » ... in collaboration with the FAIR experiments at GSI. It is based on the ALFA framework which provides a generalized implementation of the ALICE High Level Trigger approach, designed around distributed software entities coordinating and communicating via message passing. We will highlight our efforts to integrate ALFA within the ALICE O2 environment. We analyze the challenges arising from the different running environments for production and development, and conclude on requirements for a flexible and modular software framework. In particular we will present the ALICE O2 Data Processing Layer which deals with ALICE specific requirements in terms of Data Model. The main goal is to reduce the complexity of development of algorithms and managing a distributed system, and by that leading to a significant simplification for the large majority of the ALICE users.
doi:10.1051/epjconf/201921405010 fatcat:pj7lw6gj5bbdzj7dxc6ypp3a4y

Data Analysis using ALICE Run 3 Framework

Giulio Eulisse, Anton Alkin, Jan Fiete Grosse-Oetringhaus, Peter Hristov, Gian Michele Innocenti, Maja Jadwiga Kabus, C. Doglioni, D. Kim, G.A. Stewart, L. Silvestris, P. Jackson, W. Kamleh
2020 EPJ Web of Conferences  
The ALICE Experiment is currently undergoing a major upgrade program, both in terms of hardware and software, to prepare for the LHC Run 3. A new Software Framework is being developed in collaboration with the FAIR experiments at GSI to cope with the 100-fold increase in the number of recorded events. We present our progress to adapt such a framework for the end user physics data analysis. In particular, the design and technology choices are highlighted. How Apache Arrow is adopted as the
more » ... rm for the in-memory analysis data layout is discussed. The benefits of this solution are illustrated, these include: efficient and parallel data processing, interoperability with a large number of analysis tools and ecosystems, integration with the modern ROOT declarative analysis framework RDataFrame.
doi:10.1051/epjconf/202024506032 fatcat:ori3beqrw5f63ntjgxydlsiv5i

Future Computing Platforms for Science in a Power Constrained Era

David Abdurachmanov, Peter Elmer, Giulio Eulisse, Robert Knight
2015 Journal of Physics, Conference Series  
Power consumption will be a key constraint on the future growth of Distributed High Throughput Computing (DHTC) as used by High Energy Physics (HEP). This makes performance-per-watt a crucial metric for selecting cost-efficient computing solutions. For this paper, we have done a wide survey of current and emerging architectures becoming available on the market including x86-64 variants, ARMv7 32-bit, ARMv8 64-bit, Many-Core and GPU solutions, as well as newer System-on-Chip (SoC) solutions. We
more » ... ompare performance and energy efficiency using an evolving set of standardized HEP-related benchmarks and power measurement techniques we have been developing. We evaluate the potential for use of such computing solutions in the context of DHTC systems, such as the Worldwide LHC Computing Grid (WLCG).
doi:10.1088/1742-6596/664/9/092007 fatcat:25drb6amwzetbnivssorwqmvfy

Techniques and tools for measuring energy efficiency of scientific software applications

David Abdurachmanov, Peter Elmer, Giulio Eulisse, Robert Knight, Tapio Niemi, Jukka K Nurminen, Filip Nyback, Gonçalo Pestana, Zhonghong Ou, Kashif Khan
2015 Journal of Physics, Conference Series  
The scale of scientific High Performance Computing (HPC) and High Throughput Computing (HTC) has increased significantly in recent years, and is becoming sensitive to total energy use and cost. Energy-efficiency has thus become an important concern in scientific fields such as High Energy Physics (HEP). There has been a growing interest in utilizing alternate architectures, such as low power ARM processors, to replace traditional Intel x86 architectures. Nevertheless, even though such solutions
more » ... have been successfully used in mobile applications with low I/O and memory demands, it is unclear if they are suitable and more energy-efficient in the scientific computing environment. Furthermore, there is a lack of tools and experience to derive and compare power consumption between the architectures for various workloads, and eventually to support software optimizations for energy efficiency. To that end, we have performed several physical and software-based measurements of workloads from HEP applications running on ARM and Intel architectures, and compare their power consumption and performance. We leverage several profiling tools (both in hardware and software) to extract different characteristics of the power use. We report the results of these measurements and the experience gained in developing a set of measurement techniques and profiling tools to accurately assess the power consumption for scientific workloads.
doi:10.1088/1742-6596/608/1/012032 fatcat:n24zh22uh5fs3nwjhe6d3wvz5q

Initial explorations of ARM processors for scientific computing

David Abdurachmanov, Peter Elmer, Giulio Eulisse, Shahzad Muzaffar
2014 Journal of Physics, Conference Series  
Power efficiency is becoming an ever more important metric for both high performance and high throughput computing. Over the course of next decade it is expected that flops/watt will be a major driver for the evolution of computer architecture. Servers with large numbers of ARM processors, already ubiquitous in mobile computing, are a promising alternative to traditional x86-64 computing. We present the results of our initial investigations into the use of ARM processors for scientific
more » ... applications. In particular we report the results from our work with a current generation ARMv7 development board to explore ARM-specific issues regarding the software development environment, operating system, performance benchmarks and issues for porting High Energy Physics software.
doi:10.1088/1742-6596/523/1/012009 fatcat:i56v6uagvjdrfeyygs72fttary

Optimizing CMS build infrastructure via Apache Mesos

David Abdurachmanov, Alessandro Degano, Peter Elmer, Giulio Eulisse, David Mendez, Shahzad Muzaffar
2015 Journal of Physics, Conference Series  
The Offline Software of the CMS Experiment at the Large Hadron Collider (LHC) at CERN consists of 6M lines of in-house code, developed over a decade by nearly 1000 physicists, as well as a comparable amount of general use open-source code. A critical ingredient to the success of the construction and early operation of the WLCG was the convergence, around the year 2000, on the use of a homogeneous environment of commodity x86-64 processors and Linux. Apache Mesos is a cluster manager that
more » ... s efficient resource isolation and sharing across distributed applications, or frameworks. It can run Hadoop, Jenkins, Spark, Aurora, and other applications on a dynamically shared pool of nodes. We present how we migrated our continuos integration system to schedule jobs on a relatively small Apache Mesos enabled cluster and how this resulted in better resource usage, higher peak performance and lower latency thanks to the dynamic scheduling capabilities of Mesos.
doi:10.1088/1742-6596/664/6/062013 fatcat:nv6vdplzhjai5msdpvbud42a5y

HEP Software Foundation Community White Paper Working Group - Software Development, Deployment and Validation [article]

Benjamin Couturier, Giulio Eulisse, Hadrien Grasland, Benedikt Hegner, Michel Jouvin, Meghan Kane, Daniel S. Katz, Thomas Kuhr, David Lange, Patricia Mendez Lorenzo, Martin Ritter, Graeme Andrew Stewart, Andrea Valassi
2018 arXiv   pre-print
The High Energy Phyiscs community has developed and needs to maintain many tens of millions of lines of code and to integrate effectively the work of thousands of developers across large collaborations. Software needs to be built, validated, and deployed across hundreds of sites. Software also has a lifetime of many years, frequently beyond that of the original developer, it must be developed with sustainability in mind. Adequate recognition of software development as a critical task in the HEP
more » ... community needs to be fostered and an appropriate publication and citation strategy needs to be developed. As part of the HEP Softare Foundation's Community White Paper process a working group on Software Development, Deployment and Validation was formed to examine all of these issues, identify best practice and to formulare recommendations for the next decade. Its report is presented here.
arXiv:1712.07959v2 fatcat:qctl73r3bzhqtlu62gjfntorqy
« Previous Showing results 1 — 15 out of 54 results