Steve Petruzza, Aniketh Venkat, Attila Gyulassy, Giorgio Scorzelli, Frederick Federer, Alessandra Angelucci, Valerio Pascucci, Peer-Timo Bremer
2017 SIGGRAPH Asia 2017 Symposium on Visualization on - SA '17  
Modern science is inundated with ever increasing data sizes as computational capabilities and image acquisition techniques continue to improve. For example, simulations are tackling ever larger domains with higher fidelity, and high-throughput microscopy techniques generate larger data that are fundamental to gather biologically and medically relevant insights. As the image sizes exceed memory, and even sometimes local disk space, each step in a scientific workflow is impacted. Current software
more » ... solutions enable data exploration with limited interactivity for visualization and analytic tasks. Furthermore analysis on HPC systems often require complex hand-written parallel implementations of algorithms that suffer from poor portability and maintainability. We present a software infrastructure that simplifies end-to-end visualization and analysis of massive data. First, a hierarchical streaming data access layer enables interactive exploration of remote data, with fast data fetching to test analytics on subsets of the data. Second, a library simplifies the process of developing new analytics algorithms, allowing users to rapidly prototype new approaches and deploy them in an HPC setting. Third, a scalable runtime system automates mapping analysis algorithms to whatever computational hardware is available, reducing the † complexity of developing scaling algorithms. We demonstrate the usability and performance of our system using a use case from neuroscience: filtering, registration, and visualization of terascale microscopy data. We evaluate the performance of our system using a leadership-class supercomputer, Shaheen II. CCS CONCEPTS Computing methodologies → Massively parallel algorithms; Parallel programming languages; Software and its engineering → Development frameworks and environments; Integrated and visual development environments Keywords Interactive visualization and analysis; parallel custom analysis work-flows; algorithms scalability; microscopy
doi:10.1145/3139295.3139299 pmid:30148289 pmcid:PMC6105268 dblp:conf/siggraph/PetruzzaVGSFAPB17 fatcat:ppcwkpk32jd65arcaw3nrnvzne