Filters








1,471 Hits in 4.4 sec

Incremental and Parallel Analytics on Astrophysical Data Streams

Dmitry Mishin, Tamas Budavari, Alexander Szalay, Yanif Ahmad
2012 2012 SC Companion: High Performance Computing, Networking Storage and Analysis  
This article will showcase our experiences in using off-the-shelf streaming technology to implement incremental and parallel spectral analysis of galaxies from the Sloan Digital Sky Survey (SDSS) to detect  ...  Stream processing methods and online algorithms are increasingly appealing in the scientific and large-scale data management communities due to increasing ingestion rates of scientific instruments, the  ...  This article presents a scalable implementation of a partial sum analytic with data-driven synchronization on a parallel stream processing platform.  ... 
doi:10.1109/sc.companion.2012.130 dblp:conf/sc/MishinBSA12 fatcat:mcw2iozawbfkre2woggedka2ue

Towards the optimization of a parallel streaming engine for telco applications

Bart Theeten, Ivan Bedini, Peter Cogan, Alessandra Sala, Tommaso Cucinotta
2014 Bell Labs technical journal  
However, many applications in the telco ecosystem require a real time, incremental streaming approach to process data in real time and enable proactive care.  ...  Storm is a scalable, fault tolerant framework for the analysis of real time streaming data. In this paper we provide a motivation for the use of real time streaming analytics in the telco ecosystem.  ...  Streaming analytics naturally lends itself to incremental computation, rather than iterative computation.  ... 
doi:10.1002/bltj.21652 fatcat:dcxocuc6unh6xg7ips34wziqze

Preparing HPC Applications for the Exascale Era: A Decoupling Strategy [article]

Ivy Bo Peng, Roberto Gioiosa, Gokcen Kestor, Erwin Laure, Stefano Markidis
2017 arXiv   pre-print
We demonstrate the effectiveness of this strategy by decoupling the reduce, particle communication, halo exchange and I/O operations in a set of scientific and data-analytics applications.  ...  Production-quality parallel applications are often a mixture of diverse operations, such as computation- and communication-intensive, regular and irregular, tightly coupled and loosely linked operations  ...  This work was supported by the DOE Office of Science, Advanced Scientific Computing Research, under the ARGO and CENATE projects (award number 66150 and 64386).  ... 
arXiv:1708.01304v1 fatcat:w7hholkmcve33lqehfqr55ryv4

Steady state magnetic reconnection in planar geometries

I. J. D. Craig, S. M. Henton
1994 Astrophysical Journal  
In particular, the field lines are isobars of uniform current density and characteristics of the velocity stream function.  ...  This solution relates the velocity stream function to the global magnetic field structure and highlights several key properties of the resistive system.  ...  Astronomical Society • Provided by the NASA Astrophysics Data System 199  ... 
doi:10.1086/174716 fatcat:o4kbmf36zbaelh2364kfhqfy2a

Snowmass Computing Frontier: Computing for the Cosmic Frontier, Astrophysics, and Cosmology [article]

Andrew Connolly, Salman Habib, Alex Szalay, Julian Borrill, George Fuller, Nick Gnedin, Katrin Heitmann, Danny Jacobs, Don Lamb, Tony Mezzacappa, Bronson Messer, Steve Myers (+5 others)
2013 arXiv   pre-print
This document presents (off-line) computing requrements and challenges for Cosmic Frontier science, covering the areas of data management, analysis, and simulations.  ...  We invite contributions to extend the range of covered topics and to enhance the current descriptions.  ...  DATA-INTENSIVE SCIENCE CHALLENGES The challenge posed by the large data stream from experiments requires a number of responses in scalable data analytics, data-intensive computing, networking, and new  ... 
arXiv:1311.2841v1 fatcat:pqhwoukpezairkrutstjcb3sk4

Using Visual Analytics to Develop Situation Awareness in Astrophysics

Cecilia R. Aragon, Sarah S. Poon, Gregory S. Aldering, Rollin C. Thomas, Robert Quimby
2009 Information Visualization  
We present a novel collaborative visual analytics application for cognitively overloaded users in the astrophysics domain.  ...  Sunfall Data Taking has been in production use for 2 years by a major international astrophysics collaboration (the largest data volume supernova search currently in operation), and has substantially improved  ...  Acknowledgements We thank the anonymous reviewers for their thoughtful suggestions, and the scientists of the SNfactory collaboration for their time and detailed feedback.  ... 
doi:10.1057/ivs.2008.30 fatcat:pypcrm4fwbawjiqgqcs5w5vtdm

The competition of electron and ion heating during magnetic reconnection

C. C. Haggerty, M. A. Shay, J. F. Drake, T. D. Phan, C. T. McHugh
2015 Geophysical Research Letters  
We demonstrate that the development of a large-scale parallel electric field and its associated potential controls the relative heating of electrons and ions.  ...  Unexpectedly, the magnitude of the potential and therefore the relative partition of energy between electrons and ions is not a constant but rather depends on the upstream parameters and specifically the  ...  The magnitude of this potential is consistent with a simple analytic form dependent on upstream and exhaust values.  ... 
doi:10.1002/2015gl065961 fatcat:xea5mymr3fejrfxsyartgwjmkq

Using visual analytics to maintain situation awareness in astrophysics

Cecilia R. Aragon, Sarah S. Poon, Gregory S. Aldering, Rollin C. Thomas, Robert Quimby
2008 2008 IEEE Symposium on Visual Analytics Science and Technology  
We present a novel collaborative visual analytics application for cognitively overloaded users in the astrophysics domain.  ...  Sunfall Data Taking has been in use for over eighteen months by a major international astrophysics collaboration (the largest data volume supernova search currently in operation), and has substantially  ...  ACKNOWLEDGMENTS We would like to thank the anonymous reviewers for their thoughtful suggestions, and the scientists of the SNfactory collaboration for their time and detailed feedback.  ... 
doi:10.1109/vast.2008.4677353 dblp:conf/ieeevast/AragonPATQ08 fatcat:577du5rgizd6vithokc74hgr2q

Absolute and convective instabilities of parallel propagating circularly polarized Alfvén waves: numerical results

D. Simpson, M. S. Ruderman, R. Erdélyi
2006 Astronomy and Astrophysics  
The implication of these results on the interpretation of observational data from space missions is discussed.  ...  , and calculate the dependences of U l and U r on the dimensionless wave amplitude a.  ...  Kéray for patient encouragement and acknowledges the support from the NSF Hungary (OTKA, TO43741).  ... 
doi:10.1051/0004-6361:20053595 fatcat:gaydvhjetbbgdlbbmzyeacmsma

Big Data Technologies And Infrastructures

Sebnem Rusitschka, Alejandro Ramirez
2014 Zenodo  
Big Data Technologies and Infrastructures. Deliverable D1.4 BYTE Project.  ...  Since then every three years a new wave of technological capabilities evolves: waves of i) batch parallel processing, ii) incremental distributed processing, and iii) real-time stream processing of data  ...  data discovery, visual analytics, machine learning, information retrieval, and data mining, the incremental understanding of the data becomes possible.  ... 
doi:10.5281/zenodo.49166 fatcat:wn4umt6tibc6pmiz5zkibogvzm

A Distributed Learning Architecture for Scientific Imaging Problems [article]

A. Panousopoulou, S. Farrens, K. Fotiadou, A. Woiselle, G. Tsagkatakis, J-L. Starck, P. Tsakalides
2018 arXiv   pre-print
Current trends in scientific imaging are challenged by the emerging need of integrating sophisticated machine learning with Big Data analytics platforms.  ...  Ultimately, the offered discussion provides useful practical insights on the impact of key Spark tuning parameters on the speedup achieved, and the memory/disk footprint.  ...  These programming models are instantiated on dedicated platforms for big data analytics.  ... 
arXiv:1809.05956v2 fatcat:iqzxocxplvd6xpxzut4wosxwgi

Ion heating resulting from pickup in magnetic reconnection exhausts

J. F. Drake, M. Swisdak, T. D. Phan, P. A. Cassak, M. A. Shay, S. T. Lepri, R. P. Lin, E. Quataert, T. H. Zurbuchen
2009 Journal of Geophysical Research  
The detailed dynamics of these particles are explored by taking 1-D cuts of the simulation data across the exhaust, transforming to the deHoffman-Teller frame, and calculating explicitly the increment  ...  1] The heating of ions downstream of the x-line during magnetic reconnection is explored using full-particle simulations, test particle simulations, and analytic analysis.  ...  exhaust, including the proton exhaust velocity Dv pL , the ion species, the predicted temperature increment per nucleon DT th , and the measured temperature increment per nucleon DT obs . streaming ion  ... 
doi:10.1029/2008ja013701 fatcat:kseprlqfxfhj7nkvagkzhzvb2a

FlexQuery: An online query system for interactive remote visual data exploration at large scale

Hongbo Zou, Karsten Schwan, Magdalena Slawinska, Matt Wolf, Greg Eisenhauer, Fang Zheng, Jai Dayal, Jeremy Logan, Qing Liu, Scott Klasky, Tanja Bode, Michael Clark (+1 others)
2013 2013 IEEE International Conference on Cluster Computing (CLUSTER)  
Using a real-world scientific application -the Maya astrophysics code and its analytics workflow -we demonstrate FlexQuery's ability to dynamically deploy queries for low-latency remote data visualization  ...  We offer the FlexQuery online data query system that can deploy and execute data queries 'along' the I/O and analytics pipelines.  ...  In the case of the Maya astrophysics analytics workflow [24] , up to 40% of the end-to-end delay experienced with the current analytics and I/O pipelines, is spent on data transfer between different components  ... 
doi:10.1109/cluster.2013.6702635 dblp:conf/cluster/ZouSSWEZDLLKBCK13 fatcat:srvykezodbgodp65kdohn3ejkm

InfiniCortex - From Proof-of-concept to Production

2017 Supercomputing Frontiers and Innovations  
Examples abound: i) transfer of astrophysical data collected by the Square Kilometre Array to the international partners, ii) streaming of large facilities experimental data through the Pacic Research  ...  Platform collaboration of DoE, ESnet and other partners in the US and elsewhere, iii) the Supercilities vision expressed by DoE, iv) new architecture for CERN LHC data processing pipeline focussing on  ...  Acknowledgements Authors would like to acknowledge all the partners around the world that supported and worked hard to make the InniCortex project a success.  ... 
doi:10.14529/jsfi170207 fatcat:wiydbjqynbaqneqpbnplbwvgqy

Shock Acceleration of Solar Energetic Protons: The First 10 Minutes

C. K. Ng, D. V. Reames
2008 Astrophysical Journal  
Proton acceleration at a parallel coronal shock is modeled with self-consistent Alfvén wave excitation and shock transmission. 18 -50 keV seed protons at 0.1% of plasma proton density are accelerated in  ...  Shock acceleration is not significantly retarded by sunward streaming protons interacting with downstream waves. There is no significant second-order Fermi acceleration.  ...  Reames & Ng (1998) inferred streaming-limited SEP intensities and their radial dependence from Helios and GOES data.  ... 
doi:10.1086/592996 fatcat:cpxhsegbdfg2bdzuk4fmoowdga
« Previous Showing results 1 — 15 out of 1,471 results