Filters








20 Hits in 0.99 sec

DLHub: Model and Data Serving for Science [article]

Ryan Chard, Zhuozhao Li, Kyle Chard, Logan Ward, Yadu Babuji, Anna Woodard, Steve Tuecke, Ben Blaiszik, Michael J. Franklin, Ian Foster
2018 arXiv   pre-print
Here we present the Data and Learning Hub for science (DLHub), a multi-tenant system that provides both model repository and serving capabilities with a focus on science applications.  ...  Unlike other model serving frameworks, DLHub can store and serve any Python 3-compatible model or processing function, plus multiple-function pipelines.  ...  We thank Amazon Web Services for research credits and Argonne for computing resources.  ... 
arXiv:1811.11213v1 fatcat:zmmandowgnco7n3b5rqd3nao5q

A Data Ecosystem to Support Machine Learning in Materials Science [article]

Ben Blaiszik, Logan Ward, Marcus Schwarting, Jonathon Gaff, Ryan Chard, Daniel Pike, Kyle Chard, Ian Foster
2019 arXiv   pre-print
Here, we present two projects, the Materials Data Facility (MDF) and the Data and Learning Hub for Science (DLHub), that address these needs.  ...  We use examples to show how MDF and DLHub capabilities can be leveraged to link data with machine learning models and how users can access those capabilities through web and programmatic interfaces.  ...  Much as MDF connects data providers and consumers across the materials science community, DLHub [3] connects data with reusable data transformation and model serving capabilities, allowing producers  ... 
arXiv:1904.10423v1 fatcat:gibssrxzxbaqxdjsfa4ld4gd2a

Confluence of Artificial Intelligence and High Performance Computing for Accelerated, Scalable and Reproducible Gravitational Wave Detection [article]

E. A. Huerta, Asad Khan, Xiaobo Huang, Minyang Tian, Maksim Levental, Ryan Chard, Wei Wei, Maeve Heflin, Daniel S. Katz, Volodymyr Kindratenko, Dawei Mu, Ben Blaiszik (+1 others)
2020 arXiv   pre-print
We develop a workflow that connects the Data and Learning Hub for Science (DLHub), a repository for publishing machine learning models, with the Hardware Accelerated Learning (HAL) deep learning computing  ...  In this work, we demonstrate how connecting recently deployed DOE and NSF-sponsored cyberinfrastructure allows for new ways to publish models, and to subsequently deploy these models into applications  ...  DLHub is a system that provides model repository and model serving facilities, in particular for machine-learning models with science applications.  ... 
arXiv:2012.08545v1 fatcat:da43bh6mxrbuhkxymyvqn6danm

Federated Function as a Service for eScience

Yadu Babuji, Josh Bryan, Ryan Chard, Kyle Chard, Ian Foster, Ben Galewsky, Daniel S. Katz, Zhuozhao Li
2021 Zenodo  
This deposit contains a preprint of a poster paper, and the poster itself, for "Federated Function as a Service for eScience" as submitted and peer-reviewed by the 17th IEEE International Conference on  ...  This is in part due to barriers such as the need to access large research data, diverse hardware requirements, monolithic code bases, and existing systems available to researchers.  ...  The Data and Learning Hub for science for science (DLHub) [6] is a multi-tenant system that provides both a model repository and model serving capabilities with a focus on science applications.  ... 
doi:10.5281/zenodo.5519583 fatcat:7avwlncw4ngfje4ar4pdaabzzy

Infrastructure for Analysis of Large Microscopy and Microanalysis Data Sets

Jingrui Wei, Carter Francis, Dane Morgan, KJ Schmidt, Aristana Scourtas, Ian Foster, Ben Blaiszik, Paul M. Voyles
2022 Microscopy and Microanalysis  
Foundry unifies two existing services: the Materials Data Facility, which offers large scale storage and permanent accessibility to data, and DLHub, which publishes and executes ML models.  ...  FAIR is the emerging standard for open science, and FAIR ML models are more likely to find use and impact outside the group that developed them.  ... 
doi:10.1017/s1431927622011539 fatcat:3fjc7fxadfbopmvisnj53i4pby

Parsl & funcX: Build and test challenges

Daniel S. Katz, Yadu Babuj, Josh Bryan, Kyle Chard, Ryan Chard, Ben Clifford, Ian Foster, Ben Galewsky, Zhuozhao Li, Kirill Nagairtsev, Stephen Rosen, Mike Wilde
2022 Zenodo  
A panel presentation in SIAM PP22, in the session PD3: Build, Integration and Testing for Sustainable Scientific Computing Software, 25 February 2022.  ...  either distribute metadata extraction tasks to data or bring data to the cloud & act on it Machine Learning inference (DLHub) • Performs on-demand machine learning inference tasks, with inference requests  ...  for others -We can't easily do this by ourselves -we need systemwide changes Turn any machine into a function serving endpoint Remove barriers to using diverse and distributed infrastructure Functions  ... 
doi:10.5281/zenodo.6210785 fatcat:mijfhlvt7raatfa4ntt37f5qem

The Manufacturing Data and Machine Learning Platform: Enabling Real-time Monitoring and Control of Scientific Experiments via IoT [article]

Jakob R. Elias, Ryan Chard, Joseph A. Libera, Ian Foster, Santanu Chaudhuri
2020 arXiv   pre-print
In addition, new tools and technologies are required to facilitate the collection, aggregation, and manipulation of sensor data in order to simplify the application of ML models and in turn, fully realize  ...  We will show that MDML is capable of processing diverse IoT data streams, using multiple computing resources, and integrating ML models to guide an experiment.  ...  Department of Energy, Office of Science, under contract DE-AC02-06CH11357.  ... 
arXiv:2005.13669v1 fatcat:z25snzwxbfejxdnclsyp3pa7ly

It's the Best Only When It Fits You Most: Finding Related Models for Serving Based on Dynamic Locality Sensitive Hashing [article]

Lixi Zhou, Zijie Wang, Amitabh Das, Jia Zou
2020 arXiv   pre-print
This paper proposes an end-to-end process of searching related models for serving based on the similarity of the target dataset and the training datasets of the available models.  ...  Reusing models for inferencing a dataset can greatly save the human costs required for training data creation.  ...  We propose and implement an end-to-end system, called ModelHub, to efficiently automate the process of finding related models for serving based on various techniques including LSH for JS-divergence, the  ... 
arXiv:2010.09474v1 fatcat:d7x7zsfcdfhpppja6sefcjjbli

Open-Source Biomedical Image Analysis Models: A Meta-Analysis and Continuous Survey

Rui Li, Vaibhav Sharma, Subasini Thangamani, Artur Yakimovich
2022 Frontiers in Bioinformatics  
To understand how these requirements are met, we have collected 50 biomedical image analysis models and performed a meta-analysis of their respective papers, source code, dataset, and trained model parameters  ...  A multitude of open-source platforms drive image analysis pipelines and help disseminate novel analytical approaches and algorithms.  ...  Authors opting for self-hosting of model parameters also likely underestimate the workload of the long-term serving of archival data.  ... 
doi:10.3389/fbinf.2022.912809 fatcat:wworclitpfeyxnt6hx6676azya

Parsl: Pervasive Parallel Programming in Python [article]

Yadu Babuji, Anna Woodard, Zhuozhao Li, Daniel S. Katz, Ben Clifford, Rohan Kumar, Lukasz Lacinski, Ryan Chard, Justin M. Wozniak, Ian Foster, Michael Wilde, Kyle Chard
2019 arXiv   pre-print
This migration towards orchestration rather than implementation, coupled with the growing need for parallel computing (e.g., due to big data and the end of Moore's law), necessitates rethinking how parallelism  ...  We show that these capabilities satisfy the needs of many-task, interactive, online, and machine learning applications in fields such as biology, cosmology, and materials science.  ...  It relied on the Blue Waters sustainedpetascale computing project, which is supported by the National Science Foundation (OCI-0725070, ACI-1238993) and the State of Illinois.  ... 
arXiv:1905.02158v1 fatcat:okcga7i4vza6zmx5lyj63seone

ModelHub.AI: Dissemination Platform for Deep Learning Models [article]

Ahmed Hosny, Michael Schwier, Christoph Berger, Evin P Örnek, Mehmet Turan, Phi V Tran, Leon Weninger, Fabian Isensee, Klaus H Maier-Hein, Richard McKinley, Michael T Lu, Udo Hoffmann (+4 others)
2019 arXiv   pre-print
For contributors, the engine controls data flow throughout the inference cycle, while the contributor-facing standard template exposes model-specific functions including inference, as well as pre- and  ...  ModelHub.AI is domain-, data-, and framework-agnostic, catering to different workflows and contributors' preferences.  ...  This will involve the integration of user authentication, load-balancing, auto-healing, and other features needed for serving models at scale.  ... 
arXiv:1911.13218v1 fatcat:4b4mdu4frzaozlgwaz6kvf4k7e

Benchmark of DNN Model Search at Deployment Time [article]

Lixi Zhou, Arindam Jain, Zijie Wang, Amitabh Das, Yingzhen Yang, Jia Zou
2022 arXiv   pre-print
We are lacking a highly productive model search tool that selects models for deployment without the need for any manual inspection and/or labeled data from the target domain.  ...  and model downloaders to screen keyword search results for selecting a model.  ...  [3] , and DLHub [1] .  ... 
arXiv:2206.00188v1 fatcat:3tawlkzhwzebhbojr76f6uuuze

RIBBON: Cost-Effective and QoS-Aware Deep Learning Model Inference using a Diverse Pool of Cloud Computing Instances [article]

Baolin Li, Rohan Basu Roy, Tirthak Patel, Vijay Gadepally, Karen Gettings, Devesh Tiwari
2022 arXiv   pre-print
RIBBON saves up to 16% of the inference service cost for different learning models including emerging deep learning recommender system models and drug-discovery enabling models.  ...  RIBBON devises a Bayesian Optimization-driven strategy that helps users build the optimal set of heterogeneous instances for their model inference service needs on cloud computing platforms -- and, RIBBON  ...  VGG [75] A CNN model available on DLHUB [76] . Widely applied in image recognition areas. MT-WND [77] Multi-Task Wide and Deep, a recommendation model.  ... 
arXiv:2207.11434v1 fatcat:t5p6sqigizfftkyfpcgimedupq

TomoGAN: Low-Dose Synchrotron X-Ray Tomography with Generative Adversarial Networks [article]

Zhengchun Liu, Tekin Bicer, Rajkumar Kettimuthu, Doga Gursoy, Francesco De Carlo, Ian Foster
2019 arXiv   pre-print
data from 0.18 to 0.9 and from 0.18 to 0.41, respectively.  ...  We present , a denoising technique based on generative adversarial networks, for improving the quality of reconstructed images for low-dose imaging conditions.  ...  We gratefully acknowledge the computing resources provided and operated by the Joint Laboratory for System Evaluation at Argonne National Laboratory.  ... 
arXiv:1902.07582v4 fatcat:wgk23emajbbidc4hwyivqaog2a

funcX: A Federated Function Serving Fabric for Science [article]

Ryan Chard, Yadu Babuji, Zhuozhao Li, Tyler Skluzacek, Anna Woodard, Ben Blaiszik, Ian Foster, Kyle Chard
2020 pre-print
Exploding data volumes and velocities, new computational methods and platforms, and ubiquitous connectivity demand new approaches to computation in the sciences.  ...  These new approaches must enable computation to be mobile, so that, for example, it can occur near data, be triggered by events (e.g., arrival of new data), be offloaded to specialized accelerators, or  ...  Department of Energy under Contract DE-AC02-06CH11357 and used resources of the Argonne Leadership Computing Facility.  ... 
doi:10.1145/3369583.3392683 arXiv:2005.04215v1 fatcat:stvzh7eslnf65e3nzbcvl5aizu
« Previous Showing results 1 — 15 out of 20 results