A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Filters
DLHub: Model and Data Serving for Science
[article]
2018
arXiv
pre-print
Preserved Fulltext
Here we present the Data and Learning Hub for science (DLHub), a multi-tenant system that provides both model repository and serving capabilities with a focus on science applications. ...
Unlike other model serving frameworks, DLHub can store and serve any Python 3-compatible model or processing function, plus multiple-function pipelines. ...
We thank Amazon Web Services for research credits and Argonne for computing resources. ...
arXiv:1811.11213v1
fatcat:zmmandowgnco7n3b5rqd3nao5q
A Data Ecosystem to Support Machine Learning in Materials Science
[article]
2019
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2019; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Here, we present two projects, the Materials Data Facility (MDF) and the Data and Learning Hub for Science (DLHub), that address these needs. ...
We use examples to show how MDF and DLHub capabilities can be leveraged to link data with machine learning models and how users can access those capabilities through web and programmatic interfaces. ...
Much as MDF connects data providers and consumers across the materials science community, DLHub [3] connects data with reusable data transformation and model serving capabilities, allowing producers ...
arXiv:1904.10423v1
fatcat:gibssrxzxbaqxdjsfa4ld4gd2a
Confluence of Artificial Intelligence and High Performance Computing for Accelerated, Scalable and Reproducible Gravitational Wave Detection
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
Note that this fulltext copy is not of the "primary" version of this work. The version it corresponds to is:
2021 accepted arXiv:2012.08545v2 fatcat:z6tv4mgxnvedhloontaxdi5rmy
We develop a workflow that connects the Data and Learning Hub for Science (DLHub), a repository for publishing machine learning models, with the Hardware Accelerated Learning (HAL) deep learning computing ...
In this work, we demonstrate how connecting recently deployed DOE and NSF-sponsored cyberinfrastructure allows for new ways to publish models, and to subsequently deploy these models into applications ...
DLHub is a system that provides model repository and model serving facilities, in particular for machine-learning models with science applications. ...
arXiv:2012.08545v1
fatcat:da43bh6mxrbuhkxymyvqn6danm
Federated Function as a Service for eScience
2021
Zenodo
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
This deposit contains a preprint of a poster paper, and the poster itself, for "Federated Function as a Service for eScience" as submitted and peer-reviewed by the 17th IEEE International Conference on ...
This is in part due to barriers such as the need to access large research data, diverse hardware requirements, monolithic code bases, and existing systems available to researchers. ...
The Data and Learning Hub for science for science (DLHub) [6] is a multi-tenant system that provides both a model repository and model serving capabilities with a focus on science applications. ...
doi:10.5281/zenodo.5519583
fatcat:7avwlncw4ngfje4ar4pdaabzzy
Infrastructure for Analysis of Large Microscopy and Microanalysis Data Sets
2022
Microscopy and Microanalysis
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Foundry unifies two existing services: the Materials Data Facility, which offers large scale storage and permanent accessibility to data, and DLHub, which publishes and executes ML models. ...
FAIR is the emerging standard for open science, and FAIR ML models are more likely to find use and impact outside the group that developed them. ...
doi:10.1017/s1431927622011539
fatcat:3fjc7fxadfbopmvisnj53i4pby
Parsl & funcX: Build and test challenges
2022
Zenodo
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
A panel presentation in SIAM PP22, in the session PD3: Build, Integration and Testing for Sustainable Scientific Computing Software, 25 February 2022. ...
either distribute metadata extraction tasks to data or bring data to the cloud & act on it Machine Learning inference (DLHub) • Performs on-demand machine learning inference tasks, with inference requests ...
for others -We can't easily do this by ourselves -we need systemwide changes Turn any machine into a function serving endpoint Remove barriers to using diverse and distributed infrastructure Functions ...
doi:10.5281/zenodo.6210785
fatcat:mijfhlvt7raatfa4ntt37f5qem
The Manufacturing Data and Machine Learning Platform: Enabling Real-time Monitoring and Control of Scientific Experiments via IoT
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
In addition, new tools and technologies are required to facilitate the collection, aggregation, and manipulation of sensor data in order to simplify the application of ML models and in turn, fully realize ...
We will show that MDML is capable of processing diverse IoT data streams, using multiple computing resources, and integrating ML models to guide an experiment. ...
Department of Energy, Office of Science, under contract DE-AC02-06CH11357. ...
arXiv:2005.13669v1
fatcat:z25snzwxbfejxdnclsyp3pa7ly
It's the Best Only When It Fits You Most: Finding Related Models for Serving Based on Dynamic Locality Sensitive Hashing
[article]
2020
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
This paper proposes an end-to-end process of searching related models for serving based on the similarity of the target dataset and the training datasets of the available models. ...
Reusing models for inferencing a dataset can greatly save the human costs required for training data creation. ...
We propose and implement an end-to-end system, called ModelHub, to efficiently automate the process of finding related models for serving based on various techniques including LSH for JS-divergence, the ...
arXiv:2010.09474v1
fatcat:d7x7zsfcdfhpppja6sefcjjbli
Open-Source Biomedical Image Analysis Models: A Meta-Analysis and Continuous Survey
2022
Frontiers in Bioinformatics
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
To understand how these requirements are met, we have collected 50 biomedical image analysis models and performed a meta-analysis of their respective papers, source code, dataset, and trained model parameters ...
A multitude of open-source platforms drive image analysis pipelines and help disseminate novel analytical approaches and algorithms. ...
Authors opting for self-hosting of model parameters also likely underestimate the workload of the long-term serving of archival data. ...
doi:10.3389/fbinf.2022.912809
fatcat:wworclitpfeyxnt6hx6676azya
Parsl: Pervasive Parallel Programming in Python
[article]
2019
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
This migration towards orchestration rather than implementation, coupled with the growing need for parallel computing (e.g., due to big data and the end of Moore's law), necessitates rethinking how parallelism ...
We show that these capabilities satisfy the needs of many-task, interactive, online, and machine learning applications in fields such as biology, cosmology, and materials science. ...
It relied on the Blue Waters sustainedpetascale computing project, which is supported by the National Science Foundation (OCI-0725070, ACI-1238993) and the State of Illinois. ...
arXiv:1905.02158v1
fatcat:okcga7i4vza6zmx5lyj63seone
ModelHub.AI: Dissemination Platform for Deep Learning Models
[article]
2019
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
For contributors, the engine controls data flow throughout the inference cycle, while the contributor-facing standard template exposes model-specific functions including inference, as well as pre- and ...
ModelHub.AI is domain-, data-, and framework-agnostic, catering to different workflows and contributors' preferences. ...
This will involve the integration of user authentication, load-balancing, auto-healing, and other features needed for serving models at scale. ...
arXiv:1911.13218v1
fatcat:4b4mdu4frzaozlgwaz6kvf4k7e
Benchmark of DNN Model Search at Deployment Time
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
We are lacking a highly productive model search tool that selects models for deployment without the need for any manual inspection and/or labeled data from the target domain. ...
and model downloaders to screen keyword search results for selecting a model. ...
[3] , and DLHub [1] . ...
arXiv:2206.00188v1
fatcat:3tawlkzhwzebhbojr76f6uuuze
RIBBON: Cost-Effective and QoS-Aware Deep Learning Model Inference using a Diverse Pool of Cloud Computing Instances
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
2021
Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis
unpublished
doi:10.1145/3458817.3476168 fatcat:thcl5bv2ojbglowg52jxb47eue
doi:10.1145/3458817.3476168 fatcat:thcl5bv2ojbglowg52jxb47eue
RIBBON saves up to 16% of the inference service cost for different learning models including emerging deep learning recommender system models and drug-discovery enabling models. ...
RIBBON devises a Bayesian Optimization-driven strategy that helps users build the optimal set of heterogeneous instances for their model inference service needs on cloud computing platforms -- and, RIBBON ...
VGG [75] A CNN model available on DLHUB [76] . Widely applied in image recognition areas. MT-WND [77] Multi-Task Wide and Deep, a recommendation model. ...
arXiv:2207.11434v1
fatcat:t5p6sqigizfftkyfpcgimedupq
TomoGAN: Low-Dose Synchrotron X-Ray Tomography with Generative Adversarial Networks
[article]
2019
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
data from 0.18 to 0.9 and from 0.18 to 0.41, respectively. ...
We present , a denoising technique based on generative adversarial networks, for improving the quality of reconstructed images for low-dose imaging conditions. ...
We gratefully acknowledge the computing resources provided and operated by the Joint Laboratory for System Evaluation at Argonne National Laboratory. ...
arXiv:1902.07582v4
fatcat:wgk23emajbbidc4hwyivqaog2a
funcX: A Federated Function Serving Fabric for Science
[article]
2020
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Exploding data volumes and velocities, new computational methods and platforms, and ubiquitous connectivity demand new approaches to computation in the sciences. ...
These new approaches must enable computation to be mobile, so that, for example, it can occur near data, be triggered by events (e.g., arrival of new data), be offloaded to specialized accelerators, or ...
Department of Energy under Contract DE-AC02-06CH11357 and used resources of the Argonne Leadership Computing Facility. ...
doi:10.1145/3369583.3392683
arXiv:2005.04215v1
fatcat:stvzh7eslnf65e3nzbcvl5aizu
« Previous
Showing results 1 — 15 out of 20 results