294 Hits in 2.1 sec

Serving deep learning models in a serverless platform [article]

Vatche Ishakian, Vinod Muthusamy, Aleksander Slominski
2018 arXiv   pre-print
In this work we evaluate the suitability of a serverless computing environment for the inferencing of large neural network models.  ...  Our experimental evaluations are executed on the AWS Lambda environment using the MxNet deep learning framework.  ...  In particular we evaluate the performance of serving deep learning models, where a serverless function classifies images by performing a forward pass through the model.  ... 
arXiv:1710.08460v2 fatcat:rqpeh5jro5h3hjizfapbv6nz3y

Serverless Model Serving for Data Science [article]

Yuncheng Wu, Tien Tuan Anh Dinh, Guoyu Hu, Meihui Zhang, Yeow Meng Chee, Beng Chin Ooi
2021 arXiv   pre-print
In this paper, we study the viability of serverless as a mainstream model serving platform for data science applications.  ...  Other findings include a large gap in cold start time between AWS and GCP serverless functions, and serverless' low sensitivity to changes in workloads or models.  ...  CONCLUSIONS In this paper, we have conducted a comprehensive performance comparison of serverless against other cloud-based model serving systems from AWS and GCP, using two deep learning models and three  ... 
arXiv:2103.02958v1 fatcat:mdfnswimkfce3dxan6zjrliyfm

Serving Machine Learning Workloads in Resource Constrained Environments: a Serverless Deployment Example

Angelos Christidis, Roy Davies, Sotiris Moschoyiannis
2019 2019 IEEE 12th Conference on Service-Oriented Computing and Applications (SOCA)  
in a serverless environment; without compromising capability or performance.  ...  In this paper we propose a set of optimization techniques and show how these transform a codebase which was previously incompatible with a serverless deployment into one that can be successfully deployed  ...  be used when serving any type of model in a serverless environment.  ... 
doi:10.1109/soca.2019.00016 dblp:conf/soca/ChristidisDM19 fatcat:fgtjs26fmfgzhaj5ikkjposspi

Privacy-Preserving Serverless Edge Learning with Decentralized Small Data [article]

Shih-Chun Lin, Chia-Hung Lin
2021 arXiv   pre-print
This paper extends conventional serverless platforms with serverless edge learning architectures and provides an efficient distributed training framework from the networking perspective.  ...  Distributed training strategies have recently become a promising approach to ensure data privacy when training deep models.  ...  As shown in Fig. 5 , we set a deep learning model with three layers as the baseline model for comparison [14] .  ... 
arXiv:2111.14955v2 fatcat:cofiguye4fb4nm3fj7tpt7ghr4

Enabling Serverless Deployment of Large-Scale AI Workloads

Angelos Christidis, Sotiris Moschoyiannis, Ching-Hsien Hsu, Roy Davies
2020 IEEE Access  
It is even more important in a serverless environment due to the stringent constraints on serverless functions' runtime before timeout.  ...  We propose a set of optimization techniques for transforming a generic AI codebase so that it can be successfully deployed to a restricted serverless environment, without compromising capability or performance  ...  be used when serving any type of model in a serverless environment.  ... 
doi:10.1109/access.2020.2985282 fatcat:pjyqfqhjq5germhr4zxfufixg4

BARISTA: Efficient and Scalable Serverless Serving System for Deep Learning Prediction Services [article]

Anirban Bhattacharjee, Ajay Dev Chhokra, Zhuangwei Kang, Hongyang Sun, Aniruddha Gokhale, Gabor Karsai
2019 arXiv   pre-print
The stateless and highly parallelizable nature of deep learning models makes them well-suited for serverless computing paradigm.  ...  To address these challenges, we present a distributed and scalable deep-learning prediction serving system called Barista and make the following contributions.  ...  ACKNOWLEDGMENT This work was supported in part by NSF US Ignite CNS 1531079, AFOSR DDDAS FA9550-18-1-0126 and AFRL/Lockheed Martin's StreamlinedML program.  ... 
arXiv:1904.01576v2 fatcat:jzhpvqzdsbanzeqhdnwizj6f6q

Understanding Characteristics of Commodity Serverless Computing Platforms [article]

Jinfeng Wen, Yi Liu, Zhenpeng Chen, Yun Ma, Haoyu Wang, Xuanzhe Liu
2021 arXiv   pre-print
Due to the great prospects of serverless computing, in recent years, most major cloud vendors have rolled out their commodity serverless computing platforms.  ...  To fill this knowledge gap, this paper presents a comprehensive study on characterizing mainstream commodity serverless computing platforms (i.e., AWS Lambda, Azure Functions, Google Cloud Functions, and  ...  ML Training & Serving. We also provide workloads that deal with machine learning (ML) tasks using serverless computing. Such workloads mainly involve with the ML model training and ML model serving.  ... 
arXiv:2012.00992v2 fatcat:l2d6hgaa4vcxnjmfeznvnxygem

Distributed Double Machine Learning with a Serverless Architecture [article]

Malte S. Kurz
2021 arXiv   pre-print
We provide a prototype Python implementation DoubleML-Serverless for the estimation of double machine learning models with the serverless computing platform AWS Lambda and demonstrate its utility with  ...  This paper explores serverless cloud computing for double machine learning.  ...  Besides that, serverless computing is getting more and more adopted for various machine learning tasks, like for example to serve deep learning models [14, 24, 40] or more generally for ML model training  ... 
arXiv:2101.04025v2 fatcat:jl72mzsnvraudgge746hkadvua

Towards Designing a Self-Managed Machine Learning Inference Serving System inPublic Cloud [article]

Jashwant Raj Gunasekaran, Prashanth Thinakaran, Cyan Subhra Mishra, Mahmut Taylan Kandemir, Chita R. Das
2020 arXiv   pre-print
In addition, wecomprehensively evaluate prior work which tries to achievecost-effective prediction-serving.  ...  In orderto solve this complex optimization problem, we explore the highlevel design of a reinforcement-learning based system that canefficiently adapt to the changing needs of the system at scale.  ...  In addition, we design a scheme named Paragon on top of AWS platform, which incorporates some of the proposed design choices.  ... 
arXiv:2008.09491v1 fatcat:c6shxhqvzjgjjkpxaknsnnu2iq

Efficiency in the Serverless Cloud Computing Paradigm: A Survey Study [article]

Chavit Denninnart, Mohsen Amini Salehi
2021 arXiv   pre-print
Serverless computing along with Function-as-a-Service (FaaS) are forming a new computing paradigm that is anticipated to found the next generation of cloud systems.  ...  The popularity of this paradigm is due to offering a highly transparent infrastructure that enables user applications to scale in the granularity of their functions.  ...  Their use case employs a deep neural network (DNN) model for video object classification.  ... 
arXiv:2110.06508v1 fatcat:gp7dxqmmavfbhf7n5bssws2tje

Beyond Microbenchmarks: The SPEC-RG Vision for a Comprehensive Serverless Benchmark

Erwin van Eyk, Joel Scheuner, Simon Eismann, Cristina L. Abad, Alexandru Iosup
2020 Companion of the ACM/SPEC International Conference on Performance Engineering  
ACKNOWLEDGMENTS The work presented in this article has benefited from discussions within the SPEC-RG Cloud Group, and further in the Cloud Control Workshop series organized by Erik Elmroth and his team  ...  Serverless platforms are, in most cases, not intended as standalone systems. Instead, they provide deep integrations with other cloud services, such as integrations with event sources.  ...  as machine learning, and graph processing.  ... 
doi:10.1145/3375555.3384381 dblp:conf/wosp/EykSEAI20 fatcat:jrir4wiyvfa3fhwxbyytnxv74q

Serverless on FHIR: Deploying machine learning models for healthcare on the cloud [article]

Bell Raj Eapen, Kamran Sartipi, Norm Archer
2020 arXiv   pre-print
Machine Learning (ML) plays a vital role in implementing digital health. The advances in hardware and the democratization of software tools have revolutionized machine learning.  ...  We introduce a functional taxonomy and a four-tier architecture for cloud-based model deployment for digital health.  ...  Most platforms used for machine learning will allow the exporting of this representation in a proprietary or open format [17] .  ... 
arXiv:2006.04748v1 fatcat:ivk2yffpynbkndbjry7av5ma3m

A Survey on Serverless Computing [article]

Jacob John, Shashank Gupta
2021 arXiv   pre-print
This paper provides a formal account of the research contributions in the field of Serverless computing.  ...  Event-driven compute services allow users to build more agile applications using capacity provisioning and a pay-for-value billing model.  ...  In Workshop on power aware computing and deep learning models in a serverless platform.  ... 
arXiv:2106.11773v2 fatcat:y7wwn732obcbtodc5h4zw6yvpq

Performance Characterization and Modeling of Serverless and HPC Streaming Applications [article]

Andre Luckow, Shantenu Jha
2019 arXiv   pre-print
In response, we extend Pilot-Streaming to support serverless platforms.  ...  Using experiments on HPC and AWS Lambda, we demonstrate that StreamInsight provides an accurate model for a variety of application characteristics, e.g., machine learning model sizes and resource configurations  ...  ., TomoPy for light sources sciences or deep learning applications, serverless is not suitable. V.  ... 
arXiv:1909.06055v1 fatcat:srn4aojierffzlzlp2hmwm6io4

Cost-effective Deployment of BERT Models in Serverless Environment [article]

Katarína Benešová, Andrej Švec, Marek Šuppa
2021 arXiv   pre-print
As a result, we obtain models that are tuned for a specific domain and deployable in serverless environments.  ...  In this study we demonstrate the viability of deploying BERT-style models to serverless environments in a production setting.  ...  Model STSb Target Model inference engine In order to fit all of the above in a few hundreds of MBs allowed in the serverless environments, standard deep learning libraries cannot be used: the standard  ... 
arXiv:2103.10673v2 fatcat:k6firpsk7jainjd7hpsi4auuqm
« Previous Showing results 1 — 15 out of 294 results