14 Hits in 9.5 sec

Transparent Serverless execution of Python multiprocessing applications [article]

Aitor Arjona and Gerard Finol and Pedro Garcia-Lopez
2022 arXiv   pre-print
This paper presents a performance evaluation where we assess the feasibility of access transparency over state-of-the-art Cloud disaggregated resources for Python multiprocessing applications.  ...  We compare execution time and scalability of the same application running over disaggregated resources using our library, with the single-machine Python libraries in a large VM.  ...  Acknowledgments This work has been partially funded by the EU Horizon 2020 programme under grant agreement No.825184 and by the Spanish Ministry of Science and Innovation and State Research Agency (Agencia  ... 
arXiv:2205.08818v1 fatcat:mwifneoszbbd5b2z32lhbm36bu

Serverless End Game: Disaggregation enabling Transparency [article]

Pedro García-López, Aleksander Slominski, Simon Shillaker, Michael Behrendt, Barnard Metzler
2020 arXiv   pre-print
We demonstrate with two experiments that we can achieve transparency today over disaggregated serverless resources and obtain comparable performance to local executions.  ...  We elaborate in this article why resource disaggregation in serverless computing is the definitive catalyst to enable full transparency in the Cloud.  ...  It demonstrates transparency executing unmodi ed Python applications from the OpenAI Baselines machine learning project.  ... 
arXiv:2006.01251v1 fatcat:sklmosqtvjcppf44rr77okd6oa

Serverless Predictions: 2021-2030 [article]

Pedro Garcia Lopez, Aleksander Slominski, Michael Behrendt, Bernard Metzler
2021 arXiv   pre-print
In this article, we present five serverless predictions for the next decade that will realize this vision of transparency -- equivalent to Tim Wagner's Serverless SuperComputer or AnyScale's Infinite Laptop  ...  Within the next 10 years, advances on resource disaggregation will enable full transparency for most Cloud applications: to run unmodified single-machine applications over effectively unlimited remote  ...  Lithops intercepts Python language libraries (multiprocessing) in order to access remote serverless resources in a transparent way.  ... 
arXiv:2104.03075v1 fatcat:n4sy6hletzawthgicnopcicr6q

A Literature Review on Serverless Computing [article]

Jinfeng Wen, Zhenpeng Chen, Xuanzhe Liu
2022 arXiv   pre-print
The most significant advantage of serverless computing is to free software developers from the burden of complex underlying management tasks and allow them to focus on only the application logic implementation  ...  Finally, based on our analysis, we discuss some key challenges and envision promising opportunities for future research on the serverless platform side, serverless application side, and serverless computing  ...  [183] presented an extensible multi-cloud framework to execute regular Python code in any serverless platform transparently.  ... 
arXiv:2206.12275v3 fatcat:uk5rrpxu5vaixlkudoshyjrq64

Exoshuffle: Large-Scale Shuffle at the Application Level [article]

Frank Sifei Luan, Stephanie Wang, Samyukta Yagati, Sean Kim, Kenneth Lien, Isaac Ong, SangBin Cho, Eric Liang, Ion Stoica
2022 arXiv   pre-print
We show that it is possible to (1) express shuffle algorithms from previous shuffle systems in a few hundred lines of application-level Python code, (2) achieve competitive performance and scalability  ...  Shuffle is a key primitive in large-scale data processing applications. The difficulty of large-scale shuffle has inspired the development of many specialized shuffle systems.  ...  However, serverless functions only abstract remote execution, while a distributed future also abstracts the location of a function's return value.  ... 
arXiv:2203.05072v3 fatcat:y5pxg3jivbaxhohnz2dx3zetbe

MLLess: Achieving Cost Efficiency in Serverless Machine Learning Training [article]

Pablo Gimeno Sarroca, Marc Sánchez-Artigas
2022 arXiv   pre-print
Furthermore, our results show that MLLess can easily scale out to increasingly large fleets of serverless workers.  ...  Function-as-a-Service (FaaS) has raised a growing interest in how to "tame" serverless computing to enable domain-specific use cases such as data-intensive applications and machine learning (ML), to name  ...  the native multiprocessing module available in Python to enable the transparent execution of multiprocessing applications over FaaS platforms.  ... 
arXiv:2206.05786v1 fatcat:2jykgs3w55gbvoij474dexsdva

Architecture and Performance Evaluation of Distributed Computation Offloading in Edge Computing

Claudio Cicconetti, Marco Conti, Andrea Passarella
2019 Simulation modelling practice and theory  
On the other hand, the need for highly scalable execution of stateless tasks for cloud systems is driving the definition of new technologies based on serverless computing.  ...  The proposed framework relies on the use of real components on lightweight virtualization mixed with simulated computation and is well-suited to the analysis of several applications and network environments  ...  Examples include Picasso [7] , from which we reuse the concept of providing the applications with an Application Programming Interface (API) whose routines are executed by the network in a manner transparent  ... 
doi:10.1016/j.simpat.2019.102007 fatcat:e7w3arxeazg6dkdssp2xx4wmbi

Faastlane: Accelerating Function-as-a-Service Workflows

Swaroop Kotni, Ajay Nayak, Vinod Ganapathy, Arkaprava Basu
2021 USENIX Annual Technical Conference  
While threads ease sharing, implementations of languages such as Python and Node.js (widely used in FaaS applications) disallow concurrent execution of threads.  ...  In FaaS workflows, a set of functions implement application logic by interacting and exchanging data among themselves.  ...  This work was supported in part by a Ramanujan Fellowship from the Government of India, a Young Investigator fellowship from Pratiksha Trust, and research gifts from Intel Labs and VMware Inc.  ... 
dblp:conf/usenix/KotniNGB21 fatcat:qxibm654yvg4roeelm2pckytgi

Hindsight Logging for Model Training [article]

Rolando Garcia, Eric Liu, Vikram Sreekanti, Bobby Yan, Anusha Dandamudi, Joseph E. Gonzalez, Joseph M. Hellerstein, Koushik Sen
2020 arXiv   pre-print
The goal of hindsight logging is to enable analysis of past executions as if the logs had been exhaustive.  ...  We implement these ideas in Flor, a record-replay system for hindsight logging in Python.  ...  The Record Phase During the execution of model training, Flor transparently and efficiently checkpoints state with record to speed up replay.  ... 
arXiv:2006.07357v1 fatcat:fpwmpcjom5bj7e2axotgnoo5xu

TANGOS: The Agile Numerical Galaxy Organization System

Andrew Pontzen, Michael Tremmel
2018 Astrophysical Journal Supplement Series  
We present Tangos, a Python framework and web interface for database-driven analysis of numerical structure formation simulations.  ...  At the querying stage, our example of gathering properties over history is reduced to a few clicks or a simple, single-line Python command.  ...  This work used the DiRAC Complexity system, operated by the University of Leicester IT Services, which forms part of the STFC DiRAC HPC Facility (  ... 
doi:10.3847/1538-4365/aac832 fatcat:t2epm4tjv5dlpehzd5ucqumx7u

Machine Learning and Deep Learning frameworks and libraries for large-scale data mining: a survey

Giang Nguyen, Stefan Dlugolinsky, Martin Bobák, Viet Tran, Álvaro López García, Ignacio Heredia, Peter Malík, Ladislav Hluchý
2019 Artificial Intelligence Review  
used by billions of people.  ...  It also provides an overview of massive parallelism support that is capable of scaling computation effectively and efficiently in the era of Big Data.  ...  Giang Nguyen, Viet Tran, Stefan Dlugolinsky, Martin Bobák, and Ladislav Hluchý are also supported by the Project VEGA 2/0167/16 "Methods and algorithms for the semantic processing of Big Data in distributed  ... 
doi:10.1007/s10462-018-09679-z fatcat:ueffoypwlva4ndo35g5gzfrpcy

Separating Protection and Management in Cloud Infrastructures

Zhiming Shen
The containers platform supports efficient container provisioning and executing, but does not provide sufficient security and flexibility.  ...  The Department of Computer Science at Cornell is a warm community.  ...  Serverless computing is a cloud computing execution model in which clouds provide runtime environments to execute user programs on-demand, and dynamically manage the allocation of physical resources [92  ... 
doi:10.7298/x44x5606 fatcat:opqx4t2g65avvidhasocc3x2p4

Protecting applications using trusted execution environments

Christian Priebe, Peter Pietzuch, Engineering And Physical Sciences Research Council
SGX-LKL provides transparent protection of the host interface and for both disk and network I/O.  ...  However, designing applications for the execution within TEEs requires careful consideration of the elevated threats that come with running in a fully untrusted environment.  ...  ACKNOWLEDGEMENTS First of all, I would like to thank Prof. Peter Pietzuch for his supervision, guidance, and encouragement throughout the last few years.  ... 
doi:10.25560/84843 fatcat:5iei362jgjc2xjwjpesxogp4oy

Scheduling techniques for efficient execution of stream workflows in cloud environments

MSM Barika
As a consequence, the execution of stream workflow applications o [...]  ...  The use of cloud computing technology which can provide on demand and elastic resources to execute stream workflow applications is ideal, but additional challenges are raised due to the location of data  ...  As an execution framework, it handles the lifetime of resources transparently and provisions resources on demand to reduce latency while at the same time maintaining high utilisation efficiency.  ... 
doi:10.25959/100.00034812 fatcat:4wq3wizftffvjbo2jpp2dwpepi