Filters








857 Hits in 2.3 sec

Approaches for containerized scientific workflows in cloud environments with applications in life science

Ola Spjuth, Marco Capuccini, Matteo Carone, Anders Larsson, Wesley Schaal, Jon Ander Novella, Oliver Stein, Morgan Ekmefjord, Paolo Di Tommaso, Evan Floden, Cedric Notredame, Pablo Moreno (+5 others)
2021 F1000Research  
Scientific workflows are ideal for chaining containers into data analysis pipelines to aid in creating reproducible analyses.  ...  Containers are gaining popularity in life science research as they provide a solution for encompassing dependencies of provisioned tools, simplify software installations for end users and offer a form  ...  Error management for containerized workflows is an important concept.  ... 
doi:10.12688/f1000research.53698.1 fatcat:qy3gnqbsavcctctanch25ukgie

Building Containerized Environments for Reproducibility and Traceability of Scientific Workflows [article]

Paula Olaya, Jay Lofstead, Michela Taufer
2020 arXiv   pre-print
We assess our environment on four different workflows and measure containerization costs in terms of time and space.  ...  Our solution consists of a containerized environment capable to annotate workflows, capture provenance metadata, and build record trails.  ...  For space overhead, we mean the memory size of the workflow components (i.e., data and application) when containerized.  ... 
arXiv:2009.08495v1 fatcat:7byxzxywmncivkph4ngdtcglsu

Scalable Workflow-Driven Hydrologic Analysis in HydroFrame [chapter]

Shweta Purawat, Cathie Olschanowsky, Laura E. Condon, Reed Maxwell, Ilkay Altintas
2020 Lecture Notes in Computer Science  
We present the initial progress on the HydroFrame community platform using an automated Kepler workflow.  ...  We demonstrate how different modules of the workflow can be reused and repurposed for the three target user groups.  ...  This work is supported by NSF OAC CSSI 1835855, and DOE DE-SC0012630 for IPPD.  ... 
doi:10.1007/978-3-030-50371-0_20 fatcat:c6632vj24nd25dcdq73i6bxhyu

SecDocker: Hardening the Continuous Integration Workflow [article]

David Fernández González, Francisco Javier Rodríguez Lera, Gonzalo Esteban, Camino Fernández Llamas
2021 arXiv   pre-print
This paper presents an overview of current security issues in CI workflow.  ...  The tool, called \SD for its Docker-based approach, is publicly available in GitHub.  ...  Handles packages at the TCP level, so the communication looks transparent for the end-user. It also maintains the communications and gathers data for the HTTPServer module.  ... 
arXiv:2104.07899v1 fatcat:skwplgdn5zgddjqr4wi5ls6ouy

DARE: A Reflective Platform Designed to Enable Agile Data-Driven Research on the Cloud

Iraklis Klampanos, Federica Magnoni, Emanuele Casarotti, Christian Page, Mike Lindner, Andreas Ikonomopoulos, Vangelis Karkaletsis, Athanasios Davvetas, Andre Gemund, Malcolm Atkinson, Antonios Koukourikos, Rosa Filgueira (+3 others)
2019 2019 15th International Conference on eScience (eScience)  
It implements the cataloguing and execution of fine-grained and Python-based dispel4py workflows as services.  ...  This paper presents design and implementation aspects of the DARE platform, as well as it provides directions for future development.  ...  The PEs library facilitates the storing of code implementation for such computational units to enable the sharing and collaboration.  ... 
doi:10.1109/escience.2019.00079 dblp:conf/eScience/KlampanosMCPLIK19 fatcat:7immlbim2racxa534ovdylysfi

Portable, high-performance containers for HPC [article]

Lucas Benedicic, Felipe A. Cruz, Alberto Madonna, Kean Mariotti
2017 arXiv   pre-print
In this work we present an extension to the container runtime of Shifter that provides containerized applications with a mechanism to access GPU accelerators and specialized networking from the host system  ...  The presented extension makes possible to rapidly deploy high-performance software on supercomputers from containerized applications that have been developed, built, and tested in non-HPC commodity hardware  ...  For each DLL operation the compute node needs to request the location of the shared object to the Lustre Metadata server (MDS) and then fetch the memory block with the shared object from the Lustre Object  ... 
arXiv:1704.03383v1 fatcat:ap5hi77zmvb3dkmb7wmoh25h4a

HPC Cloud Architecture to Reduce HPC Workflow Complexity in Containerized Environments

Guohua Li, Joon Woo, Sang Boem Lim
2021 Applied Sciences  
This HPC cloud architecture integrates both image management and job management, which are the two main elements of HPC cloud workflows.  ...  The complexity of high-performance computing (HPC) workflows is an important issue in the provision of HPC cloud services in most national supercomputing centers.  ...  Data sharing is not applicable to this article.  ... 
doi:10.3390/app11030923 fatcat:fxexp7fhtbcdheho7codhilkqi

Serverless is More: From PaaS to Present Cloud Computing

Erwin van Eyk, Lucian Toader, Sacheendra Talluri, Laurens Versluis, Alexandru Uta, Alexandru Iosup
2018 IEEE Internet Computing  
Containerized Resources Complementary to time-sharing, virtualization abstracts away the physical machine to reduce the operational effort, and to allow the same physical resources to be shared across  ...  By enabling multiple clients to share the same physical infrastructure, time-sharing gave rise to cloud computing and the modern digital ecosystems, which are key drivers for growth in knowledge-based  ... 
doi:10.1109/mic.2018.053681358 fatcat:niayqqghojcedlr6yppfacqftm

Toward A Reproducible, Scalable Framework for Processing Large Neuroimaging Datasets [article]

Erik C Johnson, Miller Wilt, Luis M Rodriguez, Raphael Norman-Tenazas, Corban Rivera, Nathan Drenkow, Dean Kleissas, Theodore J. LaGrow, Hannah P Cowley, Joseph Downs, Jordan Matelsky, Marisa Hughes (+5 others)
2019 bioRxiv   pre-print
facilitate scientific discovery, we propose a generalized processing framework, that connects and extends existing open-source projects to provide large-scale data storage, reproducible algorithms, and workflow  ...  Acknowledgements We would like to thank the Apache Airflow and Common Workflow language teams for their open-source tools supporting reproducible workflows, as well as the research groups who produced  ...  our reference EM and XRM volumes for analysis.  ... 
doi:10.1101/615161 fatcat:g5mngz24grb5zm2f45ray27g6y

The IMBBC HPC facility: history, configuration, usage statistics and related activities

Haris Zafeiropoulos, Anastasia Gioti, Stelios Ninidakis, Antonis Potirakis, Savvas Paragkamian, Nelina Angelova, Aglaia Antoniou, Theodoros Danis, Eliza Kaitetzidou, Panagiotis Kasapidis, Jon Bent Kristoffersen, Vasileios Papadogiannis (+14 others)
2021 Zenodo  
It also intends to share them to the rest of the scientific community either directly as a data-presenting paper or by accompanying sister publications (where data interpretations and conclusions are given  ...  This preprint introduces key pieces of information of the IMBBC HPC facility regarding its history, usage statistics, user management and in-house developed containers/workflows along with accompanying  ...  The RvLab command-line back-end was recently containerized as a Singularity image. DECO DECO is a programming workflow for the automation of biodiversity historical data curation.  ... 
doi:10.5281/zenodo.4665308 fatcat:fnhssgdmw5guhkc27n7zmnllpm

The IMBBC HPC facility: history, configuration, usage statistics and related activities

Haris Zafeiropoulos, Anastasia Gioti, Stelios Ninidakis, Antonis Potirakis, Savvas Paragkamian, Nelina Angelova, Aglaia Antoniou, Theodoros Danis, Eliza Kaitetzidou, Panagiotis Kasapidis, Jon Bent Kristoffersen, Vasileios Papadogiannis (+14 others)
2021 Zenodo  
It also intends to share them to the rest of the scientific community either directly as a data-presenting paper or by accompanying sister publications (where data interpretations and conclusions are given  ...  This preprint introduces key pieces of information of the IMBBC HPC facility regarding its history, usage statistics, user management and in-house developed containers/workflows along with accompanying  ...  The RvLab command-line back-end was recently containerized as a Singularity image. DECO DECO is a programming workflow for the automation of biodiversity historical data curation.  ... 
doi:10.5281/zenodo.4646133 fatcat:ppribja4v5evvcgit256gvjove

Piloting a model-to-data approach to enable predictive analytics in health care through patient mortality prediction

Timothy Bergquist, Yao Yan, Thomas Schaffter, Thomas Yu, Vikas Pejaver, Noah Hammarlund, Justin Prosser, Justin Guinney, Sean Mooney
2020 JAMIA Journal of the American Medical Informatics Association  
We showcase the "Model to Data" (MTD) approach as a new mechanism to make private clinical data available for the development of predictive models.  ...  Objective The development of predictive models for clinical application requires the availability of electronic health record (EHR) data, which is complicated by patient privacy concerns.  ...  Gang Luo, Kari Stephens, Martin Gunn, Aaron Lee, Meliha Yetisgen, and Su-In Lee for their advice and efforts in planning this project.  ... 
doi:10.1093/jamia/ocaa083 pmid:32638010 fatcat:gphax5uy5zeojgl2rwisgredgm

COPA: Experimenter-level Container Orchestration for Networking Testbeds

Henrique C. C. De Resende, Matias A. K. Schimuneck, Cristiano B. Both, Juliano A. Wickboldt, Johann M. Marquez-Barja
2020 IEEE Access  
In this article, we propose COPA, an experimenter-level container orchestration tool for networking testbeds.  ...  This tool provides a friendly interface for the experimenter test container orchestration algorithms which can start, stop, copy, and even migrate a container from one host to another.  ...  The Commission is not liable for any use that may be made of any of the information contained therein.  ... 
doi:10.1109/access.2020.3035619 fatcat:q6zaa5ytbbclngirkbboxy43fq

Kliko - The Scientific Compute Container Format [article]

Gijs Molenaar, Spheshile Makhathini, Julien N. Girard, Oleg Smirnov
2018 arXiv   pre-print
astronomical data, and VerMeerKAT, a multi-container workflow data reduction pipeline which is being used as a prototype pipeline for the commissioning of the MeerKAT radio telescope.  ...  This paper will also discuss the implementation of a support library named Kliko which can be used to create Kliko containers, parse Kliko definitions, chain Kliko containers in workflows using a workflow  ...  Software containerization with Docker Containerization is a method for building self-contained environments (called "containers") for applications.  ... 
arXiv:1807.03264v1 fatcat:r7dgukjs6nd3hg6gqdhegzejku

Phenomenal-Preprocess Virtual Machine Image To Enable Data Producers To Locally Process Raw Data Into Standard Formats Supported In Phenomenal - D9.2.1

PhenoMeNal Consortium
2016 Zenodo  
Containerization instead of Virtual Machine 3.2. In PhenoMeNal we implement workflows that are platform-independent -for deployment flexibility and user convenience.  ...  "mzML-a community standard for mass spectrometry data."  ... 
doi:10.5281/zenodo.832588 fatcat:2hk44p4jnfbwhehbsxaicejf6y
« Previous Showing results 1 — 15 out of 857 results