315 Hits in 4.1 sec

Streamflow Programming Model for Data Streaming in Scientific Workflows

Chathura Herath, Beth Plale
2010 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing  
Many geo-science workflows have the need to interact with sensors that produce large continuous streams of data, but programming models provided by scientific workflows are not equipped to handle continuous  ...  data streams.  ...  The work presented in this paper builds upon the functionalities and the programming model presented in scientific workflow systems and provides a programming abstraction for processing data streams.  ... 
doi:10.1109/ccgrid.2010.116 dblp:conf/ccgrid/HerathP10 fatcat:i3y4c7i3d5a3vf3nrhm4cssqfq

Towards Reliable, Performant Workflows for Streaming-Applications on Cloud Platforms

Daniel Zinn, Quinn Hart, Timothy McPhillips, Bertram Ludäscher, Yogesh Simmhan, Michail Giakkoupis, Viktor K. Prasanna
2011 2011 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing  
Yet, the lack of integrated support for data models, including streaming data, structured collections and files, is limiting the ability of workflows to support emerging applications in energy informatics  ...  Scientific workflows are commonplace in eScience applications.  ...  The authors would like to thank the Los Angeles Department of Water and Power (LDWP) for discussions on the Smart Grid domain challenges.  ... 
doi:10.1109/ccgrid.2011.74 dblp:conf/ccgrid/ZinnHMLSGP11 fatcat:bfeylzpl4zb6xf7dry37rfsvwy

StreamFlow: cross-breeding cloud with HPC [article]

Iacopo Colonnelli, Barbara Cantalupo, Ivan Merelli, Marco Aldinucci
2020 arXiv   pre-print
StreamFlow is then exemplified on a novel bioinformatics pipeline for single-cell transcriptomic data analysis workflow.  ...  Workflows are among the most commonly used tools in a variety of execution environments.  ...  Acknowledgment This article describes work undertaken in the context of the Deep-Health project, "Deep-Learning and HPC to Boost Biomedical Applications for Health" ( which  ... 
arXiv:2002.01558v2 fatcat:jywyqbvx6vggxfwjzv5mkvgaxe

A CyberGIS Integration and Computation Framework for High-Resolution Continental-Scale Flood Inundation Mapping

Yan Y. Liu, David R. Maidment, David G. Tarboton, Xing Zheng, Shaowen Wang
2018 Journal of the American Water Resources Association  
We present a Digital Elevation Model (DEM)-based hydrologic analysis methodology for continental flood inundation mapping (CFIM), implemented as a cyberGIS scientific workflow in which a 1/3rd arc-second  ...  The inundation mapping process developed in our exploratory study couples HAND with National Water Model (NWM) forecast data to enable near real-time inundation forecasts for CONUS.  ...  The geospatial data involved in this work represent typical scientific big data in volume, variety, and velocity. Desktop-based GIS software is ill-suited for processing such big data collection.  ... 
doi:10.1111/1752-1688.12660 fatcat:s7ykhwdgf5g3tecsifnngowg54

Collaborative groundwater modeling: Open-source, cloud-based, applied science at a small-island water utility scale

Christopher K. Shuler, Katrina E. Mariner
2020 Environmental Modelling & Software  
The framework components include direct collection and analysis of climate and streamflow data, development of a water budget model, and initiation of a dynamic groundwater modeling process.  ...  Recent advances in cloud-computing and social-networking are influencing how we communicate professionally, work collaboratively, and approach data-science tasks.  ...  Acknowledgements We would like to express our appreciation for the many individuals and agencies who made this work possible by generously providing their time, expertise, and assistance with data collection  ... 
doi:10.1016/j.envsoft.2020.104693 fatcat:bvjkjz3uyvcmjosbsmjrkitmq4

Distributed workflows with Jupyter

Iacopo Colonnelli, Marco Aldinucci, Barbara Cantalupo, Luca Padovani, Sergio Rabellino, Concetto Spampinato, Roberto Morelli, Rosario Di Carlo, Nicolò Magini, Carlo Cavazzoni
2021 Future generations computer systems  
Language-independent approaches decouple workflow models from the host code's business logic and advocate portability.  ...  The proposed Jupyter-workflow (Jw) system is evaluated on common scenarios for High Performance Computing (HPC) and Cloud, showing its potential in lowering the barriers between prototypical Notebooks  ...  Acknowledgments This article describes work undertaken in the context of the DeepHealth project, 12 ''Deep-Learning and HPC to Boost Biomedical Applications for Health'' which has received funding from  ... 
doi:10.1016/j.future.2021.10.007 fatcat:2al5dpxqmrgeboqxgkgxbefxga

Beyond Metrics? The Role of Hydrologic Baseline Archetypes in Environmental Water Management

Belize A. Lane, Samuel Sandoval-Solis, Eric D. Stein, Sarah M. Yarnell, Gregory B. Pasternack, Helen E. Dahlke
2018 Environmental Management  
Scientific and management applications for in-stream flow allocations are expected to benefit considerably from a unified consensus stream classification system capable of supporting a broad range of programs  ...  The detailed temporal data provided in stream class DRHs also allows for comparisons with modeled streamflow conditions, making this technique particularly useful for considering climate change impacts  ... 
doi:10.1007/s00267-018-1077-7 pmid:29934651 fatcat:pflfaqbqv5gulfco634ouym3my

Model-driven deployment and management of workflows on analytics frameworks

Merlijn Sebrechts, Sander Borny, Thomas Vanhove, Gregory Van Seghbroeck, Tim Wauters, Bruno Volckaert, Filip De Turck
2016 2016 IEEE International Conference on Big Data (Big Data)  
This paper addresses the operational challenges behind deploying and managing workflows on top of analytics platforms by starting from three key requirements: data scientists want to model their workflows  ...  in a reusable way, this model should be automatically deployed, managed and connected to other services, and this solution should be compatible with existing cloud modeling languages, infrastructure,  ...  a scientific workflow.  ... 
doi:10.1109/bigdata.2016.7840930 dblp:conf/bigdataconf/SebrechtsBVSWVT16 fatcat:jv3en4igxzdhlmy3w35yhygm54

Workflow models for heterogeneous distributed systems

Iacopo Colonnelli, Marco Aldinucci
2022 Zenodo  
The role of data in modern scientific workflows becomes more and more crucial.  ...  Most modern programming paradigms for Big Data analysis adhere to the principle of data locality: moving computation closer to the data to remove transfer-related overheads and risks.  ...  Acknowledgements 1.4 Funding This work has been partially supported by the DeepHealth project 3 , "Deep-Learning and HPC to Boost Biomedical Applications for Health" which has received funding from the  ... 
doi:10.5281/zenodo.7135484 fatcat:hx2ayiku5nadblgj6nlmhplmd4

The Italian research on HPC key technologies across EuroHPC

Marco Aldinucci, Giovanni Agosta, Antonio Andreini, Claudio A. Ardagna, Andrea Bartolini, Alessandro Cilardo, Biagio Cosenza, Marco Danelutto, Roberto Esposito, William Fornaciari, Roberto Giorgi, Davide Lengani (+5 others)
2021 Proceedings of the 18th ACM International Conference on Computing Frontiers  
High-Performance Computing (HPC) is one of the strategic priorities for research and innovation worldwide due to its relevance for industrial and scientific applications. We envision HPC as composed  ...  In this section, we briefly describe two such applications in key industrial and scientific domains. Environmental modeling.  ...  The StreamFlow framework [14, 15] is a containernative Workflow Management System (WMS) written in Python 3 and based on the Common Workflow Language (CWL) Standard [5] .  ... 
doi:10.1145/3457388.3458508 fatcat:nbnzfa2frvbpflj6tcbsk4bwcq

SFRmaker and Linesink‐maker: Rapid construction of streamflow routing networks from hydrography data

Andrew T. Leaf, Michael N. Fienen, Howard W. Reeves
2021 Ground Water  
SFRmaker creates input for the MODFLOW SFR package, while Linesink-maker creates linesink string input for the GFLOW analytic element program.  ...  These programs can reduce weeks or even months of manual effort to a few minutes of execution time, and carry the added advantages of reduced potential for error, improved reproducibility and facilitation  ...  Geological Survey Water Availability and Use Science Program (WAUSP), the U.S. Geological Survey Cooperative Water Program, and the U.S. Forest Service.  ... 
doi:10.1111/gwat.13095 pmid:33745128 fatcat:zuqu2q5sbfewrnlzab7q35hovu

Uncertainty in Flood Inundation Mapping: Current Issues and Future Directions

Venkatesh Merwade, Francisco Olivera, Mazdak Arabi, Scott Edleman
2008 Journal of hydrologic engineering  
In addition, the idea of a probabilistic flood inundation map is articulated, and an integrated framework approach that will connect data, models, and uncertainty analysis techniques in producing probabilistic  ...  By using a sample data set for Strouds Creek, N.C., this paper highlights key uncertainties associated with flood inundation mapping.  ...  The writers would also like to thank three anonymous reviewers for comments which led to improvement of the manuscript.  ... 
doi:10.1061/(asce)1084-0699(2008)13:7(608) fatcat:2fjkqtwpi5bcnnnjh3zszatm5i

Vision of Cyberinfrastructure for End-to-End Environmental Explorations (C4E4)

R. S. Govindaraju, B. Engel, D. Ebert, B. Fossum, M. Huber, C. Jafvert, S. Kumar, V. Merwade, D. Niyogi, L. Oliver, S. Prabhakar, G. Rochon (+2 others)
2009 Journal of hydrologic engineering  
The writers' vision of a cyberinfrastructure for end-to-end environmental exploration ͑C4E4͒ that combines data and modeling tools in an integrated environment across different spatial and temporal scales  ...  The C4E4 framework will allow researchers to combine heterogeneous data resources with state-ofthe-art modeling and visualization tools through a user-friendly web portal.  ...  C4E4 will enable a variety of users to set up scientific workflows combining the full suite of process and impact models with an array of static or streaming data sets.  ... 
doi:10.1061/(asce)1084-0699(2009)14:1(53) fatcat:nffdzc72ejexfksd2lajc33ygm

Hydrologic Modeling as a Service (HMaaS): A New Approach to Address Hydroinformatic Challenges in Developing Countries

Michael A. Souffront Alcantara, E. James Nelson, Kiran Shakya, Christopher Edwards, Wade Roberts, Corey Krewson, Daniel P. Ames, Norman L. Jones, Angelica Gutierrez
2019 Frontiers in Environmental Science  
Web applications and services for providing improved data accessibility, and addressing the discussed hydroinformamtic challenges are also presented.  ...  A global streamflow prediction system (GSPS) would help decrease vulnerabilities in developing countries and fill gaps in areas where no local models exist by providing extensive results that can be filtered  ...  opportunity to test and validate model results; the Esri team who helped develop the global services visualization; and Microsoft's AI for Earth who provided cloud services necessary to run the model.  ... 
doi:10.3389/fenvs.2019.00158 fatcat:3tsmvslbcneljngqacn6gleori

An R Package for Open, Reproducible Analysis of Urban Water Systems, With Application to Chicago

Laura E Erban, Stephen B Balogh, Daniel E Campbell, Henry A Walker
2018 Open water  
CityWaterBalance provides a reproducible workflow for studying urban water systems by facilitating automated retrievals of open data and post-processing with open source R functions.  ...  We have developed a generic urban water system model in the R package CityWaterBalance.  ...  For streamflow, there is no alternative to the USGS stream gauge network for comprehensive data.  ... 
pmid:29546883 pmcid:PMC5846499 fatcat:iaqhqwroprdk5ksxqn3cngxvfu
« Previous Showing results 1 — 15 out of 315 results