Filters








549,080 Hits in 7.0 sec

On-demand Overlay Networks for Large Scientific Data Transfers

Lavanya Ramakrishnan, Chin Guok, Keith Jackson, Ezra Kissel, D. Martin Swany, Deborah Agarwal
2010 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing  
Large scale scientific data transfers are central to scientific processes.  ...  In this paper, we propose and evaluate a network overlay architecture to enable highthroughput, on-demand, coordinated data transfers over wide-area networks.  ...  ACKNOWLEDGEMENTS The authors would like to thank Evangelos Chaniotakis and David Robertson for helping with use of the OSCARS server.  ... 
doi:10.1109/ccgrid.2010.82 dblp:conf/ccgrid/RamakrishnanGJKSA10 fatcat:ez7eoox3yzfopaut3lhbrvtaki

An Optimized Data Obtaining Strategy for Large-Scale Sensor Monitoring Networks

Yan Wang, Junlu Wang, Fengtong Wang, Ling Wang, Wei Wei
2016 International Journal of Distributed Sensor Networks  
First, because of the large-scale features of sensor node network, this paper proposes a large-scale monitoring network area clustering optimization strategy.  ...  As the technology of the Internet of Things (IoT) becomes more widely used in large-scale monitoring networks, this paper proposes an optimized obtaining strategy (OFS) for large-scale sensor monitoring  ...  Because the number of nodes is large and their distribution is uneven, this paper proposes a type of large-scale wireless sensor network area clustering optimization strategy in which a large-scale monitoring  ... 
doi:10.1155/2016/4262565 fatcat:jczpylm4fff7tooa7jthfawwpm

Evaluation of the inter-cluster data transfer on Grid environment

S. Ogura, S. Matsuoka, H. Nakada
2003 CCGrid 2003. 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid, 2003. Proceedings.  
High-performance peer-to-peer transfer between clusters will be fundamental technology base for various Grid middleware, such as large-scale data transfer in DataGrid settings, or collective communication  ...  By dynamically and automatically adjusting transfer parameters between the two clusters, such as the number of network nodes, number of socket stripes, we could achieve optimal bandwidth even when the  ...  Background and Previous Work on High Performance Data Transfer in TCP/IP Wide-Area Networks TCP To prevent congestions among multiple TCP streams on a single network link, TCP regulates the rate of data  ... 
doi:10.1109/ccgrid.2003.1199390 dblp:conf/ccgrid/OguraMN03 fatcat:2w67qeszhjgwdlstkhgnbbtmpu

Open problems in network-aware data management in exa-scale computing and terabit networking era

Mehmet Balman, Surendra Byna
2011 Proceedings of the first international workshop on Network-aware data management - NDM '11  
Efficient use of the network infrastructure is necessary in order to address the increasing data and compute requirements of large-scale applications.  ...  Accessing and managing large amounts of data is a great challenge in collaborative computing environments where resources and users are geographically distributed.  ...  We also thank the NDM 2011 authors for their contributions, and the workshop audience for their insightful comments and feedback.  ... 
doi:10.1145/2110217.2110229 fatcat:o63xjsoafvgurgzsz6xoeypzti

Transferable deep learning model based on the phenological matching principle for mapping crop extent

Shuang Ge, Jinshui Zhang, Yaozhong Pan, Zhi Yang, Shuang Zhu
2021 International Journal of Applied Earth Observation and Geoinformation  
The procedure and strategy developed will facilitate the realization of high-performance and automated global model transfer.  ...  The average optimal Overall Accuracy of crop mapping in all the target areas exceeded 87%. The average optimal F1score of corn and rice was 0.79.  ...  large-scale areas.  ... 
doi:10.1016/j.jag.2021.102451 fatcat:xj6t5lxifbfirazhsbk5njytom

Obtaining Good Heuristics for Big Data Broadcasting Problem Using Novel Pipeline Approach

Prof. Priyadarshani Kalokhe
2016 International Journal Of Engineering And Computer Science  
Thus there is a drastic demand for building a service to distribute, manage and process massive data sets that has risen tremendously. Engineers and researchers are dealing with these large dataset.  ...  We model the big data broadcasting problem into a LOCKSTEP BROADCAST TREE (LSBT).The main idea of this model is to define a basic unit of upload bandwidth r, such that a node with capacity c broadcast  ...  message at an optimal rate in a large scale distributed network under the multi-port communication model.  ... 
doi:10.18535/ijecs/v5i12.20 fatcat:okrw4enjzvhb7e4hudyqtzswha

StorkCloud

Tevfik Kosar, Engin Arslan, Brandon Ross, Bing Zhang
2013 Proceedings of the 4th ACM workshop on Scientific cloud computing - Science Cloud '13  
Wide-area transfer of large data sets is still a big challenge despite the deployment of high-bandwidth networks with speeds reaching 100 Gbps.  ...  We have developed a "data transfer scheduling and optimization system as a Cloud-hosted service", StorkCloud, which will mitigate the large-scale end-to-end data movement bottleneck by efficiently utilizing  ...  We have developed a Cloud-hosted data transfer and optimization service, StorkCloud, which will mitigate the large-scale end-to-end data movement bottleneck by efficiently scheduling and optimizing data  ... 
doi:10.1145/2465848.2465855 dblp:conf/hpdc/KosarARZ13 fatcat:qccihbwwt5c5tbw2zxakyyemcm

Managing Very-Large Distributed Datasets [chapter]

Miguel Branco, Ed Zaluska, David de Roure, Pedro Salgado, Vincent Garonne, Mario Lassnig, Ricardo Rocha
2008 Lecture Notes in Computer Science  
the wide-area network.  ...  We describe DQ2's design and implementation, which builds upon previous work on distributed file systems, peer-to-peer systems and Data Grids.  ...  We would like to acknowledge the many contributions to the design by Torre Wenaus and David Cameron and the help of David and Benjamin Gaidioz in implementing DQ2.  ... 
doi:10.1007/978-3-540-88871-0_54 fatcat:nbx7bqkiajgmnnvwqeeh2liuve

Alternating Direction Method of Multipliers (ADMM) Based Distributed Approach For Wide-Area Control

Abilash Thakallapelli, Sukumar Kamalasadan
2019 IEEE transactions on industry applications  
After convergence, a transfer function residue corresponding to the interarea mode of interest is derived, to determine optimal wide area control loop.  ...  Second, local processors are assigned on each area that estimate a black-box transfer function model based on Lagrange multipliers using measurements.  ...  For large scale network the centralized approach for wide area signal selection may not be feasible due to various factors like data volume, data transfer capability, computational time etc.  ... 
doi:10.1109/tia.2019.2896837 fatcat:kfx4zfnauvc4ri2vuzrihplizm

Big-Data Science: Infrastructure Impact

Inder Monga, Prabhat Berkeley
2018 Proceedings of the Indian National Science Academy  
This chapter introduces infrastructure requirements of High-Performance Computing and Networking with examples drawn from NERSC and ESnet, two large Department of Energy facilities at Lawrence Berkeley  ...  The work described in the paper, Deep Learning at 15PF: Supervised and Semi-Supervised Classification for Scientific Data (https:// arxiv.org/abs/1708.05256), reported that a Cray XC40 system with a configuration  ...  End hosts not optimized for wide-area data transfers Systems used for wide area science data transfers perform far better if they are purposebuilt for and dedicated to this function.  ... 
doi:10.16943/ptinsa/2018/49338 fatcat:u6aogkxehzatvmjvcunjp3vn64

Spatial Transferability of Random Forest Models for Crop Type Classification Using Sentinel-1 and Sentinel-2

Aiym Orynbaikyzy, Ursula Gessner, Christopher Conrad
2022 Remote Sensing  
., only optical, only Synthetic Aperture Radar (SAR), and optical-SAR data combination, and the impact of spatial feature selection were systematically tested to identify the optimal approach that shows  ...  Large-scale crop type mapping often requires prediction beyond the environmental settings of the training sites.  ...  Introduction Supervised machine learning methods are widely used for large-scale crop type classification [ 1 , 2 , 3 ].  ... 
doi:10.3390/rs14061493 fatcat:v2xed7idcbht7efoa4yynjqy24

Parallel computing for obtaining regional scale rice growth conditions based on WOFOST and satellite images

Bingyu Zhao, Meiling Liu, Jianjun Wu, Xiangnan Liu, Mengxue Liu, Ling Wu
2020 IEEE Access  
Transferring such a large data between the GPU and CPU is timeconsuming.  ...  Previous studies have shown that the exponential empirical model between LAI and NDVI in this area is practicable and most widely used [71] .  ... 
doi:10.1109/access.2020.3043003 fatcat:ll267opdcneqngjmwzqm5g7j4m

The Science DMZ

Eli Dart, Lauren Rotman, Brian Tierney, Mary Hester, Jason Zurawski
2013 Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis on - SC '13  
The ever-increasing scale of scientific data has become a significant challenge for researchers that rely on networks to interact with remote computing systems and transfer results to collaborators worldwide  ...  We explain the Science DMZ model, including network architecture, system configuration, cybersecurity, and performance tools, that creates an optimized network environment for science.  ...  The wide area networks (or WANs) are often already optimized and can accommodate large data flows up to 100Gbps. However, the local area networks are still a choke point for these large data flows.  ... 
doi:10.1145/2503210.2503245 dblp:conf/sc/DartRTHZ13 fatcat:h4hdeejoqraijdg7psgvylyfoa

The Science DMZ: A Network Design Pattern for Data-Intensive Science

Eli Dart, Lauren Rotman, Brian Tierney, Mary Hester, Jason Zurawski
2014 Scientific Programming  
The ever-increasing scale of scientific data has become a significant challenge for researchers that rely on networks to interact with remote computing systems and transfer results to collaborators worldwide  ...  We explain the Science DMZ model, including network architecture, system configuration, cybersecurity, and performance tools, that creates an optimized network environment for science.  ...  The wide area networks (or WANs) are often already optimized and can accommodate large data flows up to 100Gbps. However, the local area networks are still a choke point for these large data flows.  ... 
doi:10.1155/2014/701405 fatcat:pdxcpodcfbethkyzjmewmyvd4m

Large-Scale Parallel Computing on Grids

Henri Bal, Kees Verstoep
2008 Electronical Notes in Theoretical Computer Science  
We illustrate large-scale parallel computing on grids with three example applications that search large state spaces: transposition-driven search, retrograde analysis, and model checking.  ...  Algorithmic optimizations like latency-hiding and exploiting locality can be used effectively to obtain high performance on grids, despite the relatively slow wide-area networks that connect the grid resources  ...  We also thank Michael Weber and Wan Fokkink for their suggestions improving the paper. The initial work on TDS and Awari was performed largely by John Romein.  ... 
doi:10.1016/j.entcs.2008.11.010 fatcat:esb4axh6zberfdlf2m5y6w3rpe
« Previous Showing results 1 — 15 out of 549,080 results