On-demand Overlay Networks for Large Scientific Data Transfers

Lavanya Ramakrishnan, Chin Guok, Keith Jackson, Ezra Kissel, D. Martin Swany, Deborah Agarwal
2010 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing  
Large scale scientific data transfers are central to scientific processes. Data from large experimental facilities have to be moved to local institutions for analysis or often data needs to be moved between local clusters and large supercomputing centers. In this paper, we propose and evaluate a network overlay architecture to enable highthroughput, on-demand, coordinated data transfers over wide-area networks. Our work leverages Phoebus and On-demand Secure Circuits and Advance Reservation
more » ... em (OSCARS) to provide high performance wide-area network connections. OSCARS enables dynamic provisioning of network paths with guaranteed bandwidth and Phoebus enables the coordination and effective utilization of the OSCARS network paths. Our evaluation shows that this approach leads to improved end-to-end data transfer throughput with minimal overheads. The achieved throughput using our overlay was limited only by the ability of the end hosts to sink the data.
doi:10.1109/ccgrid.2010.82 dblp:conf/ccgrid/RamakrishnanGJKSA10 fatcat:ez7eoox3yzfopaut3lhbrvtaki