MNEMONIC: A Network Environment for Automatic Optimization and Tuning of Data Movement over Advanced Networks

Patrick McClory, Ezra Kissel, Martin Swany, Michela Taufer
2009 2009 Proceedings of 18th International Conference on Computer Communications and Networks  
Computational science is considered the third pillar of science but while we continue to make significant strides in high-performance computing, the ability to transfer the vast amounts of data generated and consumed by scientists has fallen behind. This divergence has significantly constrained scientific progress. This paper describes MNEMONIC, an end-to-end cyberinfrastructure system designed to improve scientists' productivity by automatically and transparently optimizing and tuning data
more » ... ment over the network. MNEMONIC integrates advanced technologies, i.e., network interfaces, protocols, and measurement toolkits, to seamlessly manage high-performance data movement. We have integrated the Java implementation of GridFTP in MNEMONIC to provide a cross-platform data transfer capability. Preliminary results show that MNEMONIC+GridFTP outperforms the TCP+GridFTP implementation by over a factor of 10x, while remaining easy to install and operate.
doi:10.1109/icccn.2009.5235222 dblp:conf/icccn/McCloryKST09 fatcat:q2yqikozordyhpzqwnx7zqt6b4