Improving I/O Forwarding Throughput with Data Compression

Benjamin Welton, Dries Kimpe, Jason Cope, Christina M. Patrick, Kamil Iskra, Robert Ross
2011 2011 IEEE International Conference on Cluster Computing  
While network bandwidth is steadily increasing, it is doing so at a much slower rate than the corresponding increase in CPU performance. This trend has widened the gap between CPU and network speed. In this paper, we investigate improvements to I/O performance by exploiting this gap. We harness idle CPU resources to compress network traffic, reducing the amount of data transferred over the network and increasing effective network bandwidth. We created a set of compression services within the
more » ... Forwarding Scalability Layer. These services transparently compress and decompress data as it is transferred over the network. We studied the effect of the compression services on a variety of data sets and conducted experiments on a high-performance computing cluster.
doi:10.1109/cluster.2011.80 dblp:conf/cluster/WeltonKCPIR11 fatcat:57ysrt6n7be3lgfrzvbudy5eh4