A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2017; you can also visit <a rel="external noopener" href="http://users.eecs.northwestern.edu/%7Echoudhar/Publications/GaoLia09B.pdf">the original URL</a>. The file type is <code>application/pdf</code>.
Combining I/O operations for multiple array variables in parallel netCDF
<span title="">2009</span>
<i title="IEEE">
<a target="_blank" rel="noopener" href="https://fatcat.wiki/container/2ade5dt4mjgqzod5n6qf33litm" style="color: black;">2009 IEEE International Conference on Cluster Computing and Workshops</a>
</i>
Preserved Fulltext
Parallel netCDF (PnetCDF) is a popular library used in many scientific applications to store scientific datasets. It provides high-performance parallel I/O while maintaining file-format compatibility with Unidata's netCDF. Array variables comprise the bulk of the data in a netCDF dataset, and for accesses to large regions of single array variables, PnetCDF attains very high performance. However, the current PnetCDF interface only allows access to one array variable per call. If an application
<span class="external-identifiers">
<a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/clustr.2009.5289153">doi:10.1109/clustr.2009.5289153</a>
<a target="_blank" rel="external noopener" href="https://dblp.org/rec/conf/cluster/GaoLCRL09.html">dblp:conf/cluster/GaoLCRL09</a>
<a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/nawlq6oyrbc7bh4d3nkydzyepq">fatcat:nawlq6oyrbc7bh4d3nkydzyepq</a>
</span>
more »
... stead accesses a large number of small-sized array variables, this interface limitation can cause significant performance degradation, because high end network and storage systems deliver much higher performance with larger request sizes. Moreover, the record variables data is stored interleaved by record, and the contiguity information is lost, so the existing MPI-IO collective I/O optimization can not help. This paper presents a new mechanism for PnetCDF to combine multiple I/O operations for better I/O performance. This mechanism can be used in a new function that takes arguments for reading/writing multiple array variables, allowing application programmers to explicitly access multiple array variables in a single call. It can also be used in the implementation of asynchronous I/O functions, so that the combination is carried out implicitly, without changes to the application. Our performance results demonstrate significant improvement using well-known application benchmarks.
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20170808221916/http://users.eecs.northwestern.edu/%7Echoudhar/Publications/GaoLia09B.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext">
<button class="ui simple right pointing dropdown compact black labeled icon button serp-button">
<i class="icon ia-icon"></i>
Web Archive
[PDF]
<div class="menu fulltext-thumbnail">
<img src="https://blobs.fatcat.wiki/thumbnail/pdf/96/77/9677014dea33ba3ab376448f89481fab4af4ff9c.180px.jpg" alt="fulltext thumbnail" loading="lazy">
</div>
</button>
</a>
<a target="_blank" rel="external noopener noreferrer" href="https://doi.org/10.1109/clustr.2009.5289153">
<button class="ui left aligned compact blue labeled icon button serp-button">
<i class="external alternate icon"></i>
ieee.com
</button>
</a>