The Data Mover: A Machine-Independent Abstraction for Managing Customized Data Motion [chapter]

Scott B. Baden, Stephen J. Fink
2000 Lecture Notes in Computer Science  
This paper discusses an abstraction, called the Data Mover, for expressing machine-independent customized communication algorithms in a variety of block-structured applications. The Data Mover enables its user to express data motion using intuitive geometric operations that encapsulate the low-level details of the underlying communication. Communication patterns are expressed as collective operations, and are restricted to movement of rectangular array sections. We describe the Data Mover model
more » ... of communication, and present performance for various applications. The Data Mover currently serves as useful middleware for application library designers, but defines a simple machine-independent interface suitable as a target for a compiler or compiler run time library. 1 Notably, IBM's MPI implementation provides non-blocking collective communication primitives, but the hardware is only able to realize overlap under restricted conditions. 3 The Data Mover Communication Model The Data Mover has been implemented as part of the KeLP infrastructure. While a complete discussion of KeLP lies beyond the scope of this paper, we describe portions of the KeLP model relevant to the ensuing discussions. We refer the reader to a recent paper [10] or to the KeLP web site at KeLP supports distributed collections of block-structured data, and their distribution across multiple address spaces. Some examples are shown in Fig. 1 . At the core of KeLP are two capabilities: user-level meta-data, and a collective model of communication. To support this model, KeLP provides two kinds abstractions: meta-data and instantiators, which are listed in Table 1 . KeLP meta-data objects represent the abstract structure of some facet of the calculation, such as data decomposition, communication, or controlled execution under mask. Instantiation objects carry out program behavior based on information contained in meta-data objects; they allocate storage or move data. KeLP supports two levels of control flow, collective level and node level, as articulated in the Phase Abstractions programming model [18] and embodied by SPMD programming with MPI. A program performs collective operations, such as reductions, barriers and broadcasts, interspersed within a node level program. The node-level instructions form separate threads of control and execute independently. In most cases, the node level will invoke highly tuned serial numeric kernels. KeLP meta-data, and most instantiator objects, may live at either the collective level or the node level. KeLP does not assume the existence of a global shared address space, though a clever implementation may take advantage of shared memory to improve performance. methods that employ halo updates [29], using a different representation than stencil-based computations, but with a similar notion of locality. We presented a Mover implementation that implicitly performs multi-method communication [12] [19], in which it will issue a memory copy in lieu of a call to MPI when it detects a data transfer involving blocks of data assigned to the same process. An interesting possibility is to generalize KeLP's multi-method communication to other kinds of data transfer methods, for example TCP/IP, to support topologically aware communication. KeLP is currently defined to manage data motion among private memories of a multicomputer, among processors of an SMP, or both, in the case of SMP clustered technology. We are currently investigating Movers that transports data between disk and memory, or between a remotely executing program controlled by a Java front end. The KeLP Mover is a useful abstraction for managing I/O, since it supports the notion of scheduled, collective communication. KeLP currently runs on a variety of architectures including the IBM SP2, IBM ASCI Blue-Pacific, SGI-Cray Origin 2000, and clusters of workstations running under Solaris, Linux, and Digital Unix. A port to the SGI-Cray T3E is in progress.
doi:10.1007/3-540-44905-1_21 fatcat:32polp3dcbgzbikn2n6vneqh3y