A proposal for a set of parallel basic linear algebra subprograms [chapter]

Jaeyoung Choi, Jack Dongarra, Susan Ostrouchov, Antoine Petitet, David Walker, R. Clinton Whaley
1996 Lecture Notes in Computer Science  
This paper describes a proposal for a set of Parallel Basic Linear Algebra Subprograms PBLAS. The PBLAS are targeted at distributed vector-vector, matrix-vector and matrixmatrix operations with the aim of simplifying the parallelization of linear algebra codes, especially when implemented on top of the sequential BLAS. At rst glance, because of the apparent simplicity of its sequential counterpart as well as the regularity of the data structures involved in dense linear algebra computations,
more » ... lementing an equivalent set of parallel routines in terms of portability, e ciency, and ease-of-use seems relatively simple to achieve. However, when these routines are actually coded, the problem becomes much more complex due to di culties which do not occur in serial computing. First, there are many di erent parallel computer architectures available. In view of this fact, it is natural to choose a virtual machine topology that is convenient for dense linear algebra computations and map the virtual machine onto existing topologies. Second, the selected data distribution scheme must ensure good load-balance to guarantee performance and scalability. Finally, for ease-of-use and software reusability reasons, the interface of the top-level routines must closely resemble the sequential BLAS interface yet still be exible enough to take advantage of e cient parallel algorithmic techniques such as computation and communication overlapping and pipelining. This paper presents a reasonable set of adoptable solutions to successfully design and implement the Parallel Basic Linear Algebra Subprograms. These subprograms can in turn be used to develop parallel libraries such as ScaLAPACK for a large variety of distributed memory MIMD computers.
doi:10.1007/3-540-60902-4_13 fatcat:55xiy7xzqjamre6fk5ire2oaau