A Comparison of Implementation Strategies for Nonuniform Data-Parallel Computations

Salvatore Orlando, Raffaele Perego
1998 Journal of Parallel and Distributed Computing  
Data{parallel languages allow programmers to easily express parallel computations by means of high-level constructs. To reduce overheads, the compiler partitions the computations among the processors at compile{time, on the basis of the static data distribution suggested by the programmer. When execution costs are non{uniform and unpredictable, some processors may be assigned more work than others. Workload imbalance can be mitigated by cyclically distributing data and associated computations,
more » ... r by employing adaptive strategies which build a more balanced schedule at run{time, on the basis of the actual execution costs. This paper discusses static and hybrid (static + dynamic) scheduling strategies which can be used to balance the workloads derived from the execution of non{uniform parallel loops. A m ulti-dimensional ame simulation kernel has been used to evaluate di erent implementation strategies on a C r a y T 3 E . W e fed the benchmark code with synthetic input data sets built on the basis of a load imbalance model and we report and compare the results obtained.
doi:10.1006/jpdc.1998.1456 fatcat:dpu3rbatdned7evd5pvvmmnsii