Resource analyses for parallel and distributed coordination

P. W. Trinder, M. I. Cole, K. Hammond, H-W. Loidl, G. J. Michaelson
2011 Concurrency and Computation  
Predicting the resources that are consumed by a program component is crucial for many parallel or distributed systems. In this context, the main resources of interest are execution time, space and communication/synchronisation costs. There has recently been significant progress in resource analysis technology, notably in type-based analyses and abstract interpretation. At the same time, parallel and distributed computing are becoming increasingly important. This paper synthesises progress in
more » ... h areas to survey the state-of-the-art in resource analysis for parallel and distributed computing. We articulate a general model of resource analysis and describe parallel/distributed resource analysis together with the relationship to sequential analysis. We use three parallel or distributed resource analyses as examples and provide a critical evaluation of the analyses. We investigate why the chosen analysis is effective for each application and identify general principles governing why the resource analysis is effective. emphasis on resource-constrained computing in areas such as embedded systems and cloud computing. Practical uses of resource analysis include providing guarantees for safety-critical systems, ensuring QoS for networks or embedded systems or providing information that can be used to make sensible decisions about the allocation of resources in for example database systems. Large-scale uses include the ASTREE system, which has been used to analyse the worst-case execution time (WCET) of the flight-control software for the Airbus A380 [15] , and the use of the SPEED symbolic complexity analyser to analyse much of the .NET code base [16] . This paper surveys one particularly important application area, namely parallel or distributed computing. Parallel systems are gaining importance with the expansion of multicore and manycore machines. Similarly, distributed systems are gaining in importance with the widespread adoption of internet and cloud technologies. Allocating resources effectively is important to achieving good performance as the number of cores rises in current and future architectures. A number of different resources are of interest for parallel and distributed systems, for example execution time for parallelism or power consumption in a network of low power devices such as sensors. Moreover, the resource information may be used for a number of purposes; for example, execution time predictions can be used to optimise the performance of parallel programs or to aid load management or scheduling within distributed systems. In summary, effective resource analyses provide information that enables better coordination of parallel and distributed computation. Contributions made by the paper We start by motivating resource prediction for parallel/distributed systems (Section 2). We then present and illustrate an informal general model of resource analysis (Section 3). The model codifies folklore, that is 'what is usually done'. It is, however, general with regard to the resources analysed and the bounds asserted. That is, the resources of interest may include execution time, memory usage, file handles or any other limited and quantifiable resource. Similarly, predictions may, for example, be formally verifiable worst-case bounds, probabilistic measures of worst-case or average-case behaviours, or simple estimations of resource usage. The paper then makes the following contributions: RESOURCE ANALYSES FOR PARALLEL AND DISTRIBUTED COORDINATION T D S C P speedup D T =.S C P =N / So speedup is bound by the inherently sequential portion and depends on the near-optimal deployment of the processors to share the potentially parallel portion. Thus, two important objectives in parallel programming are to minimise the inherently sequential portion of a program and to ensure that each of the N processors carries out a very similar proportion of the parallel portion of the program. We must also communicate data and results between the processors and coordinate their activities, and doing so introduces time overheads that must be accounted for. These overheads may be either inherently sequential (e.g. to distribute initial data and receive final results) or potentially parallel (e.g. to transmit intermediate information between subsets of processors). Hence, key issues for assessing potential parallelism are as follows: Which portions of a program are inherently sequential and which are potentially parallel? What are the sequential execution times of these portions? Which communication and coordination construct to introduce to best enable parallelism? What additional time overheads do these constructs bear? Parallel programming Despite the existence of mature methodologies for parallel programming (e.g. [19]), combinations of folklore, code inspection and profiling seem to prevail in common use. The folklore holds, for example, that communication is more costly than processing; parallelism is most beneficial where substantial activities may be carried out with minimal communication; This section introduces resource analysis, giving a general model of resource analysis, and an example sequential analysis. The aim of resource analysis is to take a program component and apply an analysis that will give an accurate picture of what the component costs on some implementation platform, takes considerably less time and effort than profiling the component when it is executing on the platform and can be straightforwardly applied to other implementation platforms.
doi:10.1002/cpe.1898 fatcat:abmlmqopxraofkdqbvk5r575ya