High Performance Software Coherence for Current and Future Architectures
Journal of Parallel and Distributed Computing
Shared memory provides an attractive and intuitive programming model for large-scale parallel computing, but requires a coherence mechanism to allow caching for performance while ensuring that processors do not use stale data in their computation. Implementation options range from distributed shared memory emulations on networks of workstations to tightly coupled fully cache-coherent distributed shared memory multiprocessors. Previous work indicates that performance varies dramatically from one
... end of this spectrum to the other. Hardware cache coherence is fast, but also costly and timeconsuming to design and implement, while DSM systems provide acceptable performance on only a limit class of applications. We claim that an intermediate hardware optionmemory-mapped network interfaces that support a global physical address space, without cache coherence-can provide most of the performance benefits of fully cache-coherent hardware, at a fraction of the cost. To support this claim we present a software coherence protocol that runs on this class of machines, and use simulation to conduct a performance study. We look at both programming and architectural issues in the context of software and hardware coherence protocols. Our results suggest that software coherence on NCC-NUMA machines in a more cost-effective approach to large-scale sharedmemory multiprocessing than either pure distributed shared memory or hardware cache coherence. a 1995 Academic press, I~C . LEONIDAS KONTOTHANASSIS is a Ph.D. candidate in computer science at the University of Rochester, expected to complete his degree in 1995. His research interests include operating systems and runtime environments and their interaction with computer architecture. His thesis research is exploring the design space of shared memory architectures with an emphasis on flexible coherence mechanisms. He is a member of the ACM and IEEE computer societies. MICHAEL SCOTT is an associate professor of computer science at the University of Rochester. He received his Ph.D. in computer science from the University of Wisconsin at Madison in 1985. His research focuses on systems software for parallel computing, with an emphasis on shared memory programming models. He is the designer of the Lynx programming language and a co-designer of the Psyche parallel operating system. He received an IBM Faculty Development Award in 1986.