An efficient communication architecture for commodity supercomputers
Proceedings of the 1999 ACM/IEEE conference on Supercomputing (CDROM) - Supercomputing '99
The goal of the Swiss-Tx project is to develop, build and install the first Swiss tera-flop supercomputer called Swiss-T2, which is mainly based on commodity parts. Only the communication hardware and communication software is custom-made, because available off-theshelf products, such as Ethernet with the socket interface, do not offer the necessary bandwidth, latency, and functionality. In this paper, we present a new efficient communication architecture for commodity supercomputing called
... Communication Interface (FCI), and we introduce T-NET, the custom-made high-performance communication hardware for the Swiss-Tx supercomputers. The highlights are low-latency, highbandwidth, and portability. Portability means that the communication hardware and software is mainly platform independent and that a large number of modern workstations and standard operating systems can be used as they are. A full implementation of the standardized MPI (Message Passing Interface), written entirely on top of FCI, is also available. Proceedings of the ACM/IEEE SC99 Conference (SC'99) 1-58113-091-0/99 $ 17.00 © 1999 IEEE Proceedings of the ACM/IEEE SC99 Conference (SC'99) 1-58113-091-0/99 $ 17.00 © 1999 IEEE Proceedings of the ACM/IEEE SC99 Conference (SC'99) 1-58113-091-0/99 $ 17.00 © 1999 IEEE Implementation of the Communication Network T-NET Communication Adapter The T-NET communication adapter is a 32 bit PCI adapter board with DMA capabilities, offering one bidirectional link to the T-NET network with a peak bandwidth of 100 Mbyte/s in each direction. The adapter board is functionally partitioned into four subsystems: the PCI bridge, the communication controller, the link controller and the link transceiver. FIFO buffers, one for data to the network (TX-FIFO) and one for data from the network (RX-FIFO), decouple the communication and the link controller. Additional on-board memory is used to store communication related tables, i.e. the ID validation table, the page table, and the index-to-address translation table (see figure 5 ).