Design and implementation of virtual memory-mapped communication on Myrinet

This paper describes the design and implementation of the virtual memory-mapped communication model (VMMC) on a Myrinet network of PCI-based PCs. VMMC has been designed and implemented for the SHRIMP multicomputer where it delivers user-to-user latency and bandwidth close to the limits imposed by the underlying hardware. The goal of this work is to provide an implementation of VMMC on a commercially available hardware platform; to determine whether the benefits of VMMC can be realized on the new hardware; and to investigate network interface design tradeoffs by comparing SHRIMP with Myrinet and its respective VMMC implementation. Our Myrinet implementation of VMMC achieves 9.8 microseconds one-way latency and provides 108.4 MBytes per second user-to-user bandwidth. Compared to SHRIMP, the Myrinet implementation of VMMC incurs relatively higher overhead and demands more network interface resources (LANai processor, on-board SRAM) but requires less operating system support.