Latency, occupancy, and bandwidth in DSM multiprocessors: A performance evaluation

Mainak Chaudhuri, Mark Heinrich, Chris Holt, Jaswinder Pal Singh, Edward Rothberg, John Hennessy

Research output: Contribution to journalArticlepeer-review

11 Scopus citations


While the desire to use commodity parts in the communication architecture of a DSM multiprocessor offers advantages in cost and design time, the impact on application performance is unclear. We study this performance impact through detailed simulation, analytical modeling, and experiments on a flexible DSM prototype, using a range of parallel applications. We adapt the logP model to characterize the communication architectures of DSM machines. The l (network latency) and o (controller occupancy) parameters are the keys to performance in these machines, with the g (node-to-network bandwidth) parameter becoming important only for the fastest controllers. We show that, of all the logP parameters, controller occupancy has the greatest impact on application performance. Of the two contributions of occupancy to performance degradation the latency it adds and the contention it induces-it is the contention component that governs performance regardless of network latency, showing a quadratic dependence on o. As expected, techniques to reduce the impact of latency make controller occupancy a greater bottleneck. Surprisingly, the performance impact of occupancy is substantial, even for highly-tuned applications and even in the absence of latency hiding techniques. Scaling the problem size is often used as a technique to overcome limitations in communication latency and bandwidth. Through experiments on a DSM prototype, we show that there are important classes of applications for which the performance lost by using higher occupancy controllers cannot be regained easily, if at all, by scaling the problem size.

Original languageEnglish (US)
Pages (from-to)862-880
Number of pages19
JournalIEEE Transactions on Computers
Issue number7
StatePublished - Jul 2003

All Science Journal Classification (ASJC) codes

  • Software
  • Theoretical Computer Science
  • Hardware and Architecture
  • Computational Theory and Mathematics


  • Bandwidth
  • Communication controller
  • Distributed shared memory multiprocessors
  • Flexible node controller
  • Latency
  • Occupancy
  • Queuing model


Dive into the research topics of 'Latency, occupancy, and bandwidth in DSM multiprocessors: A performance evaluation'. Together they form a unique fingerprint.

Cite this