Harnessing parallelism in multicore clusters with the all-pairs and wavefront abstractions

Li Yu, Christopher Moretti, Scott Emrich, Kenneth Judd, Douglas Thain

Research output: Chapter in Book/Report/Conference proceedingConference contribution

13 Scopus citations

Abstract

Both distributed systems and multicore computers are difficult programming environments. Although the expert programmer may be able to tune distributed and multicore computers to achieve high performance, the non-expert may struggle to achieve a program that even functions correctly. We argue that high level abstractions are an effective way of making parallel computing accessible to the non-expert. An abstraction is a regularly structured framework into which a user may plug in simple sequential programs to create very large parallel programs. By virtue of a regular structure and declarative specication, abstractions may be materialized on distributed, multicore, and distributed multicore systems with robust performance across a wide range of problem sizes. In previous work, we presented the All-Pairs abstraction for computing on distributed systems of single CPUs. In this paper, we extend All-Pairs to multicore systems, and introduce Wavefront, which represents a number of problems in economics and bioinformatics. We demonstrate good scaling of both abstractions up to 32-cores on one machine and hundreds of cores in a distributed system.

Original languageEnglish (US)
Title of host publicationProc. 18th ACM International Symposium on High Performance Distributed Computing, HPDC 09, Co-located with the 2009 International Symposium on High Performance Distributed Computing Conf., HPDC'09
Pages1-10
Number of pages10
DOIs
StatePublished - Nov 23 2009
Externally publishedYes
Event18th ACM International Symposium on High Performance Distributed Computing, HPDC 09, Co-located with the 2009 International Symposium on High Performance Distributed Computing Conference, HPDC'09 - Garching, Germany
Duration: Jun 11 2009Jun 13 2009

Publication series

NameProc. 18th ACM International Symposium on High Performance Distributed Computing, HPDC 09, Co-located with the 2009 International Symposium on High Performance Distributed Computing Conf., HPDC'09

Conference

Conference18th ACM International Symposium on High Performance Distributed Computing, HPDC 09, Co-located with the 2009 International Symposium on High Performance Distributed Computing Conference, HPDC'09
CountryGermany
CityGarching
Period6/11/096/13/09

All Science Journal Classification (ASJC) codes

  • Computational Theory and Mathematics
  • Computer Science Applications
  • Software

Keywords

  • Abstractions
  • Bioinformatics
  • Economics
  • Multicore

Fingerprint Dive into the research topics of 'Harnessing parallelism in multicore clusters with the all-pairs and wavefront abstractions'. Together they form a unique fingerprint.

  • Cite this

    Yu, L., Moretti, C., Emrich, S., Judd, K., & Thain, D. (2009). Harnessing parallelism in multicore clusters with the all-pairs and wavefront abstractions. In Proc. 18th ACM International Symposium on High Performance Distributed Computing, HPDC 09, Co-located with the 2009 International Symposium on High Performance Distributed Computing Conf., HPDC'09 (pp. 1-10). [1551613] (Proc. 18th ACM International Symposium on High Performance Distributed Computing, HPDC 09, Co-located with the 2009 International Symposium on High Performance Distributed Computing Conf., HPDC'09). https://doi.org/10.1145/1551609.1551613