Using small abstractions to program large distributed systems

Douglas Thain, Christopher M. Moretti, Hoang Bui, Li Yu, Nitesh Chawla, Patrick Flynn

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Distributed systems such as clusters, clouds and grids remain a difficult platform for executing large data intensive workloads. Even sophisticated users struggle to shape complex workloads into the simple "assembly language" of file transfer and job submission. To address this, problem, we advocate the use of abstractions, which are simple frameworks for expressing large structured problems. In this talk, we will discuss three examples of abstractions developed at the University of Notre Dame for scientific applications. In each case, we have been able to scale up workloads one to wto orders of magnitude larger than we previously feasible. Through each example, we will address some persistent obstacles in the field of distributed computing.

Original languageEnglish (US)
Title of host publicationProceedings - 4th IEEE International Conference on eScience, eScience 2008
Pages723-724
Number of pages2
DOIs
StatePublished - Dec 1 2008
Externally publishedYes
Event4th IEEE International Conference on eScience, eScience 2008 - Indianapolis, IN, United States
Duration: Dec 7 2008Dec 12 2008

Publication series

NameProceedings - 4th IEEE International Conference on eScience, eScience 2008

Conference

Conference4th IEEE International Conference on eScience, eScience 2008
CountryUnited States
CityIndianapolis, IN
Period12/7/0812/12/08

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Computer Science Applications
  • Software

Fingerprint Dive into the research topics of 'Using small abstractions to program large distributed systems'. Together they form a unique fingerprint.

Cite this