Highly scalable genome assembly on campus grids

Christopher Moretti, Michael Olson, Scott Emrich, Douglas Thain

Research output: Chapter in Book/Report/Conference proceedingConference contribution

8 Scopus citations

Abstract

Bioinformatics researchers need efficient means to process large collections of sequence data. One application of interest, genome assembly, has great potential for parallelization, however most previous attempts at parallelization require uncommon high-end hardware. This paper introduces a scalable modular genome assembler that can achieve significant speedup using large numbers of conventional desktop machines, such as those found in a campus computing grid. The system is based on the Celera open-source assembly toolkit, and replaces two independent sequential modules with scalable replacements: a scalable candidate selector exploits the distributed memory capacity of a campus grid, while the scalable aligner exploits the distributed computing capacity. For large problems, these modules provide robust task and data management while also achieving speedup with high efficiency on several scales of resources. We show results for several datasets ranging from 738 thousand to over 121 million alignments using campus grid resources ranging from a small cluster to more than a thousand nodes spanning three institutions. Our largest run so far achieves a 927x speedup with 71.3 percent efficiency.

Original languageEnglish (US)
Title of host publicationProceedings of the 2nd ACM Workshop on Many-Task Computing on Grids and Supercomputers 2009, MTAGS '09
DOIs
StatePublished - Dec 1 2009
Externally publishedYes
Event2nd ACM Workshop on Many-Task Computing on Grids and Supercomputers 2009, MTAGS '09 - Portland, OR, United States
Duration: Nov 16 2009Nov 16 2009

Publication series

NameProceedings of the 2nd ACM Workshop on Many-Task Computing on Grids and Supercomputers 2009, MTAGS '09

Conference

Conference2nd ACM Workshop on Many-Task Computing on Grids and Supercomputers 2009, MTAGS '09
CountryUnited States
CityPortland, OR
Period11/16/0911/16/09

All Science Journal Classification (ASJC) codes

  • Hardware and Architecture
  • Software

Fingerprint Dive into the research topics of 'Highly scalable genome assembly on campus grids'. Together they form a unique fingerprint.

  • Cite this

    Moretti, C., Olson, M., Emrich, S., & Thain, D. (2009). Highly scalable genome assembly on campus grids. In Proceedings of the 2nd ACM Workshop on Many-Task Computing on Grids and Supercomputers 2009, MTAGS '09 [1646480] (Proceedings of the 2nd ACM Workshop on Many-Task Computing on Grids and Supercomputers 2009, MTAGS '09). https://doi.org/10.1145/1646468.1646480