SETSUDO: Perturbation-based testing framework for scalable distributed systems

Pallavi Joshi, Malay Ganai, Gogul Balakrishnan, Aarti Gupta, Nadia Papakonstantinou

Research output: Chapter in Book/Report/Conference proceedingConference contribution

16 Scopus citations

Abstract

Modern scalable distributed systems are designed to be partition-tolerant. They are often required to support increasing load in service requests elastically, and to provide seamless services even when some servers malfunction. Partition-tolerance enables such systems to withstand arbitrary loss of messages as "perceived" by the communicating nodes. However, partition-tolerance and robustness are not tested rigorously in practice. Often severe system-level design defects stay hidden even after deployment, possibly resulting in loss of revenue or customer satisfaction. We propose a novel perturbation-based rigorous testing framework, named SETSUDO 1, especially targeted to expose system-level defects in scalable distributed systems. It applies perturbations (i.e., controlled changes) from the environment of a system during testing, and leverages awareness of system-internal states to precisely control their timing. It uses a flexible instrumentation framework to select relevant internal states and to implement the system code for perturbations. It also provides a test policy language framework, where sequences of perturbation scenarios at a high level are converted automatically to system-level test code. This test code is weaved-in automatically with application code during testing, and any observed defects are reported. We have implemented our perturbation testing framework and demonstrate its evaluation on several open source projects, where it was successful in exposing known, as well as some unknown, defects. Our framework leverages small-scale testing, and avoids upfront infrastructure costs typically needed for large-scale stress testing.

Original languageEnglish (US)
Title of host publicationProceedings of the 1st ACM SIGOPS Conference on Timely Results in Operating Systems, TRIOS 2013
PublisherAssociation for Computing Machinery, Inc
ISBN (Electronic)9781450324632
DOIs
StatePublished - Nov 3 2013
Externally publishedYes
Event1st ACM SIGOPS Conference on Timely Results in Operating Systems, TRIOS 2013 - Farmington, United States
Duration: Nov 3 2013Nov 6 2013

Publication series

NameProceedings of the 1st ACM SIGOPS Conference on Timely Results in Operating Systems, TRIOS 2013

Other

Other1st ACM SIGOPS Conference on Timely Results in Operating Systems, TRIOS 2013
CountryUnited States
CityFarmington
Period11/3/1311/6/13

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • Software
  • Computational Theory and Mathematics

Fingerprint Dive into the research topics of 'SETSUDO: Perturbation-based testing framework for scalable distributed systems'. Together they form a unique fingerprint.

  • Cite this

    Joshi, P., Ganai, M., Balakrishnan, G., Gupta, A., & Papakonstantinou, N. (2013). SETSUDO: Perturbation-based testing framework for scalable distributed systems. In Proceedings of the 1st ACM SIGOPS Conference on Timely Results in Operating Systems, TRIOS 2013 (Proceedings of the 1st ACM SIGOPS Conference on Timely Results in Operating Systems, TRIOS 2013). Association for Computing Machinery, Inc. https://doi.org/10.1145/2524211.2524217