The joint null criterion for multiple hypothesis tests

Jeffrey T. Leek, John D. Storey

Research output: Contribution to journalArticle

13 Scopus citations

Abstract

Simultaneously performing many hypothesis tests is a problem commonly encountered in high-dimensional biology. In this setting, a large set of p-values is calculated from many related features measured simultaneously. Classical statistics provides a criterion for defining what a "correct" p-value is when performing a single hypothesis test. We show here that even when each p-value is marginally correct under this single hypothesis criterion, it may be the case that the joint behavior of the entire set of p-values is problematic. On the other hand, there are cases where each p-value is marginally incorrect, yet the joint distribution of the set of p-values is satisfactory. Here, we propose a criterion defining a well behaved set of simultaneously calculated p-values that provides precise control of common error rates and we introduce diagnostic procedures for assessing whether the criterion is satisfied with simulations. Multiple testing p-values that satisfy our new criterion avoid potentially large study specific errors, but also satisfy the usual assumptions for strong control of false discovery rates and family-wise error rates. We utilize the new criterion and proposed diagnostics to investigate two common issues in high-dimensional multiple testing for genomics: dependent multiple hypothesis tests and pooled versus test-specific null distributions.

Original languageEnglish (US)
Article number28
JournalStatistical Applications in Genetics and Molecular Biology
Volume10
Issue number1
DOIs
StatePublished - 2011

All Science Journal Classification (ASJC) codes

  • Statistics and Probability
  • Molecular Biology
  • Genetics
  • Computational Mathematics

Keywords

  • false discovery rate
  • multiple testing dependence
  • pooled null statistics

Fingerprint Dive into the research topics of 'The joint null criterion for multiple hypothesis tests'. Together they form a unique fingerprint.

  • Cite this