Using controls to limit false discovery in the era of big data

Matthew M. Parks, Benjamin J. Raphael, Charles E. Lawrence

Research output: Contribution to journalArticlepeer-review

Abstract

Background: Procedures for controlling the false discovery rate (FDR) are widely applied as a solution to the multiple comparisons problem of high-dimensional statistics. Current FDR-controlling procedures require accurately calculated p-values and rely on extrapolation into the unknown and unobserved tails of the null distribution. Both of these intermediate steps are challenging and can compromise the reliability of the results. Results: We present a general method for controlling the FDR that capitalizes on the large amount of control data often found in big data studies to avoid these frequently problematic intermediate steps. The method utilizes control data to empirically construct the distribution of the test statistic under the null hypothesis and directly compares this distribution to the empirical distribution of the test data. By not relying on p-values, our control data-based empirical FDR procedure more closely follows the foundational principles of the scientific method: that inference is drawn by comparing test data to control data. The method is demonstrated through application to a problem in structural genomics. Conclusions: The method described here provides a general statistical framework for controlling the FDR that is specifically tailored for the big data setting. By relying on empirically constructed distributions and control data, it forgoes potentially problematic modeling steps and extrapolation into the unknown tails of the null distribution. This procedure is broadly applicable insofar as controlled experiments or internal negative controls are available, as is increasingly common in the big data setting.

Original languageEnglish (US)
Article number323
JournalBMC bioinformatics
Volume19
Issue number1
DOIs
StatePublished - Sep 14 2018

All Science Journal Classification (ASJC) codes

  • Applied Mathematics
  • Molecular Biology
  • Structural Biology
  • Biochemistry
  • Computer Science Applications

Keywords

  • Big data
  • False discovery rate (FDR)
  • High dimensional inference
  • Hypothesis testing

Fingerprint

Dive into the research topics of 'Using controls to limit false discovery in the era of big data'. Together they form a unique fingerprint.

Cite this