Skip to content

How to Apply FDR Correction to Your Own Omics Data

Oak
A ranked-scaled iridescent reptile-bird specimen on a black plinth, most scales dull grey, a small ascending band glowing green-gold.

By the end of this guide you will have a reproducible script that takes a table of p-values from your own molecular data and returns q-values, a count of discoveries at a stated false discovery rate, and the diagnostics that say whether that number means anything. You need a table with one row per feature (a gene, transcript, protein, or CpG site) and a column of raw p-values from a test you trust. You need R 4.x with qvalue and IHW from Bioconductor, or Python with statsmodels. We will use R for most examples because the reference implementations live there.

The short answer to the common question is this. A false discovery rate correction converts each p-value into a q-value, and a q-value threshold of 0.05 means that among the features you call significant, you expect about 5% to be false positives. That is a different promise from the one Bonferroni makes, and the difference matters when you are testing 20,000 genes instead of five biomarkers.

1. Define the family of tests before you correct anything

The FDR is a property of a set, so the first decision is which tests belong in the set. If you ran differential expression across 18,000 expressed genes in one RNA-seq comparison, the family is those 18,000 tests. If you then ran the same comparison on 7,000 SomaScan proteins, that is a second family. Correcting each separately is standard practice and is what almost every omics tool does by default.

The failure mode here is defining the family after you look at the results. Testing 18,000 genes, noticing that a lipid metabolism gene looks interesting, and then correcting only within a 40-gene lipid pathway is a way of manufacturing significance. Decide the family from the design, write it down, and keep it fixed.

The opposite error is also common. Pooling unrelated assays into one family of 30,000 tests costs you power for no statistical benefit, because the tests were never a single hypothesis set. In multi-omic work the usual convention is to control FDR within each data layer and then look for agreement across layers, which is how integrated omics analyses are typically reported 1.

2. Look at the p-value histogram before you correct

This is the step people skip, and it is the one that catches most real problems. Plot the raw p-values in 50 bins from 0 to 1.

hist(res$pvalue, breaks = 50, col = "grey", main = "raw p-values")

A well-behaved result looks like a flat plateau across most of the range with a spike near zero. The plateau is the null tests, which are uniform under a correct test. The spike is your signal. Every FDR method assumes that plateau exists.

Three shapes should stop you. A hump in the middle around 0.3 to 0.6 usually means your test statistic is conservative, often from a variance estimate that is too large or from discrete count data with few reads. A rising slope toward 1.0 means something is badly misspecified, frequently a paired design analyzed as unpaired. A histogram with no plateau at all, sloping steeply from 0 to 1 across the whole range, usually means uncorrected batch structure or confounding rather than 12,000 real discoveries.

If the histogram is wrong, fix the model. Running Benjamini-Hochberg on invalid p-values produces q-values that are precise and meaningless. The interpretation of any omics result rests on the assumption that the underlying test was appropriate for the data 2.

3. Compute Benjamini-Hochberg q-values

Benjamini-Hochberg is a step-up procedure. Sort the m p-values in ascending order. For rank i, compare p(i) to (i/m) × α. Find the largest i where the comparison passes, and reject every hypothesis up to that rank. The adjusted p-value form of the same procedure is what you will use in practice.

The full implementation is six lines, and writing it once is worth more than reading a description of it:

bh <- function(p) {
  m  <- length(p)
  o  <- order(p, decreasing = TRUE)
  ro <- order(o)
  pmin(1, cummin(p[o] * m / (m:1)))[ro]
}

The cummin running from the largest p-value downward enforces monotonicity, so a q-value is never smaller than the q-value of a more significant feature. That step is why a single q-value cannot be computed from a single p-value. You need the whole vector.

In practice, use the built-in, which is the same algorithm:

res$qvalue <- p.adjust(res$pvalue, method = "BH")
sum(res$qvalue < 0.05, na.rm = TRUE)

In Python:

from statsmodels.stats.multitest import multipletests

rej, q, _, _ = multipletests(pvals, alpha=0.05, method="fdr_bh")
print(rej.sum())

Bonferroni, for comparison, is p.adjust(p, method = "bonferroni"), which multiplies each p-value by m and caps it at 1. It controls the probability of even one false positive across the entire family. On 18,000 genes that means a raw p-value threshold of 2.8 × 10⁻⁶. Use Bonferroni when a single false positive is expensive, such as returning a variant call for clinical follow-up. Use BH when you are producing a ranked candidate list and can tolerate a known fraction of noise in it.

4. Use Storey q-values when a large fraction of tests are non-null

Benjamini-Hochberg assumes, conservatively, that all m hypotheses could be null. If 30% of your proteins genuinely differ between conditions, BH is leaving power on the table by a factor of roughly 1/π₀, where π₀ is the true null proportion. Storey’s method estimates π₀ from the flat part of the p-value histogram and scales the q-values accordingly.

library(qvalue)
qobj <- qvalue(res$pvalue)
qobj$pi0                       # estimated null proportion
res$qval_storey <- qobj$qvalues
sum(res$qval_storey < 0.05)

If pi0 comes back at 0.92, Storey and BH will give nearly identical results and you should prefer BH for simplicity. If pi0 is 0.60, Storey will typically return 10% to 25% more discoveries at the same nominal FDR. If pi0 comes back at exactly 1.0, or the function warns about the π₀ estimate, your histogram probably has no clean plateau. Go back to step 2.

One guardrail: Storey needs a reasonable number of tests to estimate π₀ well. With fewer than about 1,000 p-values the estimate gets unstable, and BH is the safer choice.

5. Filter before testing and not after

Every feature you test costs power for every other feature. In RNA-seq, thousands of genes have near-zero counts and no realistic chance of reaching significance, but they still inflate m. Removing them before correction increases discoveries without inflating the FDR, provided the filter statistic is independent of the p-value under the null.

Read depth is such a statistic. A concrete filter for a gene-level count matrix:

keep <- rowSums(counts(dds) >= 10) >= 3
dds  <- dds[keep, ]

DESeq2 does a version of this automatically inside results(), sweeping a quantile of mean normalized count to maximize rejections. Set alpha to the threshold you will use, because the filter is tuned to it:

res <- results(dds, alpha = 0.05)
sum(res$padj < 0.05, na.rm = TRUE)

If you leave alpha at its default of 0.1 and then apply a 0.05 cutoff, the filtering step was optimized for the wrong target and you lose discoveries.

The generalization is independent hypothesis weighting, which uses a covariate to assign each test a weight rather than a hard keep-or-drop decision:

library(IHW)
ihwRes <- ihw(pvalue ~ baseMean, data = as.data.frame(res), alpha = 0.05)
rejections(ihwRes)
res$qval_ihw <- adj_pvalues(ihwRes)

Good covariates are mean expression for RNA-seq, mean intensity or detection frequency for proteomics, and minor allele frequency for variant association. The covariate must be independent of the p-value under the null and informative about power. Group size in a differential test is not a valid covariate, because it is entangled with the test statistic itself.

6. Handle correlation and discreteness

Benjamini-Hochberg controls FDR exactly under independence and under positive regression dependency, which covers most expression data where co-regulated genes correlate positively. It is not guaranteed under arbitrary dependence. The Benjamini-Yekutieli variant divides the threshold by the harmonic sum of 1 through m, which at m = 18,000 is roughly 10.5:

res$qval_by <- p.adjust(res$pvalue, method = "BY")

That is a tenfold penalty, and it will usually collapse your discovery list. We reach for BY only when we have concrete reason to expect strong negative dependence, which is rare in molecular data. The more useful alternative for correlated tests is a permutation-based null. Shuffle the sample labels 1,000 times, recompute the test statistic for every feature in every permutation, and estimate the FDR at any threshold as the mean number of permuted features passing it divided by the number of observed features passing it. This preserves the correlation structure across features because you permute samples. The features are not permuted.

Discrete p-values are the other special case. Tests on small counts, such as Fisher’s exact test on a 2 × 2 table with fewer than 20 observations, can only produce a limited set of p-values. The uniform null assumption fails, and BH becomes conservative. If your histogram shows spikes at a handful of discrete values, use a discrete-aware method rather than the standard procedure.

Correlation also matters inside single-person longitudinal data. If you test 5,000 proteins against time across 12 draws, consecutive draws are autocorrelated, and treating them as independent observations inflates the test statistic before any correction happens. Model the time structure explicitly. Methods that handle temporal dependence in omics association testing exist precisely because ignoring it distorts the error rate 3.

7. Read the number correctly

A q-value of 0.03 for a gene means the following. If you draw your significance line at that gene, the expected proportion of false positives among everything at or above the line is 3%. The q-value describes the set and not the individual feature. It is not the probability that this particular gene is a false positive. The local false discovery rate, available as qobj$lfdr from the qvalue package, does estimate something close to that.

So a 5% FDR on a list of 400 significant genes means you expect about 20 of them to be null. Which 20 is unknown. That is the correct mental model for treating the output as a ranked candidate list rather than a set of facts.

Report both numbers. “412 of 17,900 tested genes at BH q < 0.05, π₀ estimated at 0.81” is interpretable. “412 significant genes” is not, because the reader cannot tell what family or threshold produced it.

A note on scope: none of this makes a q-value a clinical finding. A significant association in your own data is a hypothesis about your biology. Anything that touches diagnosis or management belongs with a physician who can order confirmatory testing.

8. Decide what to follow up

FDR control gives you a list. It does not tell you which items on the list will replicate in a second sample or a second assay. Control and replication are different questions, and the second one has its own statistical treatment. Work on validation target selection shows that the features most likely to replicate are not simply the ones with the smallest p-values, and that effect size and measurement reliability should enter the ranking 4.

Our practical rule is to rank candidates by three things together: q-value, absolute effect size, and whether the signal appears in more than one molecular layer. A protein that moves with q = 0.04 and whose transcript moves in the same direction at q = 0.11 is a better follow-up than a transcript at q = 0.001 with a log₂ fold change of 0.08. Cross-layer agreement is the cheapest replication you have when you own genome, transcriptome, and proteome data on the same person.

If you are building a feature selection step rather than reporting discoveries, note that selection procedures with built-in FDR control exist and are designed for exactly the omics-wide setting where the number of features far exceeds the number of samples 5.

Common problems

The q-values are all 1.0. Either you have no signal, or m is very large and the smallest p-value is above roughly α/m. Check the smallest raw p-value against that ratio. If your minimum p is 1 × 10⁻⁴ and m is 20,000, the best possible BH q-value is 2.0. That is capped at 1.0, so nothing can pass.

p.adjust returned fewer significant results than you counted by hand with the step-up rule. You likely stopped at the first p-value that failed the (i/m)α comparison. The step-up procedure rejects up to the largest passing rank. It does not stop at the first failing one. A later, larger p-value can pull earlier ones back in.

Your table has NA p-values. p.adjust with n = length(p) counts them in m by default in some code paths, and DESeq2 sets padj to NA for filtered genes deliberately. Always use na.rm = TRUE when counting discoveries, and confirm what m your function used.

You corrected twice. Passing a padj column into p.adjust a second time is a real and frequent bug in pipelines that stitch tools together. Check the column name and the documentation of whatever produced it. Most differential expression tools return adjusted values already.

The discovery count changes when you rerun the pipeline. Filtering thresholds, gene annotation versions, and the alpha argument all change m. Pin the annotation version, record the filter, and record α alongside the result.

You are testing one person at many timepoints and everything is significant. This is usually autocorrelation rather than biology. Check whether your test treats repeated draws from the same person as independent samples. Block permutation, where you shuffle contiguous time blocks instead of individual points, gives a more defensible null.

You want to combine evidence across layers instead of correcting each separately. That is a reasonable goal and there are formal ways to do it, including causal inference frameworks applied to paired molecular datasets 6. Do not improvise it by taking the minimum q-value across layers, because that quietly reintroduces the multiplicity you just corrected for.

Footnotes

  1. Marc Chadeau‐Hyam, Gianluca Campanella, Thibaut Jombart, et al. Deciphering the complex: Methodological overview of statistical models to derive OMICS‐based biomarkers. Environmental and Molecular Mutagenesis, 2013. https://doi.org/10.1002/em.21797 ↩

  2. Ryo Yamada, Daigo Okada, Juan Wang, et al. Interpretation of omics data analyses. Journal of Human Genetics, 2020. https://doi.org/10.1038/s10038-020-0763-5 ↩

  3. Milan N Parikh, Erika Rasnick Manning, Liang Niu, et al. Increasing temporal sensitivity of omics association studies with epigenome-wide distributed lag models. American Journal of Epidemiology, 2024. https://doi.org/10.1093/aje/kwae375 ↩

  4. Jeffrey T Leek, Margaret A Taub, Jason L Rasgon. A statistical approach to selecting and confirming validation targets in -omics experiments. BMC Bioinformatics, 2012. https://doi.org/10.1186/1471-2105-13-150 ↩

  5. Zi-tong Lu, Xue-ting Song, Yu-fan Gao, et al. A novel model-free feature selection method with FDR control for omics-wide association analysis. 2024. https://doi.org/10.1101/2024.03.01.582911 ↩

  6. Chendong Jiang, Zhan Sun, Zhanyan Gao, et al. Shared Genetic Basis of Autoimmune Diseases and Follicular Lymphoma by Mendelian Randomization and Multi-Omics. Genes, 2026. https://doi.org/10.3390/genes17091031 ↩

Pothos builds longitudinal molecular profiles of individuals: whole-genome sequencing, single-cell RNA sequencing, and plasma proteomics, delivered as a detailed report, every data file, and stored cells. Build your profile.