Skip to content

How to Run Gene Ontology Analysis on RNA-seq Data in R

Oak
A giant branching root system in a misty glowing forest, surrounded by clusters of luminous fungi in distinct colors and shapes.

By the end of this guide you will have two enrichment tables and a small set of figures from one RNA-seq comparison: an over-representation result from a list of differentially expressed genes, and a gene set enrichment result from the full ranked gene list. You need R 4.3 or newer, a count matrix with gene-level counts, and a sample table describing the contrast. You need these Bioconductor packages as well.

  • DESeq2
  • clusterProfiler
  • org.Hs.eg.db
  • GO.db
  • enrichplot
  • goseq

You also need a real contrast. Gene ontology analysis compares one group of genes against a background, so a single RNA-seq sample with nothing to compare it to has no input. For a personal longitudinal profile the contrast is usually one timepoint against a set of your own earlier timepoints.

1. Understand what GO analysis answers and what it does not

The Gene Ontology is a structured vocabulary of about 40,000 terms describing gene products along three axes: biological process (BP), molecular function (MF), and cellular component (CC). Terms are arranged in a directed acyclic graph, so “T cell activation” sits under “leukocyte activation”, and a gene annotated to the child is implicitly annotated to every ancestor. Enrichment analysis asks whether your gene list hits any term more often than chance would predict, then adjusts for testing thousands of terms at once 1.

The output is a hypothesis about which processes moved. It does not measure any process. A GO term reaching significance means that the genes you selected are annotated to that term more densely than the background is. Annotation density reflects the history of what biologists have studied, so well-worked areas like immune signaling and cell cycle light up easily and poorly annotated areas rarely do. Nothing here supports a clinical conclusion, and any result that looks medically meaningful belongs in a conversation with a physician rather than in a script.

2. Produce a differential expression table you trust

Enrichment inherits every flaw in the table you feed it, so this step matters more than any parameter you will tune later. Start from raw counts rather than TPM or FPKM. Salmon or kallisto quantification imported with tximport, or an alignment-based count matrix from featureCounts, both work.

library(DESeq2)
dds <- DESeqDataSetFromMatrix(countData = counts,
                              colData   = samples,
                              design    = ~ batch + condition)
dds <- dds[rowSums(counts(dds) >= 10) >= 3, ]   # expression filter
dds <- DESeq(dds)
res <- lfcShrink(dds, coef = "condition_post_vs_pre", type = "apeglm")
res <- as.data.frame(res)
res$ensembl <- sub("\\..*$", "", rownames(res))  # strip version suffix

Two choices here shape everything downstream. The expression filter defines your background universe, which we return to in step 4. Shrunken log fold changes from apeglm keep low-count genes from dominating a ranked list with noisy effect sizes.

Read depth determines how many genes you can test at all. For standard differential expression of protein-coding genes, roughly 20 to 30 million paired-end reads per sample is the usual working range, and biological replicates buy you more power than extra depth past that point 2. Deeper libraries mainly help with lowly expressed transcripts and isoform-level work 3. Turnaround from a submitted library to a count matrix is typically two to four weeks in a service lab, with a day or two of that being compute.

3. Map identifiers before anything else

Identifier mapping is where most GO workflows quietly lose 10 to 20 percent of their genes. clusterProfiler wants Entrez gene IDs by default, and your table almost certainly holds Ensembl gene IDs with version suffixes. Strip the suffix first, as above, then map.

library(clusterProfiler)
library(org.Hs.eg.db)

map <- bitr(res$ensembl, fromType = "ENSEMBL",
            toType = "ENTREZID", OrgDb = org.Hs.eg.db)
sum(duplicated(map$ENSEMBL))   # one-to-many mappings
sum(duplicated(map$ENTREZID))  # many-to-one mappings
res <- merge(res, map, by.x = "ensembl", by.y = "ENSEMBL")

Check both duplicate counts and decide deliberately. One Ensembl ID mapping to several Entrez IDs inflates your list. Several Ensembl IDs collapsing onto one Entrez ID deflates it. We keep the first mapping per Ensembl ID and drop the rest, which is crude but reproducible. Record the fraction of genes lost in mapping, because a loss above about 25 percent usually means your annotation source and your quantification reference disagree on a genome build.

You can also skip Entrez entirely by passing keyType = "ENSEMBL" to the enrichment functions. That avoids one mapping step but pushes the same ambiguity inside org.Hs.eg.db. We prefer mapping explicitly so the losses are visible.

4. Choose the background universe on purpose

The default background in most tutorials is every gene in the annotation database. That is wrong for RNA-seq. Your assay could only ever detect genes that are expressed in the tissue you sequenced, so including 10,000 genes that were never transcribed makes every tissue-specific term look enriched.

Use the set of genes that passed your expression filter and survived identifier mapping.

universe_entrez <- unique(res$ENTREZID)          # expressed and mappable
sig <- subset(res, padj < 0.05 & abs(log2FoldChange) > 0.585)  # 1.5x
sig_entrez <- unique(sig$ENTREZID)
length(universe_entrez); length(sig_entrez)

A typical whole-blood experiment gives a universe of 13,000 to 16,000 genes. If your significant list has fewer than about 50 genes, over-representation analysis will be underpowered and you should rely on the GSEA route in step 6 instead. If it has more than about 2,000, nearly every broad term will pass and the result will be uninformative.

5. Run over-representation analysis

Over-representation analysis (ORA) applies a one-sided Fisher exact test, equivalently a hypergeometric test, for each GO term. It asks how surprising it is to draw k genes from that term when sampling your list size from the universe.

ego <- enrichGO(gene          = sig_entrez,
                universe      = universe_entrez,
                OrgDb         = org.Hs.eg.db,
                keyType       = "ENTREZID",
                ont           = "BP",
                pAdjustMethod = "BH",
                pvalueCutoff  = 0.05,
                qvalueCutoff  = 0.10,
                minGSSize     = 10,
                maxGSSize     = 500,
                readable      = TRUE)
head(as.data.frame(ego), 20)

Run BP, MF, and CC as three separate calls rather than using ont = "ALL". Multiple testing correction is applied within a call, so the combined run changes your adjusted p-values for reasons that have nothing to do with biology. minGSSize = 10 and maxGSSize = 500 remove terms too small to be stable and terms so broad that they carry no information.

Read the GeneRatio and BgRatio columns rather than the p-value alone. A term with GeneRatio of 8/200 and BgRatio of 12/15000 is a real fourfold-plus concentration. A term with 90/200 against 4000/15000 is barely moving, even at a tiny p-value, because large gene sets accumulate significance from list size.

We also run ORA separately on up-regulated and down-regulated genes. Mixing directions produces terms where half the genes went one way and half the other, which is nearly impossible to interpret.

6. Run GSEA on the full ranked list

ORA throws away everything below your significance cutoff. Gene set enrichment analysis (GSEA) uses the entire ranked list, which makes it far more sensitive to coordinated small shifts across a pathway. For personal longitudinal data, where effect sizes are usually modest, this is the method we reach for first.

geneList <- res$log2FoldChange
names(geneList) <- res$ENTREZID
geneList <- sort(geneList[!is.na(geneList)], decreasing = TRUE)

set.seed(42)
gse <- gseGO(geneList     = geneList,
             OrgDb        = org.Hs.eg.db,
             keyType      = "ENTREZID",
             ont          = "BP",
             minGSSize    = 15,
             maxGSSize    = 500,
             pvalueCutoff = 0.05,
             eps          = 0,
             seed         = TRUE,
             by           = "fgsea")

The ranking metric is the important decision. Shrunken log2 fold change is a reasonable default. The Wald statistic from results(dds)$stat is a good alternative because it folds in the standard error, which helps when some genes have very uneven coverage. Avoid ranking by p-value alone, since that destroys direction.

Set eps = 0 so fgsea reports p-values below its default floor instead of clamping them. Set the seed explicitly, because permutation p-values shift between runs otherwise. Read NES, the normalized enrichment score, as the direction and magnitude. A positive value means the set concentrates at the top of your ranking and a negative value means it concentrates at the bottom.

Time course designs deserve a different treatment. Comparing every timepoint to baseline generates correlated tests and misses the shape of the trajectory, and methods built for temporal structure handle this better 4. If you have four or more timepoints, cluster genes by trajectory first, then run ORA on each cluster.

7. Collapse redundant terms

Your first BP result will contain “immune response” and “immune system process” alongside “regulation of immune response”. It will also list a dozen near-synonyms, all driven by the same 30 genes. This is an artifact of the ontology graph. The terms are not a set of independent findings.

ego_simple <- simplify(ego, cutoff = 0.7, by = "p.adjust",
                       select_fun = min, measure = "Wang")
nrow(as.data.frame(ego)); nrow(as.data.frame(ego_simple))

simplify computes pairwise semantic similarity between significant terms and keeps the best-scoring representative of each cluster above the cutoff. The Wang measure uses graph topology, while alternatives such as Resnik and Lin use information content derived from annotation frequency 5. A cutoff of 0.7 typically removes half to two thirds of terms. Lower it to 0.5 if the result is still dominated by one branch.

Inspect what was dropped before trusting the reduced table. simplify picks the representative by p-value, which is sometimes the vaguest member of a cluster.

8. Plot and export

Three figures cover most needs. A dot plot ranks terms by gene ratio with dot size for count and color for adjusted p-value. A cnetplot shows which genes drive which terms, which is the fastest way to spot a result resting on five genes. A gseaplot2 shows the running enrichment score for a single set.

library(enrichplot)
dotplot(ego_simple, showCategory = 20) + ggplot2::ggtitle("BP, up-regulated")
ego_pt <- pairwise_termsim(ego_simple)
emapplot(ego_pt, showCategory = 30)
gseaplot2(gse, geneSetID = gse@result$ID[1], title = gse@result$Description[1])

write.csv(as.data.frame(ego_simple), "go_bp_ora.csv", row.names = FALSE)
write.csv(as.data.frame(gse),        "go_bp_gsea.csv", row.names = FALSE)

Record your package and annotation versions alongside the CSVs. GO annotations change every month, and a result from org.Hs.eg.db 3.18 will not reproduce exactly under 3.20. sessionInfo() at the end of the script is the minimum. If you want a lighter-weight path with built-in identifier conversion and plotting, Genekitr wraps much of this workflow with fewer moving parts 6.

Common problems

Everything is “immune response” and nothing else. In whole blood, shifts in cell composition dominate the transcriptome. A change in neutrophil fraction moves hundreds of genes together and produces immune terms regardless of what else happened. Run cell type deconvolution and either adjust for the estimated proportions in your DESeq2 design or interpret the result as compositional.

Gene length bias. Longer genes accumulate more reads and are more likely to reach significance at a given expression level, and long genes are not randomly distributed across GO terms. ORA has no defense against this. goseq does, by fitting a probability weighting function on gene length and using it as a null.

library(goseq)
de_vector <- as.integer(res$ENTREZID %in% sig_entrez)
names(de_vector) <- res$ensembl
pwf <- nullp(de_vector, "hg38", "ensGene")
go_lenadj <- goseq(pwf, "hg38", "ensGene", test.cats = "GO:BP")

Compare the goseq ranking to your enrichGO ranking. Terms that appear in one and not the other are the ones to distrust.

No terms pass correction. Usually the DE list is too small or too diffuse. Move to GSEA, which does not require a cutoff, before loosening thresholds. Relaxing qvalueCutoff to 0.25 and calling the result exploratory is defensible if you label it that way.

Duplicate or missing mappings after bitr. A warning that some percentage of input failed to map is normal at a few percent and alarming above 25 percent. Check that your quantification reference and annotation package use the same genome build.

The universe silently defaults to everything. If you omit universe, enrichGO uses all annotated genes in the OrgDb. Always pass it explicitly and print its length.

Ties in the ranked list. If many genes share the same fold change, fgsea warns about ties and the permutation null becomes less reliable. Rank by the Wald statistic instead, which almost never ties.

Treating GO as the only lens. GO and KEGG pathway enrichment give overlapping but distinct views, and running both is standard practice for functional interpretation 1. Reactome through ReactomePA is a useful third. When three independent resources agree on a theme, the finding is worth following.

RNA-seq tells you which genes were transcribed and at what relative abundance in the tissue sampled at that moment 7. It does not tell you about protein levels, post-translational modification, or causation. GO analysis is a compression of that measurement into readable language, useful for generating the next question and not for answering a clinical one 8. Take anything that looks health-relevant to a clinician.

Footnotes

  1. Vijaykumar Yogesh Muley. Functional Insights Through Gene Ontology, Disease Ontology, and KEGG Pathway Enrichment. Methods in Molecular Biology, 2025. https://doi.org/10.1007/978-1-0716-4546-8_4 ↩ ↩2

  2. Ana Conesa, Pedro Madrigal, Sonia Tarazona, et al. A survey of best practices for RNA-seq data analysis. Genome Biology, 2016. https://doi.org/10.1186/s13059-016-0881-8 ↩

  3. Rory Stark, Marta Grzelak, James Hadfield. RNA sequencing: the teenage years. Nature Reviews Genetics, 2019. https://doi.org/10.1038/s41576-019-0150-2 ↩

  4. Daniel Spies, Constance Ciaudo. Dynamics in Transcriptomics: Advancements in RNA-seq Time Course and Downstream Analysis. Computational and Structural Biotechnology Journal, 2015. https://doi.org/10.1016/j.csbj.2015.08.004 ↩

  5. Guangchuang Yu. Gene Ontology Semantic Similarity Analysis Using GOSemSim. Methods in Molecular Biology, 2020. https://doi.org/10.1007/978-1-0716-0301-7_11 ↩

  6. Yunze Liu, Gang Li. Empowering biologists to decode omics data: the Genekitr R package and web server. BMC Bioinformatics, 2023. https://doi.org/10.1186/s12859-023-05342-9 ↩

  7. Xi Qian, Yi Ba, Qianfeng Zhuang, et al. RNA-Seq Technology and Its Application in Fish Transcriptomics. OMICS: A Journal of Integrative Biology, 2014. https://doi.org/10.1089/omi.2013.0110 ↩

  8. Paul A McGettigan. Transcriptomics in the RNA-seq era. Current Opinion in Chemical Biology, 2013. https://doi.org/10.1016/j.cbpa.2012.12.008 ↩

Pothos builds longitudinal molecular profiles of individuals: whole-genome sequencing, single-cell RNA sequencing, and plasma proteomics, delivered as a detailed report, every data file, and stored cells. Build your profile.