How to Analyze Your Own Genome and Transcriptome in R
By the end of this guide you will have three artifacts on disk: a partitioned Parquet dataset of every variant in your genome with population frequency and coding consequence attached, a SummarizedExperiment holding gene-level RNA-seq counts across your timepoints, and a small set of R functions that join the two by gene. You need R 4.4 or later and Bioconductor 3.20 or later. Plan on about 32 GB of RAM and roughly 200 GB of free disk. On the data side you need a single-sample VCF from whole-genome sequencing (a .vcf.gz plus its .tbi index), the reference build it was called against, and one quant.sf file per RNA-seq sample if you used Salmon. If you have BAM or CRAM alignments as well, keep them: some of the questions below can only be answered by going back to the reads.
1. Install the toolchain and pin it
Genomics packages in R change fast, and annotation packages change faster. Pin everything from the start so a rerun in six months produces the same table.
install.packages(c("BiocManager", "renv", "data.table", "arrow", "duckdb"))
BiocManager::install(version = "3.20")
BiocManager::install(c(
"VariantAnnotation", "GenomicRanges", "plyranges", "GenomicFeatures",
"ensembldb", "AnnotationHub", "GenomicScores", "SummarizedExperiment",
"tximport", "DESeq2", "BSgenome.Hsapiens.UCSC.hg38", "rtracklayer"
))
Then run renv::init() in the project directory and commit the lockfile. Also record the AnnotationHub snapshot date, because AnnotationHub() will silently give you a newer Ensembl release next year:
library(AnnotationHub)
ah <- AnnotationHub()
snapshotDate(ah) # write this into your README
Run BiocManager::valid() after any install. Mixed Bioconductor versions are the single most common cause of cryptic S4 method errors.
2. Normalize the VCF before R sees it
Do this step outside R with bcftools. It is faster there, and it removes two classes of problem that are painful to fix in-memory. First, multi-allelic records make every downstream data.frame ragged. Second, indels that are not left-aligned will fail to match ClinVar or gnomAD records for the same event.
bcftools norm -m -any -f GRCh38.primary_assembly.genome.fa \
-Oz -o sample.norm.vcf.gz sample.vcf.gz
bcftools index -t sample.norm.vcf.gz
bcftools view -f PASS -Oz -o sample.pass.vcf.gz sample.norm.vcf.gz
bcftools index -t sample.pass.vcf.gz
Keep the pre-filter file. A 30x human genome typically yields four to five million SNVs and indels against the reference, and the FILTER column is where the caller records its own doubt. Removing non-PASS records is the right default, but you will occasionally want to look at what was dropped.
Check the contig naming convention now, since it determines how much renaming you do later:
bcftools view -h sample.pass.vcf.gz | grep '^##contig' | head -3
3. Read the VCF in chunks
readVcf will happily try to materialize every INFO field for five million records and take your session with it. The fix is to restrict fields with ScanVcfParam and stream with yieldSize.
library(VariantAnnotation)
library(data.table)
tab <- TabixFile("sample.pass.vcf.gz", yieldSize = 200000)
param <- ScanVcfParam(
fixed = c("ALT", "QUAL", "FILTER"),
info = c("DP"),
geno = c("GT", "AD", "DP", "GQ")
)
flatten <- function(vcf) {
gr <- rowRanges(vcf)
ad <- geno(vcf)$AD[, 1]
data.table(
chrom = as.character(seqnames(gr)),
pos = start(gr),
ref = as.character(ref(vcf)),
alt = as.character(unlist(alt(vcf))),
qual = qual(vcf),
gt = geno(vcf)$GT[, 1],
gq = as.integer(geno(vcf)$GQ[, 1]),
dp = as.integer(geno(vcf)$DP[, 1]),
ad_ref = vapply(ad, function(x) as.integer(x[1]), integer(1)),
ad_alt = vapply(ad, function(x) as.integer(x[2]), integer(1))
)
}
open(tab)
chunks <- list()
repeat {
vcf <- readVcf(tab, "hg38", param = param)
if (nrow(vcf) == 0L) break
chunks[[length(chunks) + 1L]] <- flatten(vcf)
}
close(tab)
variants <- rbindlist(chunks)
The as.character(unlist(alt(vcf))) line only works because you normalized to one ALT per row in step 2. On a non-normalized file it will misalign the whole table, which is a quiet and expensive bug.
Sanity-check the result before going further. Expect a heterozygous-to-homozygous-alt ratio near 2 for a single outbred genome, and expect transitions to outnumber transversions roughly two to one genome-wide.
variants[, .N, by = gt]
variants[, mean(ad_alt / (ad_ref + ad_alt)), by = gt]
Allele fractions clustering near 0.5 for 0/1 calls and near 1.0 for 1/1 calls is the signal that the genotyping is behaving. Systematic deviation points at coverage or mapping problems rather than biology. Low-complexity and repetitive regions are where caller error concentrates, and they account for a disproportionate share of discordant calls between pipelines 1.
4. Write the variant table to Parquet and query it with DuckDB
Five million rows fit in memory, but you will regenerate this table often and you will want to join it against annotation sources that do not fit. Parquet plus DuckDB gives you SQL over the whole thing with no import step.
library(arrow)
write_dataset(variants, "vardata/", partitioning = "chrom", format = "parquet")
library(duckdb)
con <- dbConnect(duckdb())
dbGetQuery(con, "
SELECT chrom, count(*) AS n
FROM read_parquet('vardata/*/*.parquet', hive_partitioning = true)
WHERE gq >= 30 GROUP BY chrom ORDER BY n DESC")
We prefer this over keeping everything in data.table for one reason: the annotation files you are about to join are larger than your genome. The gnomAD sites VCF for GRCh38 is hundreds of gigabytes in raw form. DuckDB lets you push predicates down instead of loading.
5. Attach coding consequences
Now bring in transcript models. Use ensembldb through AnnotationHub rather than a static GTF, because the EnsDb object gives you a queryable database instead of a parsed text file.
library(ensembldb)
query(ah, c("EnsDb", "Homo sapiens", "113")) # note the AH id it prints
edb <- ah[["AH<id from above>"]]
The variants must be in the same coordinate space as the transcripts and the reference sequence. Ensembl uses 1, UCSC uses chr1, and predictCoding will return zero rows rather than warn you about it. Set the style explicitly:
library(BSgenome.Hsapiens.UCSC.hg38)
vcf_all <- readVcf("sample.pass.vcf.gz", "hg38")
seqlevelsStyle(vcf_all) <- "UCSC"
seqlevels(vcf_all, pruning.mode = "coarse") <- paste0("chr", c(1:22, "X", "Y", "M"))
txdb <- GenomicFeatures::makeTxDbFromGRanges(
GenomicFeatures::transcripts(edb)
)
seqlevelsStyle(txdb) <- "UCSC"
coding <- predictCoding(vcf_all, txdb, seqSource = BSgenome.Hsapiens.UCSC.hg38)
table(coding$CONSEQUENCE)
Expect on the order of ten to twenty thousand nonsynonymous changes and a few hundred nonsense or frameshift calls in a typical genome. Most of the apparent loss-of-function calls will be in genes with known tolerance to it, or will be artifacts in poorly mapped regions. Treat the raw count as a starting point for filtering rather than a result.
For population frequency, GenomicScores wraps gnomAD allele frequencies as a score resource, which is far lighter than joining the full sites VCF:
library(GenomicScores)
query(ah, "MafDb.gnomAD")
mafdb <- ah[["AH<id from above>"]]
af <- gscores(mafdb, rowRanges(vcf_all))$AF
Rarity is the main axis of prioritization: filter to allele frequency below 0.001 and the coding set shrinks by well over an order of magnitude. Layering ClinVar on top requires downloading clinvar.vcf.gz from NCBI and reading only the positions you care about:
cv <- readVcf(TabixFile("clinvar.vcf.gz"), "hg38",
param = ScanVcfParam(which = rowRanges(vcf_all)[!is.na(af) & af < 0.001],
info = c("CLNSIG", "GENEINFO")))
A ClinVar hit is a literature pointer and not a finding about you. Assertions vary in evidence level, get reclassified, and depend on the clinical context of the patients they came from. Clinical whole-genome interpretation is done by trained analysts against structured evidence frameworks, and even in well-characterized disease settings it produces a mix of confident calls and variants of uncertain significance 2. If a result in this table would change a medical decision, the next step is a clinician and a validated clinical test. More R is not the next step.
6. Do range arithmetic with plyranges
Most genomic questions are interval questions. plyranges gives GenomicRanges a dplyr grammar, which makes the intent of a filter readable six months later.
library(plyranges)
vgr <- makeGRangesFromDataFrame(
variants[, .(chrom, start = pos, end = pos + nchar(ref) - 1, ref, alt, gt, gq)],
keep.extra.columns = TRUE
)
seqlevelsStyle(vgr) <- "UCSC"
prom <- promoters(GenomicFeatures::genes(txdb), upstream = 2000, downstream = 200)
vgr |>
filter(gq >= 30) |>
join_overlap_inner(prom) |>
group_by(gene_id) |>
summarise(n_variants = n()) |>
arrange(desc(n_variants))
Two habits prevent most bugs here. Always confirm genome(x) matches between objects before an overlap, and always use pruning.mode = "coarse" when dropping scaffolds so ranges and seqinfo stay consistent.
7. Load RNA-seq counts and treat yourself as the experiment
Import transcript-level quantifications and sum them to genes. tximport handles the effective-length correction that makes gene-level counts usable by DESeq2.
library(tximport)
library(SummarizedExperiment)
tx2gene <- transcripts(edb, columns = c("tx_id", "gene_id"), return.type = "data.frame")
files <- setNames(
file.path("quants", c("t1", "t2", "t3", "t4"), "quant.sf"),
c("t1", "t2", "t3", "t4")
)
txi <- tximport(files, type = "salmon", tx2gene = tx2gene,
ignoreTxVersion = TRUE, countsFromAbundance = "lengthScaledTPM")
Here is the part most tutorials get wrong for personal data. A single RNA-seq sample supports no differential expression test at all, because there is no replication from which to estimate dispersion. With four timepoints from one person you have replication, but the contrast you can test is within-subject change over time and nothing else.
library(DESeq2)
coldata <- data.frame(row.names = names(files),
timepoint = factor(c("t1", "t2", "t3", "t4"), ordered = FALSE),
batch = factor(c("A", "A", "B", "B")))
dds <- DESeqDataSetFromTximport(txi, coldata, design = ~ batch)
dds <- dds[rowSums(counts(dds) >= 10) >= 2, ]
vsd <- vst(dds, blind = TRUE)
plotPCA(vsd, intgroup = "batch")
Run the PCA before any model. With longitudinal personal data, library preparation batch and blood cell composition usually dominate the first two components. If batch separates your samples more than time does, the interesting variance is confounded and you should say so rather than model around it.
For the single-sample case, the useful move is to position your expression against a reference panel instead of testing it. GTEx provides median TPM per gene per tissue across hundreds of donors, which lets you convert your values into ranks or outlier-resistant z-scores within the matching tissue 3. Rank-based comparison is the conservative choice: absolute TPM differs systematically across quantification pipelines and library chemistries, while the ordering of genes within a sample is much more stable.
gtex <- fread("gtex_median_tpm_wholeblood.tsv") # gene_id, median_tpm
me <- data.table(gene_id = sub("\\..*", "", rownames(txi$abundance)),
tpm = txi$abundance[, "t1"])
cmp <- merge(me, gtex, by = "gene_id")
cmp[, delta := log2((tpm + 1) / (median_tpm + 1))]
cmp[order(-abs(delta))][1:20]
Interpret this as a description of your sample relative to a population reference. It is not a test, and a large delta for one gene in one draw is usually technical.
8. Join the genome to the transcriptome
The reason to hold both layers in one project is that each constrains the interpretation of the other. GTEx showed that common regulatory variants have measurable and often tissue-specific effects on expression levels 3. Large integrated cancer studies made the same point from the other direction, where DNA-level events only became interpretable once expression and methylation were placed alongside them 4.
A concrete and checkable join is allele-specific expression. Take heterozygous SNVs from the genome, count RNA reads carrying each allele, and look for sites where one allele dominates.
het <- vgr |> filter(gt %in% c("0/1", "1/0"), gq >= 30, width == 1)
library(GenomicAlignments)
pu_param <- PileupParam(min_base_quality = 20, distinguish_strands = FALSE,
max_depth = 5000)
pu <- pileup("rna_t1.bam", scanBamParam = ScanBamParam(which = het),
pileupParam = pu_param)
Then take sites with at least 20 RNA reads and test each against a 0.5 expectation with binom.test. Two caveats decide whether the output means anything. Reads carrying the alternate allele map slightly worse to the reference, which biases the reference allele upward, so filter out sites in regions of low mappability. Read counts at nearby sites are correlated through shared reads and haplotype structure, so treat each gene rather than each site as the unit of inference.
Store the joined result as a SummarizedExperiment with expression in the assay and variant annotations in rowData. That keeps gene identifiers aligned across layers, which is the single largest source of silent error in multi-omic code.
Common problems
Reference build mismatch is the failure that wastes the most time. A GRCh37 VCF against GRCh38 transcripts produces plausible-looking output that is entirely wrong. Check genome() on every object, and if you must convert, use rtracklayer::liftOver with the UCSC chain file and accept that some intervals will not map.
Seqlevel style mismatch produces zero-row results with no warning. Set seqlevelsStyle() immediately after every import and make it a habit.
Memory blowups in readVcf come from unrestricted INFO fields. Always pass ScanVcfParam with an explicit field list, and always set yieldSize when streaming.
Gene identifier drift breaks joins quietly. Use versionless Ensembl gene IDs as the key and keep symbols as a display column only, since symbols are reassigned between releases.
Unfiltered variant tables invite over-reading. Recurrent false positives concentrate in homopolymers, segmental duplications and other low-complexity sequence, and they survive naive quality filters 1. Requiring both a genotype quality threshold and reasonable allele balance removes most of them.
Single-sample differential expression is not a valid analysis. If you have one RNA-seq run, describe it against a reference panel and stop there.
Finally, do not read a filtered variant list as a set of conclusions about your health. Interpretation across molecular layers is a structured process built on cohort-scale evidence 5, and the clinical step belongs to a clinician working from a validated assay.
Footnotes
-
D. C. Koboldt, L. Ding, E. R. Mardis, et al. Challenges of sequencing human genomes. Briefings in Bioinformatics, 2010. https://doi.org/10.1093/bib/bbq016 ↩ ↩2
-
Sarra L. Ryan, John F. Peden, Zoya Kingsbury, et al. Whole genome sequencing provides comprehensive genetic testing in childhood B-cell acute lymphoblastic leukaemia. Leukemia, 2023. https://doi.org/10.1038/s41375-022-01806-8 ↩
-
GTEx Consortium. Genetic effects on gene expression across human tissues. Nature, 2017. https://doi.org/10.1038/nature24277 ↩ ↩2
-
The Cancer Genome Atlas Network. Comprehensive molecular portraits of human breast tumours. Nature, 2012. https://doi.org/10.1038/nature11412 ↩
-
The Cancer Genome Atlas Research Network. Comprehensive genomic characterization defines human glioblastoma genes and core pathways. Nature, 2008. https://doi.org/10.1038/nature07385 ↩
Pothos builds longitudinal molecular profiles of individuals: whole-genome sequencing, single-cell RNA sequencing, and plasma proteomics, delivered as a detailed report, every data file, and stored cells. Build your profile.