Computational Genomics with Python: The Stack That Works on Your Own Data
Python is the query and modeling layer for personal genomics. It is not the alignment layer. The heavy lifting (read alignment, variant calling, transcript quantification) belongs to compiled tools that have been tuned for a decade: bwa-mem2 or DRAGEN, GATK or DeepVariant, STAR or salmon. What Python gives you is everything after that. It pulls variants out of a 2 GB VCF in milliseconds and joins them against gene models. It folds in expression and protein and glucose data, and builds a longitudinal model of one person. This page describes the stack we would install and the specific conventions that break analyses when you ignore them.
What Python is good for here
A 30x whole genome produces a CRAM of roughly 15 to 20 GB and a VCF with about 4 to 5 million variants, most of them common SNVs. None of that fits comfortably in a naive pandas DataFrame, and none of it needs to. The useful pattern is to keep the big files in their indexed binary formats and use Python bindings to reach into them.
This is why the first instinct of many engineers, to load the VCF with pandas.read_csv, is the wrong one. You lose the index, you spend gigabytes of RAM on string columns, and you have to reimplement genotype parsing. Use a library that reads the compressed, indexed file directly.
The stack we would install
Install these and stop there for the first month:
cyvcf2for VCF and BCF reading. It wraps htslib and is roughly an order of magnitude faster thanpyvcffor whole-genome scans.pysamfor BAM/CRAM access, pileups, and FASTA random access.pyrangesfor genomic interval arithmetic (overlaps, nearest gene, coverage).polarsfor tabular work. It handles multi-million-row joins in a fraction of the memory pandas needs, and its lazy API reads Parquet without materializing everything.scanpyandanndataif you have single-cell or many-sample expression data.anndatais a good container even for bulk RNA-seq.pydeseq2for differential expression when you have replicate structure.snakemakeornextflowto make any of this reproducible.
Add hail only if you are working with cohorts of thousands. For one person it is overhead.
Reading a VCF without loading it into pandas
The core loop looks like this. Assume a bgzipped, tabix-indexed VCF from your sequencing provider.
from cyvcf2 import VCF
vcf = VCF("sample.hard-filtered.vcf.gz", gts012=True)
for v in vcf("chr13:32315000-32400000"): # BRCA2 locus
if v.FILTER is not None: # None means PASS
continue
if v.gt_types[0] == 0: # 0 = hom-ref
continue
print(v.CHROM, v.POS, v.REF, v.ALT[0],
v.gt_types[0], v.gt_depths[0], v.QUAL)
Two details matter. In cyvcf2, v.FILTER is None for PASS records, which reverses the intuition of most people reading it for the first time. And gts012=True gives you the sane encoding. It codes 0 for hom-ref, 1 for het, 2 for hom-alt, and 3 for unknown. Without it you get the htslib encoding and will misclassify genotypes.
Region queries against the tabix index are effectively free. A full-genome scan of 5 million records with light per-record work runs in about 30 to 90 seconds on a laptop. That is fast enough that you should not cache intermediate variant tables until you have a reason.
Before any of this, normalize the file once with bcftools:
bcftools norm -m -any -f GRCh38.primary_assembly.fa \
-Oz -o norm.vcf.gz sample.vcf.gz
bcftools index -t norm.vcf.gz
-m -any splits multi-allelic records into one row per alternate allele, and -f left-aligns indels against the reference. Skipping this step is the most common reason a variant you expect to find is missing from your Python filter results: it was hiding as the second ALT allele of a multi-allelic record.
Interval arithmetic is where hand-rolled code breaks
Overlap queries look easy and are not. Writing a nested loop over variants and gene intervals is O(n·m) and will take hours on a whole genome. pyranges builds an interval tree and does the same join in seconds.
import pyranges as pr
genes = pr.read_gtf("gencode.v44.annotation.gtf.gz")
genes = genes[genes.Feature == "gene"]
variants = pr.PyRanges(chromosomes=chroms, starts=pos-1, ends=pos)
annotated = variants.join(genes)
Note starts=pos-1. VCF and GFF/GTF are 1-based and inclusive. BED and pyranges are 0-based and half-open. Mixing them shifts every feature by one base, which is invisible in aggregate statistics and wrong at every boundary. We write a single conversion helper and use it everywhere rather than doing the arithmetic inline.
RNA-seq: from quant.sf to a modelable table
If you have salmon or kallisto output, each sample gives you a quant.sf with roughly 250,000 transcript rows. You almost always want gene-level values. Build a transcript-to-gene map from the same GENCODE GTF you used for annotation, then sum TPM within gene and sum estimated counts within gene.
Use counts, not TPM, as the input to any differential model. TPM is already normalized per sample and violates the count assumptions of pydeseq2. Keep TPM for plotting and for comparing one gene across your own timepoints.
For a single person sampled repeatedly, the analysis is usually descriptive. Compute a per-gene z-score against your own prior timepoints, filter to genes with mean count above roughly 10, and rank by deviation. Reserve pydeseq2 for designs where you genuinely have three or more samples per group. Batch effects between sequencing runs will dominate anything subtler than a twofold change, so record the run date and library prep as covariates from the start.
Alignment-free methods are worth knowing about
Not every question requires a BAM. K-mer based methods work directly on FASTQ and are straightforward to prototype in Python with numpy hashing before you port the hot loop to Rust or C. Recent work applies exactly this idea to circulating tumor DNA, estimating tumor fraction from whole-genome sequencing without aligning reads at all1. That approach is a research method rather than something to interpret on your own, but it illustrates the general point: the alignment step is a choice, and sometimes an expensive one.
Making the environment reproducible enough for an agent
Most genomics tooling is split across Python, R, and command-line binaries. Dependency resolution is the main reason a notebook that worked in March fails in September. Pin everything with conda-lock or pixi, and record tool versions in the output files themselves. Work on automated reconstruction of cross-language bioinformatics environments is aimed at exactly this problem, so that an AI agent can run a pipeline without a human resolving dependencies by hand2. If you plan to hand your data to an agent, the environment specification is part of the dataset.
Failure modes to check before you trust a result
- Chromosome naming.
chr1versus1. A join between a UCSC-derived BED and an Ensembl GTF will silently return zero rows. - Reference build. GRCh37 and GRCh38 coordinates differ by megabases in places. Check the VCF header
##contiglines for the assembly, and useCrossMapif you must lift over. Liftover fails for a small percentage of positions, and those failures cluster in segmental duplications. - Alt contigs. GRCh38 includes ALT scaffolds. Filter to primary assembly for most analyses or you will double-count.
- Sex chromosomes and mitochondria. Ploidy assumptions in variant callers differ, and chrM is often called with a separate model.
- Coverage. A het call at 8x read depth is a coin flip. We filter on
gt_depths >= 15for anything we intend to look at closely.
Where interpretation stops
Annotating a variant with VEP and looking it up in ClinVar is a data operation you can do yourself. Deciding what a variant means for your health is not. Pathogenicity classification follows formal ACMG criteria and depends on family history, penetrance estimates, and assay context that a lookup table does not capture. If a variant in your file is flagged pathogenic or likely pathogenic, the next step is a clinician or a certified genetic counselor, and confirmation on a clinical-grade assay. Research sequencing is not a diagnostic test.
Questions people also ask
Do I need R as well as Python? For most personal-scale work, no. The main gap is Bioconductor packages with no Python equivalent, such as some specialized annotation resources. Call them through rpy2 when you need them rather than moving your whole workflow.
pandas or polars? Polars for anything above a few million rows, which in genomics is most things. Its memory profile on a variant table is typically a third of pandas. Keep pandas for small joined tables where library compatibility matters.
How much disk and RAM do I need? Budget 200 GB of disk for a 30x genome plus RNA-seq and intermediates, and 32 GB of RAM. You can do most Python-layer analysis on 16 GB if you stream rather than load.
Can an AI agent run this for me? It can run the queries and write the joins competently. It is much less reliable about build mismatches and coordinate conventions, because those errors produce plausible-looking output. Check the row counts yourself.
Footnotes
-
Carmen Oroperv, Amanda Frydendahl, Tenna Vesterman Henriksen, et al. An alignment-free strategy for circulating tumor DNA detection and tumor fraction estimation from whole-genome sequencing data. PLOS Computational Biology, 2026. https://doi.org/10.1371/journal.pcbi.1013356 ↩
-
Nianping Liu, Xuanzhi Chen, Miao Cui, et al. Bio-Babel: autonomous cross-language reconstruction of agent-ready computational biology ecosystems. 2026. https://doi.org/10.64898/2026.08.30.743167 ↩
Pothos builds longitudinal molecular profiles of individuals: whole-genome sequencing, single-cell RNA sequencing, and plasma proteomics, delivered as a detailed report, every data file, and stored cells. Build your profile.