Skip to content

Hard-Filtering Your Own Germline Variants with GATK

Oak
An iridescent feathered creature on a dark studio plinth, seen through a taut wire sieve where glowing beads are caught or fall through.

By the end of this guide you will have taken a raw single-sample VCF from GATK HaplotypeCaller and produced a filtered VCF in which every record carries either PASS or a named filter tag, with SNPs and insertions/deletions (indels) filtered on separate criteria. You will also have measured what your thresholds cost you, using a truth set rather than intuition. You need a GATK 4 installation of version 4.4 or later, bcftools, and the reference FASTA you aligned against with its .fai and .dict. You also need roughly 30x whole-genome sequencing data for one person and about 20 GB of scratch space. If your sample is one of the Genome in a Bottle reference individuals you can skip the benchmarking caveats, but for your own genome you will need a proxy truth set, which step 6 covers.

1. Understand why hard filters exist and when they are the right choice

HaplotypeCaller is deliberately permissive. It emits candidate variants with low confidence so that downstream filtering, rather than calling, decides what survives. The GATK Best Practices pipeline splits this into two stages: call everything plausible, then model the difference between true and false calls using annotations in the INFO field.1 Hard-filtering is the simpler of the two filtering options. You pick numeric thresholds on individual annotations and discard records outside them.

The alternative is Variant Quality Score Recalibration (VQSR). It fits a Gaussian mixture model to the annotation distributions of variants that overlap known true sites, then scores every call by how well it fits the “true” cluster. VQSR is more sensitive and more sensible in principle, and the GATK team recommends it for whole genomes. It also needs enough variants to train on. For a single 30x human genome you have roughly four to five million variants, which is generally sufficient for SNPs but marginal for indels.

We use hard filters as the default for single-sample work, for three reasons. The thresholds are legible, so you can explain any individual exclusion. The behavior is deterministic across reruns. And the newer GATK deep-learning filters and VQSR both introduce a dependence on resource bundles whose contents change between releases. The cost is sensitivity, mostly at low allele balance and in repetitive regions. Comparative work on variant calling pipelines has repeatedly shown that the filtering step, not the caller, drives most of the disagreement between pipelines.2

2. Split SNPs and indels before filtering anything

The annotations behave differently for the two variant classes, so a single threshold set will be wrong for one of them. Indels have systematically worse strand-bias and mapping-quality behavior because they arise near repeats. Separate the two classes first.

REF=GRCh38.fa
IN=sample.raw.vcf.gz

gatk SelectVariants \
  -R $REF \
  -V $IN \
  --select-type-to-include SNP \
  -O sample.snps.vcf.gz

gatk SelectVariants \
  -R $REF \
  -V $IN \
  --select-type-to-include INDEL \
  --select-type-to-include MIXED \
  -O sample.indels.vcf.gz

Include MIXED in the indel set. Multi-allelic sites that contain both a SNP and an indel allele are typed as MIXED and will silently vanish if you only select the two main types. Check the counts add up:

for f in sample.raw sample.snps sample.indels; do
  echo -n "$f "
  bcftools view -H $f.vcf.gz | wc -l
done

3. Apply the SNP filters

These are the thresholds GATK publishes for germline short variants, expressed as VariantFiltration expressions. Each --filter-expression names a condition for failure, and each paired --filter-name becomes the tag written to the FILTER column.

gatk VariantFiltration \
  -R $REF \
  -V sample.snps.vcf.gz \
  --filter-expression "QD < 2.0"                 --filter-name "QD2" \
  --filter-expression "QUAL < 30.0"              --filter-name "QUAL30" \
  --filter-expression "SOR > 3.0"                --filter-name "SOR3" \
  --filter-expression "FS > 60.0"                --filter-name "FS60" \
  --filter-expression "MQ < 40.0"                --filter-name "MQ40" \
  --filter-expression "MQRankSum < -12.5"        --filter-name "MQRankSum-12.5" \
  --filter-expression "ReadPosRankSum < -8.0"    --filter-name "ReadPosRankSum-8" \
  -O sample.snps.filtered.vcf.gz

What each annotation measures matters more than the number itself. QD is QUAL divided by the depth of reads supporting the alternate allele, which normalizes confidence by coverage and prevents high-depth sites from passing on depth alone. FS is the Phred-scaled Fisher exact test p-value for strand bias, and SOR is the symmetric odds ratio of the same 2x2 table. SOR handles high-coverage sites better than FS, because the Fisher p-value grows with depth even when the bias ratio is unchanged. MQ is the root-mean-square mapping quality of reads at the site, and MQRankSum compares mapping quality between reference-supporting and alternate-supporting reads. ReadPosRankSum tests whether the alternate allele appears preferentially near read ends, which is the signature of alignment artifacts around indels.

Note that these filters mark records rather than remove them. That is the behavior you want during development, since you can then count what each tag would have cost.

bcftools query -f '%FILTER\n' sample.snps.filtered.vcf.gz \
  | tr ';' '\n' | sort | uniq -c | sort -rn

4. Apply the indel filters

The indel set uses fewer annotations, because MQRankSum and SOR are less informative there.

gatk VariantFiltration \
  -R $REF \
  -V sample.indels.vcf.gz \
  --filter-expression "QD < 2.0"                 --filter-name "QD2" \
  --filter-expression "QUAL < 30.0"              --filter-name "QUAL30" \
  --filter-expression "FS > 200.0"               --filter-name "FS200" \
  --filter-expression "ReadPosRankSum < -20.0"   --filter-name "ReadPosRankSum-20" \
  -O sample.indels.filtered.vcf.gz

The FS threshold is 200 rather than 60, and ReadPosRankSum is -20 rather than -8. Both reflect the fact that true indels genuinely do show strand and position skew, so the SNP thresholds would discard real calls. Then merge the two filtered sets back together:

gatk MergeVcfs \
  -I sample.snps.filtered.vcf.gz \
  -I sample.indels.filtered.vcf.gz \
  -O sample.filtered.vcf.gz

MergeVcfs requires both inputs to be sorted against the same reference dictionary. If it complains about contig ordering, your two VCFs came from different reference builds or one lost its header during an intermediate step.

5. Add a genotype-level and depth filter

Site-level filters say whether a position is a real variant. They say nothing about whether this individual’s genotype call at that position is trustworthy. For a single genome you also want a genotype quality floor and a depth window.

gatk VariantFiltration \
  -R $REF \
  -V sample.filtered.vcf.gz \
  --genotype-filter-expression "GQ < 20"  --genotype-filter-name "lowGQ" \
  --genotype-filter-expression "DP < 10"  --genotype-filter-name "lowDP" \
  --genotype-filter-expression "DP > 300" --genotype-filter-name "highDP" \
  -O sample.filtered.gt.vcf.gz

GQ 20 corresponds to a 1-in-100 chance that a different genotype is correct, which is a reasonable floor for interpreting your own results. The upper depth bound catches collapsed repeats and segmental duplications, where reads from several genomic copies pile onto one locus and produce confident-looking heterozygous calls that are artifacts of the reference assembly. For 30x whole-genome data, 300 is roughly ten times the mean and will flag a few tens of thousands of sites. Compute your actual mean depth and set the cap at 3 to 5 standard deviations above it rather than copying a number.

To convert flagged genotypes to no-calls and then drop the sites that lose all information:

gatk SelectVariants \
  -V sample.filtered.gt.vcf.gz \
  --set-filtered-gt-to-nocall \
  -O sample.final.vcf.gz

6. Measure what your thresholds cost

This is the step most guides omit, and it is the one that decides whether your filters are any good. Published default thresholds were derived from deep whole-genome data from a particular era of Illumina chemistry. They do not transfer cleanly to targeted panels, to RNA-seq derived calls, or to unusual coverage profiles. Work on targeted gene panels found that tuning QD, FS and MQ against simulated data materially improved precision and recall over the defaults.3 Broader analyses of GATK false positives reach the same conclusion: fixed thresholds leave a recoverable error population behind.4

The clean way to measure this is hap.py or rtg vcfeval against a truth set. If you have sequenced a Genome in a Bottle sample, use the corresponding NIST high-confidence VCF and its BED of confident regions:

hap.py \
  HG002_GRCh38_1_22_v4.2.1_benchmark.vcf.gz \
  sample.final.vcf.gz \
  -f HG002_GRCh38_1_22_v4.2.1_benchmark_noinconsistent.bed \
  -r $REF \
  -o happy.out \
  --engine vcfeval \
  --threads 8

Read the happy.out.summary.csv for METRIC.Recall, METRIC.Precision and METRIC.F1_Score, split by SNP and INDEL. On 30x whole-genome data with these filters you should expect SNP F1 above 0.99 and indel F1 in the 0.95 to 0.99 range. If indel recall is below 0.90, your indel filters are too aggressive or your alignment is the problem.

For your own genome you have no truth set, so use proxies. Three work reasonably well together. First, the heterozygous-to-homozygous ratio for non-reference calls, which should land near 1.5 to 2.0 for a typical human genome. Second, the transition/transversion ratio, which sits near 2.0 to 2.1 genome-wide and near 3.0 in exons. A ratio that drops toward 1.0 after you loosen a filter means you admitted random-looking noise. Third, the fraction of your calls present in gnomAD or dbSNP, which should exceed 95% for SNPs in a healthy genome.

bcftools stats -f PASS sample.final.vcf.gz > stats.txt
grep -E "^TSTV|^SN" stats.txt

Sweep one threshold at a time and watch these three numbers move together. Approaches that score variants on combinations of annotations rather than on independent per-annotation cutoffs consistently outperform fixed thresholds, which is the reason VQSR and machine-learning filters exist at all.5 If your metrics look poor after tuning and you have enough variants, switching to VQSR or the GATK convolutional neural network filter is the right response.

7. Adjust for RNA-seq calls

If you are calling variants from your RNA sequencing rather than DNA, the DNA thresholds are wrong and the pipeline differs upstream. You need SplitNCigarReads after alignment so that reads spanning splice junctions are handled correctly, and you need to relax mapping quality handling because STAR assigns MAPQ 255 to uniquely mapped reads. The GATK RNA-seq short variant workflow uses a cluster-based filter instead of the DNA annotation set.6

gatk VariantFiltration \
  -R $REF \
  -V rna.raw.vcf.gz \
  --window 35 --cluster 3 \
  --filter-expression "QD < 2.0" --filter-name "QD2" \
  --filter-expression "FS > 30.0" --filter-name "FS30" \
  -O rna.filtered.vcf.gz

--window 35 --cluster 3 flags any variant in a group of three or more within a 35-base window, which catches the misalignment clusters typical of junction-spanning reads. Expect much lower recall than DNA calling, and expect allele-specific expression to skew allele balance at real heterozygous sites. RNA-derived genotypes are useful for expression questions. They are a poor substitute for DNA genotypes.

Common problems

VariantFiltration silently passing everything is the most frequent failure. It happens when an annotation is absent from the INFO field, because a JEXL expression referencing a missing key evaluates to false and the record is not flagged. Check that your annotations exist before you trust the output:

bcftools view -h sample.raw.vcf.gz | grep -E "ID=(QD|FS|SOR|MQ|MQRankSum|ReadPosRankSum)"

MQRankSum and ReadPosRankSum are missing at homozygous variant sites by design, since there are no reference reads to rank against. That is expected and those sites will not be flagged by the rank-sum filters. Passing --missing-values-evaluate-as-failing will flag them, which is almost never what you want.

Contig mismatches between your VCF and the reference dictionary produce errors that look like filtering failures. GRCh38 with and without chr prefixes, and with or without ALT contigs, are different references as far as GATK is concerned. Rebuild the .dict with gatk CreateSequenceDictionary against the exact FASTA you aligned to.

A very low PASS rate usually points upstream rather than at your thresholds. Check for missing base quality score recalibration, duplicate marking that never ran, or contamination. Quality-based filtering only works when the quality scores mean what they claim to, which is the entire point of recalibration in the standard workflow.7 Documented reference implementations of the full pipeline are worth comparing against if you suspect a missing step.8

Finally, remember what a filtered VCF is and is not. The file is a list of positions where your sequence differs from the reference, with a confidence assessment attached. Turning any single variant into a statement about your health requires a clinical geneticist and, for anything consequential, orthogonal confirmation by a different assay in an accredited laboratory. Research-grade filtering thresholds are tuned for genome-wide accuracy. They are not tuned for the reliability of one specific call.9

Footnotes

  1. Geraldine A. Van der Auwera, Mauricio O. Carneiro, Christopher Hartl, et al. From FastQ Data to High‐Confidence Variant Calls: The Genome Analysis Toolkit Best Practices Pipeline. Current Protocols in Bioinformatics, 2013. https://doi.org/10.1002/0471250953.bi1110s43 ↩

  2. Mehdi Pirooznia, Melissa Kramer, Jennifer Parla, et al. Validation and assessment of variant calling pipelines for next-generation sequencing. Human Genomics, 2014. https://doi.org/10.1186/1479-7364-8-14 ↩

  3. Simona De Summa, Giovanni Malerba, Rosamaria Pinto, et al. GATK hard filtering: tunable parameters to improve variant calling for next generation sequencing targeted gene panel data. BMC Bioinformatics, 2017. https://doi.org/10.1186/s12859-017-1537-8 ↩

  4. Yuxiang Tan, Yu Zhang, Hengwen Yang, et al. FPfilter: A false-positive-specific filter for whole-genome sequencing variant calling from GATK. 2020. https://doi.org/10.1101/2020.03.23.003525 ↩

  5. Vladimir Kovacevic, Predrag Obradovic. Smart variant filtering. International Journal of Data Mining and Bioinformatics, 2021. https://doi.org/10.1504/ijdmb.2021.126836 ↩

  6. Jean-Simon Brouard, Nathalie Bissonnette. Variant Calling from RNA-seq Data Using the GATK Joint Genotyping Workflow. Methods in Molecular Biology, 2012. https://doi.org/10.1007/978-1-0716-2293-3_13 ↩

  7. Peter N. Robinson, Rosario M. Piro, Marten Jäger. Variant Calling and Quality-Based Filtering. Computational Exome and Genome Analysis, 2017. https://doi.org/10.1201/9781315154770-12 ↩

  8. Miguel Pinheiro, Jorge Miguel Silva, José Luis Oliveira. Optimizing Variant Calling for Human Genome Analysis: A Comprehensive Pipeline Approach. Lecture Notes in Computer Science, 2023. https://doi.org/10.1007/978-3-031-34960-7_6 ↩

  9. V. P. Snijesh, K Shivakumar, K. M. Punya, et al. Best Practices for Variant Calling Using the Genome Analysis Toolkit. Next-Generation Sequencing, 2025. https://doi.org/10.1201/9781003354062-4 ↩

Pothos builds longitudinal molecular profiles of individuals: whole-genome sequencing, single-cell RNA sequencing, and plasma proteomics, delivered as a detailed report, every data file, and stored cells. Build your profile.