GATK VariantFiltration: Worked Examples for Hard-Filtering Your Own WGS and RNA-seq Variants
By the end of this guide you will have taken a raw joint-genotyped VCF and produced a filtered VCF in which every site carries either PASS or a named reason for exclusion, plus a second copy with low-confidence genotypes set to no-call. You need GATK 4.5 or newer and the same reference FASTA you aligned against, with its .fai and .dict. You also need bcftools 1.19 or newer and a VCF produced by HaplotypeCaller followed by GenotypeGVCFs. Roughly 8 GB of RAM is enough for a single-genome VCF, and the filtering itself is I/O bound rather than compute bound. Everything below assumes hg38 and a single human genome, which is the common case for a personal profile.
One framing note before the commands. The VariantFiltration tool writes strings into the seventh column of the VCF and, optionally, into the per-sample FT field. The tool does not delete anything. Removing records is a separate step with SelectVariants or bcftools view. This is a good design, because it means your filtering decisions are reversible and auditable, and you can change thresholds later without re-calling.
1. Inspect which annotations your VCF contains
Hard filtering works by thresholding the annotations that HaplotypeCaller wrote into the INFO column. If an annotation is absent, the filter on it silently does nothing. So start by looking at what you have.
bcftools view -h raw.vcf.gz | grep '^##INFO' | cut -d, -f1
bcftools stats raw.vcf.gz > raw.stats.txt
head -40 raw.stats.txt
You want to see each of the following.
QDFSSORMQMQRankSumReadPosRankSumDP
For a single-genome VCF from a 30x whole-genome run, expect somewhere around 4 to 5 million variant sites before filtering, of which roughly 80 percent are SNVs. If MQRankSum and ReadPosRankSum are missing from the header entirely, your caller was run with a reduced annotation set and you should re-run HaplotypeCaller with -G StandardAnnotation -G AS_StandardAnnotation rather than trying to filter without them.
It also helps to look at the distributions before choosing cutoffs. The published GATK thresholds are generic starting points rather than tuned values, and work on targeted panels has shown that tuning them against a known-truth sample changes sensitivity and specificity substantially.1
bcftools query -f '%QD\t%FS\t%SOR\t%MQ\n' raw.vcf.gz \
| awk '$1!="."' > annot.tsv
# then plot, or:
awk '{s+=$1; n++} END {print "mean QD:", s/n}' annot.tsv
For a healthy 30x genome the QD distribution is bimodal with a large mode near 12 to 20 and a small left tail below 2. MQ should pile up tightly at 60. If your MQ mode sits at 55 or below, you have an alignment problem to fix before you filter.
2. Split SNPs and indels
The two variant classes have different error profiles, so they need different thresholds. Filter them separately and merge afterwards.
gatk SelectVariants \
-R GRCh38.primary_assembly.fa \
-V raw.vcf.gz \
--select-type-to-include SNP \
-O raw.snps.vcf.gz
gatk SelectVariants \
-R GRCh38.primary_assembly.fa \
-V raw.vcf.gz \
--select-type-to-include INDEL \
--select-type-to-include MIXED \
-O raw.indels.vcf.gz
Include MIXED in the indel set. Mixed sites are multiallelic records that carry both an SNV and an indel allele, and if you leave them out of both selections they disappear from your merged output. Check the arithmetic afterwards with bcftools index -n on all three files.
3. Filter the SNPs
This is the canonical VariantFiltration invocation. Each -filter expression must be immediately paired with a --filter-name, and the tool matches them by order of appearance, so keep them adjacent for your own sanity.
gatk VariantFiltration \
-R GRCh38.primary_assembly.fa \
-V raw.snps.vcf.gz \
-filter "QD < 2.0" --filter-name "QD2" \
-filter "QUAL < 30.0" --filter-name "QUAL30" \
-filter "SOR > 3.0" --filter-name "SOR3" \
-filter "FS > 60.0" --filter-name "FS60" \
-filter "MQ < 40.0" --filter-name "MQ40" \
-filter "MQRankSum < -12.5" --filter-name "MQRankSum-12.5" \
-filter "ReadPosRankSum < -8.0" --filter-name "ReadPosRankSum-8" \
-O filtered.snps.vcf.gz
The expressions are JEXL, a small Java expression language. They must be quoted, because the shell would otherwise interpret < as a redirect. Filter names must not contain whitespace or semicolons, since the FILTER column is semicolon-delimited.
Two of these thresholds deserve comment. QD is variant confidence divided by allele depth. It is the single most useful site-level annotation in practice, because it normalizes quality by coverage and therefore does not drift with depth. FS is the Phred-scaled Fisher exact test for strand bias. The cutoff of 60 is deliberately permissive so that real variants in strand-biased regions survive. These values come from the GATK best-practice documentation and are widely reproduced in pipeline descriptions.2
4. Filter the indels
Indels get looser strand-bias and read-position cutoffs, because the alignment ambiguity around them genuinely produces skewed distributions.
gatk VariantFiltration \
-R GRCh38.primary_assembly.fa \
-V raw.indels.vcf.gz \
-filter "QD < 2.0" --filter-name "QD2" \
-filter "QUAL < 30.0" --filter-name "QUAL30" \
-filter "FS > 200.0" --filter-name "FS200" \
-filter "ReadPosRankSum < -20.0" --filter-name "ReadPosRankSum-20" \
-O filtered.indels.vcf.gz
Note the absence of MQ and MQRankSum here. Mapping quality around indels is not informative in the same way, and applying the SNP cutoffs to indels removes many true calls in repetitive sequence.
Then merge, using MergeVcfs rather than bcftools concat, because MergeVcfs reconciles the two header sets and re-sorts against the sequence dictionary.
gatk MergeVcfs \
-I filtered.snps.vcf.gz \
-I filtered.indels.vcf.gz \
-O filtered.vcf.gz
5. Add genotype-level filters
Site-level filters ask whether a position is a real variant. Genotype-level filters ask whether this sample’s genotype call at that position is trustworthy. They are separate questions and VariantFiltration handles both.
gatk VariantFiltration \
-R GRCh38.primary_assembly.fa \
-V filtered.vcf.gz \
--genotype-filter-expression "GQ < 20" --genotype-filter-name "lowGQ" \
--genotype-filter-expression "DP < 10" --genotype-filter-name "lowDP" \
-O filtered.gt.vcf.gz
This writes lowGQ or lowDP into the per-sample FT field and leaves the genotype itself untouched. Most downstream tools ignore FT, which is a common source of quiet error. If you want the filter to have an effect, convert the failing genotypes to no-calls explicitly.
gatk SelectVariants \
-R GRCh38.primary_assembly.fa \
-V filtered.gt.vcf.gz \
--set-filtered-gt-to-nocall \
-O filtered.nocall.vcf.gz
We keep both files. The FT-annotated version preserves the original call for later review, and the no-call version is what we feed to annotation and interpretation tools. A GQ floor of 20 corresponds to roughly a 1 percent chance that the genotype is wrong, and a DP floor of 10 is appropriate for a 30x genome. For a 100x exome you can raise DP to 20 without losing much.
You can also filter on genotype state. The JEXL attributes isHet, isHomVar and isHomRef are available. That means --genotype-filter-expression "isHet == 1" flags all heterozygous calls. This matters when you are looking for homozygous recessive candidates and want to mark everything else.
6. Handle RNA-seq calls differently
If you are calling variants from RNA sequencing reads, the dominant artifact is clustered false positives near splice junctions and in regions of low expression. GATK’s RNA-seq workflow uses a cluster filter for this, and VariantFiltration implements it directly.
gatk VariantFiltration \
-R GRCh38.primary_assembly.fa \
-V rna.raw.vcf.gz \
--cluster-size 3 \
--cluster-window-size 35 \
-filter "QD < 2.0" --filter-name "QD2" \
-filter "FS > 30.0" --filter-name "FS30" \
-O rna.filtered.vcf.gz
The two cluster options mark any site that falls in a window of 35 bases containing three or more variants, and the resulting filter string is SnpCluster. The FS cutoff of 30 is tighter than the DNA setting, because RNA-seq alignments are more strand-biased. Do not treat RNA-derived genotypes as a substitute for DNA genotypes. Allele-specific expression makes a true heterozygote look homozygous whenever one allele is silent.
7. Verify that the filtering did something sensible
Never ship a filtered VCF without counting what you removed. Start with the filter tally.
bcftools query -f '%FILTER\n' filtered.vcf.gz | sort | uniq -c | sort -rn
For a 30x genome with the thresholds above, we typically see 92 to 97 percent of sites passing. If your pass rate is below 85 percent, something upstream is wrong, most often duplicate marking or a reference mismatch. If it is above 99.5 percent, check that your annotations are present, because filters on missing attributes evaluate to false and let the site through.
Next, check the transition-to-transversion ratio, which is the most sensitive single-number quality proxy for an SNV set.
bcftools stats filtered.vcf.gz | grep '^TSTV'
Whole-genome human SNVs should land near 2.0 to 2.1, and exonic subsets near 3.0. Filtering should raise the ratio slightly, because sequencing errors are transversion-enriched. A ratio that falls after filtering means a threshold is removing real variants.
Finally, look at the count of rare coding candidates rather than the total. Work on rare disease cohorts has shown that a well-ordered chain of quality, population-frequency and inheritance filters reduces a whole genome to a handful of candidate variants. The same work shows that the expected yield per genome is predictable enough to serve as a sanity check on your own pipeline.3 If your equivalent count is an order of magnitude off, the problem is usually in the filtering chain and not in the biology.
8. Decide whether hard filtering is the right choice at all
Hard filtering is the approach we recommend for a single genome, because the machine-learning alternatives need either a large cohort or careful calibration. VariantRecalibrator models the annotation distribution across many samples, and on one genome the model is poorly constrained. GATK’s newer VariantEvaluation tools score sites with a trained model instead, and their performance depends heavily on hyperparameter choices that most users never touch.4 There is a broader literature on machine-learned variant filtering that reports gains over fixed thresholds when training data are adequate.5
So the practical rule is this. One genome, standard 30x coverage, hg38: use the hard filters above. Ten or more genomes sequenced on the same instrument and chemistry: consider a trained filter, and validate it against a truth sample such as GIAB HG002 before trusting it. Targeted panel data: tune the thresholds yourself, because the generic values were derived from whole-genome and whole-exome distributions and transfer poorly.1
Whatever you choose, keep the filter thresholds in version control alongside the pipeline. Reproducibility here is a matter of recording a dozen numbers, and pipelines that manage these parameters explicitly are much easier to re-run months later.6
Common problems
Missing annotations pass every filter. This is the failure mode that bites hardest. MQRankSum and ReadPosRankSum are only computed when there are reads supporting both alleles, so homozygous-variant sites lack them entirely. A filter of MQRankSum < -12.5 at such a site evaluates against a missing attribute, and GATK logs a warning about an undefined variable and lets the site pass. If you want the opposite behavior, add --missing-values-evaluate-as-failing, but understand that this will fail a large fraction of your homozygous calls. We leave it off and accept that rank-sum filters only apply to heterozygous sites.
JEXL quoting mistakes are the second most common problem. Use double quotes around the whole expression and avoid $ characters. To guard an expression explicitly, write vc.hasAttribute('QD') && QD < 2.0, which only evaluates the comparison when the attribute exists.
Filter names collide across runs. If you run VariantFiltration twice with the same filter name, the second run appends a duplicate header line and the FILTER column can end up with the name repeated. Use distinct names per pass, or strip the old ones first with bcftools annotate -x FILTER.
Multiallelic sites are filtered as a unit. VariantFiltration operates per record, so a site with one good allele and one artifactual allele is either kept or removed whole. If this matters for your analysis, split with bcftools norm -m -any before filtering, then be aware that QD and AD no longer correspond cleanly to the split records.
Sequence dictionary mismatches stop the run. The error reads like a contig ordering complaint. Regenerate the dictionary with gatk CreateSequenceDictionary -R your.fa from the exact FASTA used for alignment, and do not mix UCSC-style chr1 naming with Ensembl-style 1.
Filtered records are still in the file. VariantFiltration labels rather than removes, which surprises people who then feed the output to an annotator and wonder why the artifact count did not drop. Use gatk SelectVariants --exclude-filtered true or bcftools view -f PASS when you want a hard-pruned file.
One closing point about interpretation. A PASS label is a statement about sequencing and alignment evidence, and nothing more. It carries no information about whether a variant affects your health. Clinical interpretation requires curated evidence and appropriate annotation pipelines. It also requires a qualified clinician or genetic counselor to place a result in context.7 Filtering gets you to a clean, well-documented variant set. Deciding what any specific variant means for a person is a different task, and one you should not do alone.
Footnotes
-
Simona De Summa, Giovanni Malerba, Rosamaria Pinto, et al. GATK hard filtering: tunable parameters to improve variant calling for next generation sequencing targeted gene panel data. BMC Bioinformatics, 2017. https://doi.org/10.1186/s12859-017-1537-8 ↩ ↩2
-
V. P. Snijesh, K Shivakumar, K. M. Punya, et al. Best Practices for Variant Calling Using the Genome Analysis Toolkit. Next-Generation Sequencing, 2025. https://doi.org/10.1201/9781003354062-4 ↩
-
Brent S. Pedersen, Joe M. Brown, Harriet Dashnow, et al. Effective variant filtering and expected candidate variant yield in studies of rare human disease. npj Genomic Medicine, 2021. https://doi.org/10.1038/s41525-021-00227-3 ↩
-
Wanying Dou, Samuel K. Lee, Kiran V. Garimella. A framework for hyperparameter optimization of genomic cloud pipelines: improving the performance of GATK VETS variant filtration. 2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), 2024. https://doi.org/10.1109/bibm62325.2024.10822323 ↩
-
Vladimir Kovacevic, Predrag Obradovic. Smart variant filtering. International Journal of Data Mining and Bioinformatics, 2021. https://doi.org/10.1504/ijdmb.2021.126836 ↩
-
TUDP, F. Musacchia, A. Ciolfi, et al. VarGenius executes cohort-level DNA-seq variant calling and annotation and allows to manage the resulting data through a PostgreSQL database. BMC Bioinformatics, 2018. https://doi.org/10.1186/s12859-018-2532-4 ↩
-
Erin L. Crowgey, Deborah L. Stabley, Chuming Chen, et al. An Integrated Approach for Analyzing Clinical Genomic Variant Data from Next-Generation Sequencing. Journal of Biomolecular Techniques : JBT, 2015. https://doi.org/10.7171/jbt.15-2601-002 ↩
Pothos builds longitudinal molecular profiles of individuals: whole-genome sequencing, single-cell RNA sequencing, and plasma proteomics, delivered as a detailed report, every data file, and stored cells. Build your profile.