Filtering a VCF by a BED File
By the end of this guide you will have a bgzip-compressed, indexed VCF containing only the variants that fall inside a set of regions you define, plus a count you can defend. You need three things: a VCF or BCF from your own sequencing (this guide assumes a single-sample whole-genome VCF), a BED file of regions, and bcftools 1.15 or newer. Older bcftools works but lacks the --regions-overlap and --targets-overlap flags, which is where most of the interesting behavior lives. A reference FASTA matching your VCF is useful for normalization, and bedtools is useful for preparing the region file. Everything below is command-line work on a single machine. A 30x human WGS VCF with roughly 4-5 million variants filters in seconds to a couple of minutes depending on the approach.
1. Check that your VCF and BED agree on coordinates and names
Before filtering, confirm the two files describe the same genome. The most common failure is a contig naming mismatch, and it fails quietly: bcftools returns zero variants and exits successfully. Check both sides.
# contig names present in the VCF header
bcftools view -h sample.vcf.gz | grep '^##contig' | head -5
# contig names used in records
bcftools index --stats sample.vcf.gz | head -5
# contig names in the BED
cut -f1 regions.bed | sort -u | head -5
If the VCF uses chr1 and the BED uses 1, fix the BED rather than the VCF. Rewriting a BED is cheap and reversible. If you need to go the other way, bcftools annotate --rename-chrs chr_map.txt takes a two-column file of old and new names and rewrites both header and records.
awk 'BEGIN{OFS="\t"} {if ($1 !~ /^chr/) $1="chr"$1; print}' regions.bed > regions.chr.bed
Also confirm the assembly itself. A GRCh37 BED against a GRCh38 VCF produces results that look plausible and are wrong, because positions shift by hundreds to millions of bases depending on the chromosome. If the BED came from a capture kit vendor or a published benchmark, read the filename and the accompanying README for the build. If you cannot determine the build, do not guess. Lift over explicitly with CrossMap or liftOver and record which chain file you used.
Finally, make sure the VCF is bgzipped and indexed. Plain gzip will not work, because random access needs the block structure.
bcftools view sample.vcf -Oz -o sample.vcf.gz # if not already compressed
bcftools index -t sample.vcf.gz # .tbi; use -c for .csi on long contigs
2. Normalize the BED file
A BED file that bcftools accepts is not necessarily a BED file that gives you the count you expect. Three properties matter: sorted order, no overlapping intervals, and no header lines.
BED is a three-column minimum format with 0-based, half-open coordinates. The interval chr1 1000 1005 covers the five 1-based positions 1001 through 1005. VCF POS is 1-based and inclusive. bcftools handles this conversion for you when the file is named with a .bed or .bed.gz suffix. If you name it regions.txt, bcftools treats it as a 1-based tab-delimited region list and every interval shifts by one base. Keep the suffix.
Sort and merge before using the file:
grep -v '^\(track\|browser\|#\)' regions.raw.bed \
| sort -k1,1 -k2,2n \
| bedtools merge -i - > regions.bed
wc -l regions.bed
awk '{s+=$3-$2} END {print s/1e6 " Mb covered"}' regions.bed
The covered-bases number tells you what to expect downstream. A whole-exome capture BED covers roughly 35-65 Mb depending on the kit, about 1-2% of the genome. The Genome in a Bottle high-confidence regions for a GRCh38 sample cover roughly 2.5 Gb. If your merged BED covers 45 Mb and your filtered VCF keeps 40% of variants, something is wrong with the coordinates.
Merging matters for a specific reason covered in the next step: with index-based region lookup, overlapping intervals can cause the same variant to be emitted twice.
If you are building a gene panel BED rather than using a vendor file, derive it from a GENCODE or RefSeq annotation and be explicit about padding. Exonic coordinates alone will miss splice-site variants a few bases into the intron.
# exons of a gene set, padded 10 bp each side
zcat gencode.v44.annotation.gtf.gz \
| awk 'BEGIN{OFS="\t"} $3=="exon"' \
| grep -Ff gene_names.txt \
| awk 'BEGIN{OFS="\t"} {print $1, $4-1, $5}' \
| sort -k1,1 -k2,2n | bedtools merge -i - \
| bedtools slop -b 10 -g GRCh38.genome > panel.bed
3. Choose between -R and -T
bcftools offers two region arguments that look interchangeable and are not. The difference is the single most important thing to understand about this task.
-R, --regions-file uses the VCF index to seek directly to each interval. It is fast when the BED has few intervals scattered across the genome, because it reads only the relevant blocks. It requires an index. It can emit duplicate records if intervals overlap, and it processes intervals in the order given, so an unsorted BED gives unsorted output.
-T, --targets-file streams the entire VCF and tests each record against the region set. It needs no index and works on a pipe. It is faster when the BED is dense or has millions of intervals, because index seeks stop paying off once you are reading most of the file anyway.
Our rule: use -T when the BED covers more than roughly 10% of the genome or contains more than a few hundred thousand intervals. Use -R for panels, single genes, and sparse interval sets. When in doubt, time both on chromosome 1.
The second difference is the default overlap semantics, and it surprises people. For -R, the default --regions-overlap 1 keeps a record if the record overlaps the region, where the record’s span accounts for the length of the REF allele. For -T, the default --targets-overlap 0 keeps a record only if POS itself falls inside the region. A 12 bp deletion whose POS sits one base before a region boundary is kept by -R and dropped by -T. This is the source of the frequently reported discrepancy between commands that appear equivalent.
Set the flag explicitly so the behavior is recorded in your command rather than inherited from a default:
# sparse regions, index-based, variant-aware overlap
bcftools view \
-R panel.bed \
--regions-overlap 2 \
-Oz -o sample.panel.vcf.gz \
sample.vcf.gz
bcftools index -t sample.panel.vcf.gz
# dense regions, streaming
bcftools view \
-T high_conf.bed \
--targets-overlap 2 \
-Oz -o sample.hc.vcf.gz \
sample.vcf.gz
Value 2 means variant-aware: the record is kept if the variant it describes overlaps the region, using the END INFO field when present. Value 1 uses the record span. Value 0 uses POS only. For most analyses we want 2, because it treats an indel as the interval it really occupies.
4. Normalize variants before filtering and do not wait until after
Region membership depends on how the variant is represented, so representation should be settled first. Left-aligning indels against the reference can move POS by several bases, which is enough to cross a boundary. Multiallelic records complicate matters further, because one record can contain a SNV inside the region and a long deletion extending outside it.
Split and left-align first:
bcftools norm -m -any -f GRCh38.fa -c w \
-Oz -o sample.norm.vcf.gz sample.vcf.gz
bcftools index -t sample.norm.vcf.gz
-m -any splits multiallelics into one record per ALT allele. -f supplies the reference for left-alignment. -c w warns rather than fails when a REF allele disagrees with the reference FASTA, which is the diagnostic you want if the VCF was built against a different build. Then filter the normalized file. If a downstream tool needs multiallelic records, rejoin with bcftools norm -m +any after filtering.
This ordering also makes the variant counts reproducible. Tools that prioritize variants for interpretation generally assume a normalized, per-allele representation, and the published filtration pipelines built around VCF input are designed on that assumption.12
5. Handle gVCF reference blocks separately
If your file is a gVCF, the obvious command gives a misleading answer. A gVCF contains non-variant blocks with <NON_REF> or <*> as the ALT and an END INFO tag marking where the block stops. A single block can span tens of kilobases. Filtering by POS alone drops blocks whose start precedes the region, losing the evidence that those positions were confidently called as reference.
Two approaches work. If you only want variant sites, drop the reference blocks explicitly and then filter:
bcftools view -e 'ALT="<NON_REF>" || ALT="<*>"' sample.g.vcf.gz \
| bcftools view -T regions.bed --targets-overlap 2 -Oz -o sample.var.vcf.gz
If you need the reference blocks preserved, use --regions-overlap 2 or --targets-overlap 2 so that END is respected. Note that the block is emitted whole, with its original START and END, and may extend past the region boundary. Any coverage computation you do on the result must clip the block to the region itself. Do that clipping in your own code rather than assuming the VCF did it.
6. Verify the output
Filtering that silently returns nothing is the failure mode to guard against, so check the numbers every time.
bcftools index -n sample.vcf.gz # records in, from the index
bcftools index -n sample.panel.vcf.gz # records out
bcftools view -H sample.panel.vcf.gz | cut -f1 | uniq -c
bcftools index -n reads the record count from the index without decompressing, which is instant on a 4 million variant file. Compare the retained fraction against the fraction of the genome your BED covers. They will not match exactly, because variant density is higher in some regions than others. Coding regions in particular are depleted of common variants relative to intergenic sequence. An order-of-magnitude disagreement means a coordinate or naming problem.
Then spot-check boundaries by hand. Pick one interval and list what survived at each edge:
head -1 regions.bed # e.g. chr1 1000000 1001000
bcftools view -H -r chr1:999990-1001010 sample.panel.vcf.gz | cut -f1,2,4,5
Every POS should fall in 1000001-1001000 under --regions-overlap 0, and indels starting just before 1000001 may appear under value 2. Seeing what you expect here is worth more than reading the manual again.
A useful cross-check is to run the same filter with an independent implementation and diff the results. bedtools intersect applies straightforward interval logic with no VCF-specific special cases:
bedtools intersect -a sample.norm.vcf.gz -b regions.bed -header -u \
| bgzip -c > sample.bt.vcf.gz
bcftools index -n sample.bt.vcf.gz
-u reports each VCF record at most once regardless of how many intervals it hits, which is what you want. Without -u a variant overlapping two intervals appears twice. bedtools uses the REF allele span for VCF input, so expect its output to match --regions-overlap 1 more closely than 0. If the counts differ by a handful of records, those records are almost always boundary indels, and listing them with bcftools isec tells you which convention each tool applied.
GATK SelectVariants -L regions.bed --interval-padding 10 is a third implementation. It is much slower than bcftools on a whole-genome file and requires a sequence dictionary, but it is the natural choice if the rest of your pipeline is already GATK. Region restriction of this kind is standard inside variant callers too, where an analysis interval list bounds the work the caller does at all.3
7. Wrap it in a reproducible script
Once the command is settled, record it with the inputs and versions. The point is that a count you report six months from now can be regenerated.
#!/usr/bin/env bash
set -euo pipefail
VCF=$1
BED=$2
OUT=$3
bcftools --version | head -1
md5sum "$VCF" "$BED"
bcftools view -T "$BED" --targets-overlap 2 -Oz -o "$OUT" "$VCF"
bcftools index -t "$OUT"
bcftools index -n "$OUT"
For repeated work on the same VCF, convert to BCF once. bcftools view -Ob -o sample.bcf produces a binary format that parses several times faster than text VCF, which matters when you are filtering the same file against dozens of region sets. Storage and format choices of this kind dominate runtime and cost at biobank scale, where the same data is queried many times in different subsets.4 At single-person scale the same logic applies with smaller numbers, and BCF plus a CSI index is the format we keep on disk.
If you find yourself writing region-membership logic repeatedly, a small compiled tool is a reasonable alternative to shell pipelines. Go libraries for genomic intervals exist for exactly this purpose and keep interval arithmetic readable while staying fast.5 Dedicated filtering tools also exist if you prefer a configured pipeline over flags, and they accept the same bgzipped, indexed inputs.6
Common problems
Zero variants returned with exit status 0. Almost always a contig naming or assembly mismatch. Compare bcftools index --stats against cut -f1 regions.bed | sort -u.
Every interval shifted by one base. The region file was not named .bed, so bcftools parsed it as a 1-based region list. Rename it and rerun.
Duplicate records in the output. Overlapping intervals combined with -R. Run bedtools merge on the BED, or deduplicate with bcftools norm -d exact.
Output not position-sorted. The BED passed to -R was unsorted, and bcftools visits intervals in file order. Sort the BED before use.
Indels missing near boundaries. You are on --targets-overlap 0, which tests POS only. Set overlap mode 2 and rerun.
Counts differ between bcftools and bedtools. Different overlap conventions. Identify the disagreeing records with bcftools isec -p diff and check whether they are boundary indels before assuming a bug.
Reference blocks vanished from a gVCF. POS-based filtering dropped blocks that started before the region. Use overlap mode 2, and remember the surviving blocks extend beyond the boundary until you clip them.
Header still lists contigs that no longer appear. Cosmetic, but it breaks some strict parsers. bcftools view does not prune contig lines by default. Rebuild the header with bcftools reheader if a downstream tool complains.
Everything above is measurement. Restricting a VCF to a region set narrows what you look at and changes which variants you see. It says nothing about what any surviving variant means for your health. Interpretation of a specific variant, and any decision that follows from it, belongs with a clinical geneticist or genetic counselor working from a validated clinical report rather than a research-grade file you filtered yourself. Region-based filtering is a step in analysis, and the interpretive frameworks built on top of filtered VCFs are designed to be read by trained clinicians.2
Footnotes
-
Jean-Pierre Desvignes, Marc Bartoli, Valérie Delague, et al. VarAFT: a variant annotation and filtration system for human next generation sequencing data. Nucleic Acids Research, 2018. https://doi.org/10.1093/nar/gky471 ↩
-
Heiko Müller, Raul Jimenez-Heredia, Ana Krolo, et al. VCF.Filter: interactive prioritization of disease-linked genetic variants from sequencing data. Nucleic Acids Research, 2017. https://doi.org/10.1093/nar/gkx425 ↩ ↩2
-
David Jones, Keiran M. Raine, Helen Davies, et al. cgpCaVEManWrapper: Simple Execution of CaVEMan in Order to Detect Somatic Single Nucleotide Variants in NGS Data. Current Protocols in Bioinformatics, 2016. https://doi.org/10.1002/cpbi.20 ↩
-
Xihao Li, Andrew R. Wood, Yuxin Yuan, et al. Streamlining large-scale genomic data management: Insights from the UK Biobank whole-genome sequencing data. Cell Genomics, 2025. https://doi.org/10.1016/j.xgen.2025.101009 ↩
-
Eric H Au, Christiana Fauci, Yanting Luo, et al. Gonomics: uniting high performance and readability for genomics with Go. Bioinformatics, 2023. https://doi.org/10.1093/bioinformatics/btad516 ↩
-
Milad Eidi, Samaneh Abdolalizadeh, Soheila Moeini, et al. 123VCF: an intuitive and efficient tool for filtering VCF files. BMC Bioinformatics, 2024. https://doi.org/10.1186/s12859-024-05661-5 ↩
Pothos builds longitudinal molecular profiles of individuals: whole-genome sequencing, single-cell RNA sequencing, and plasma proteomics, delivered as a detailed report, every data file, and stored cells. Build your profile.