FASTQ

FASTQ is a line-oriented text format where each sequencing read occupies exactly four lines: @ plus an identifier, the base calls, a + separator, and a quality string with one ASCII character per base. It is the standard handoff format from a sequencer to everything else.

A single record from a paired-end Illumina run looks like this:

@A00758:412:HMJ7YDSX3:1:1101:5936:1000 1:N:0:ATCCACTG+AGGTGCGT
NGATCACAGGTCTATCACCCTATTAACCACTCACGGGAGCTCTCCATGCAT
+
#FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:F

The header carries instrument, run, flowcell, lane, tile, and x/y coordinates, then the read number (1), a filter flag, and the index sequences. The quality string is Phred+33: subtract 33 from the ASCII code to get Q. # is Q2, : is Q25, F is Q37. Q30 means a 1-in-1000 estimated probability that the base call is wrong.

How it works

A read is one contiguous stretch of DNA the instrument observed, not a coordinate. Nothing in a FASTQ file says where in the genome anything came from. Alignment does that, and alignment is the first lossy step: reads that map ambiguously, reads in repeats, and soft-clipped ends all get flattened in a BAM. This is why long-read workflows targeting repeats and methylation start from raw signal or raw reads rather than someone else’s alignment.¹

Paired-end data arrives as two files, _R1.fastq.gz and _R2.fastq.gz, with records in the same order. Record n in R1 and record n in R2 are the two ends of the same DNA fragment. Order matters. Sorting one file and not the other silently corrupts everything downstream.

Modern instruments do not emit full 0–41 quality ranges. Illumina’s RTA3 bins qualities into a handful of values (roughly Q2, Q12, Q23, Q37), which costs little accuracy and compresses far better, since the quality string is the dominant term in FASTQ size.² That is why a 30× human whole genome lands in the 40–60 GB range gzipped rather than the ~200 GB the uncompressed text would occupy.

In your own data

Count reads by lines, not by @:

zcat sample_R1.fastq.gz | wc -l | awk '{print $1/4}'

grep -c "^@" is wrong. @ is a legal quality character (Q31), so it appears at the start of quality lines and inflates the count. For a full summary, use seqkit stats -a *.fastq.gz, which gives read count, total bases, N50, and Q20/Q30 percentages in one pass.

What to check when a delivery lands:

• Verify checksums against what the lab sent (md5sum -c manifest.md5). Truncated gzip transfers are the single most common failure, and a truncated .gz often still decompresses for hundreds of millions of reads before failing.
• Confirm gzip -t passes on every file.
• Confirm R1 and R2 have identical record counts. A mismatch means a broken transfer or a bad demultiplex, and every aligner will either error out or, worse, misassociate pairs.
• Run fastp for a first-pass QC and trim: fastp --in1 R1.fq.gz --in2 R2.fq.gz --out1 R1.trim.fq.gz --out2 R2.trim.fq.gz --detect_adapter_for_pe --thread 16 --html qc.html --json qc.json. It does quality profiling, adapter detection, polyG trimming for two-color chemistry, and optional duplicate estimation in one pass over the data, which matters when the input is tens of gigabytes.³ PRINSEQ++ is a reasonable alternative with a similar multi-threaded design and more granular filter predicates.⁴

Read the qc.json. The numbers we look at first: percentage of bases ≥ Q30 (expect above 85–90% for a healthy Illumina WGS run), duplication rate, adapter content, and GC distribution. A bimodal GC curve usually means contamination or a mixed library.

Two more traps. First, fastq-dump from SRA without --split-files concatenates mates into single reads. Use fasterq-dump --split-3. Second, if you re-compress with plain gzip, you lose nothing functionally, but bgzip blocks let tools seek and parallelize. Prefer bgzip -@ 8.

Limitations

FASTQ stores no reference, no mapping, no sample metadata beyond what is crammed into the header, and no per-file schema. Two labs can produce structurally valid files that differ in header convention, line wrapping, and quality binning. Older data may use Phred+64 (Illumina 1.3–1.7), which will be read as absurdly high qualities by modern tools unless you detect and convert it.

Quality scores are the instrument’s model of its own error, not ground truth. They do not capture PCR errors introduced before the read, and they say nothing about whether a region is mappable. Benchmarks built on fully resolved diploid assemblies exist precisely because per-base Q values understate error in repeats and segmental duplications.⁵

FASTQ also does not tell you anything about health. Read counts and Q30 fractions are measurements of a machine, and any interpretation of the variants you eventually call from them belongs with a clinician and a clinically validated assay.

Woolf Software builds longitudinal molecular profiles of individuals: whole-genome sequencing, RNA sequencing, proteomics, blood biomarkers, and continuous glucose data, integrated into one model of you. Build your profile.