Custom Filter Development

Learn how to create custom filter expressions for specialized analysis needs. VariantCentrifuge supports multiple filtering approaches at different stages of the pipeline.

Filtering Options Overview

1. SnpSift Filters (--filters)

Applied early in the pipeline on VCF data. Use SnpSift’s Java-like expression syntax.

2. bcftools Pre-filters (--bcftools-prefilter)

Applied during variant extraction for performance. Uses bcftools expression syntax.

3. Final Filters (--final-filter)

Applied at the end of the pipeline on the final TSV. Uses pandas query syntax.

SnpSift Filter Syntax

SnpSift uses a Java-like expression syntax:

# Basic comparisons
"QUAL >= 30"
"AC[0] <= 5"
"AF[0] < 0.01"

# Field existence
"exists ClinVar_CLNSIG"
"na gnomAD_exomes_AF"

# String matching
"ClinVar_CLNSIG =~ '[Pp]athogenic'"
"ANN[ANY].IMPACT has 'HIGH'"

# Complex combinations
"((QUAL >= 30) & (AC[0] <= 5)) | (ClinVar_CLNSIG =~ '[Pp]athogenic')"

Note

ANN[ANY], LOF[*], and NMD[*] are record-level predicates unless SnpEff annotations are split before SnpSift. For transcript-filtered gene burden or association analyses where the retained row itself must satisfy a consequence filter, use --split-snpeff-lines before_filters. The after_filters and no-split modes can still be useful, but they filter the original multi-annotation record before transcript selection.

bcftools Pre-filter Syntax

bcftools uses a different syntax optimized for speed:

# Basic filters
--bcftools-prefilter 'FILTER="PASS"'
--bcftools-prefilter 'QUAL>30'
--bcftools-prefilter 'INFO/AC<10'
--bcftools-prefilter 'INFO/AF<0.01'

# Combined filters
--bcftools-prefilter 'FILTER="PASS" && QUAL>30 && INFO/AC<10'
--bcftools-prefilter 'INFO/AF<0.001 || INFO/AC<5'

Final Filter Syntax (pandas query)

The --final-filter option uses pandas query syntax, allowing filtering on any column including computed values:

# Numeric comparisons
--final-filter 'inheritance_score > 0.8'
--final-filter 'CADD_phred >= 20'
--final-filter 'Inheritance_Confidence > 0.9'

# String equality
--final-filter 'IMPACT == "HIGH"'
--final-filter 'Inheritance_Pattern == "de_novo"'

# String contains
--final-filter 'Custom_Annotation.str.contains("cancer_panel")'
--final-filter 'GENE.str.startswith("BRCA")'

# IN operator
--final-filter 'Inheritance_Pattern in ["de_novo", "compound_heterozygous"]'
--final-filter 'GENE in ["BRCA1", "BRCA2", "TP53"]'

# Complex expressions
--final-filter '(IMPACT == "HIGH" and inheritance_score > 0.7) or CADD_phred > 30'
--final-filter 'Inheritance_Pattern == "de_novo" and Inheritance_Confidence > 0.8'

Filtering Strategy Examples

Performance-Optimized Filtering

# Use bcftools pre-filter to reduce data early
variantcentrifuge \
  --bcftools-prefilter 'FILTER="PASS" && INFO/AC<20' \
  --preset rare,coding \
  --final-filter 'inheritance_score > 0.5' \
  ...

Score-Based Filtering

# Use late filtering or final filter for computed columns
variantcentrifuge \
  --scoring-config-path scoring/my_model \
  --late-filtering \
  --filters "my_score > 0.7" \
  ...

# Or use final filter
variantcentrifuge \
  --scoring-config-path scoring/my_model \
  --final-filter 'my_score > 0.7 and IMPACT != "LOW"' \
  ...

Inheritance-Based Filtering

# Filter for high-confidence de novo variants
variantcentrifuge \
  --ped family.ped \
  --inheritance-mode columns \
  --final-filter 'Inheritance_Pattern == "de_novo" and Inheritance_Confidence > 0.9' \
  ...

Example Custom Filter Presets

{
  "presets": {
    "custom_rare_high_impact": "(((gnomAD_exomes_AF < 0.0001) | (na gnomAD_exomes_AF)) & ((ANN[ANY].IMPACT has 'HIGH') | ((ANN[ANY].IMPACT has 'MODERATE') & (dbNSFP_CADD_phred >= 25))))"
  }
}