Germline Mutation Calling
This Snakemake workflow follows the GATK best-practice recommandations to call small germline variants.
The pipeline requires as inputs aligned BAM files (e.g. with BWA) where the duplicates are already marked (e.g. with Picard or sambamba). It then performed Base Quality Score Recalibration and joint genotyping of multiple samples, which is automatically parallized over user defined intervals (for examples see intervals.txt) and chromosomes.
Filtering is performed using GATKs state-of-the-art Variant Quality Score Recalibration
At the end of the worklow, the Variant Effect Predictor is used to annotate the identified germline mutations.
A high level overview of the performed steps can be seen below:
As seen by the execution graph, an arbitrary number of samples/BAM files can be processed in parallel up to the joint variant calling.
Installation
Required tools:
The majority of the listed tools can be quite easily installed with conda which is recommanded.
Usage
First, modify the config_wgs.yaml and resources.yaml files. Both files contain detailed description what is expected. The config_wgs.yaml also contains links to some reference resources. Be careful, they are all specific for the GRCh37/hg19/b37 genome assembly.
After setting up all the config files and installing all tools, you can simply run:
snakemake --latency-wait 300 -j 5 --cluster "sbatch --mem={resources.mem_mb} --time {resources.runtime_min} --cpus-per-task {threads} --job-name={rule}.%j --output snakemake_cluster_submit_log/{rule}.%j.out --mail-type=FAIL"
This assumes that the cluster you are using is running SLURM. If this is not the case, you have to adjust the command after --cluster. The log information of each job will be safed in the snakemake_cluster_submit_log directory. This directory will not be created automatically.
-j specifies the number of jobs/rules should be submitted in parallel.
I recommand running this command in a detached session with tmux or screen.
Output
Below is the output of the tree command, after the workflow has finished for one patient H005-00ML. Usually you would include many patients simultaneously (>50). This is just to illustrate the created output files.
.
├── cohort
│ ├── benchmark
│ │ ├── ApplyVQSR_indel.txt
│ │ ├── ApplyVQSR_snp.txt
│ │ ├── CombineGVCFs.txt
│ │ ├── GenotypeGVCFs.txt
│ │ ├── MergeCohortVCFs.txt
│ │ ├── SelectVariants.txt
│ │ ├── VEP.txt
│ │ ├── VQSR_indel.txt
│ │ └── VQSR_snp.txt
│ ├── cohort.recalibrated.pass.vep.vcf.gz
│ ├── cohort.recalibrated.pass.vep.vcf.gz_summary.html
│ ├── cohort.recalibrated.vcf.gz
│ ├── cohort.recalibrated.vcf.gz.tbi
│ └── logs
│ ├── ApplyVQSR_indel.out
│ ├── ApplyVQSR_snp.out
│ ├── CombineGVCFs
│ ├── CombineGVCFs.1.out
│ ├── CombineGVCFs.2.out
│ ├── ...
│ ├── ...
│ ├── CombineGVCFs.Y.out
│ ├── GenotypeGVCFs.1.out
│ ├── GenotypeGVCFs.2.out
│ ├── ...
│ ├── ...
│ ├── GenotypeGVCFs.Y.out
│ ├── MakeSitesOnly.out
│ ├── MergeCohortVCFs.out
│ ├── SelectVariants.err
│ ├── VEP.out
│ ├── VQSR_indel.out
│ └── VQSR_snp.out
├── config
│ ├── config_wgs.yaml
│ └── resources.yaml
├── H005-00ML
│ ├── benchmark
│ │ ├── ApplyBQSR.txt
│ │ ├── BaseRecalibrator.txt
│ │ ├── GatherBQSRReports.txt
│ │ ├── GatherRecalBamFiles.txt
│ │ ├── HaplotypeCaller.txt
│ │ ├── IndexBam.txt
│ │ ├── MergeHaplotypeCaller.txt
│ │ └── SortBam.txt
│ ├── H005-00ML.germline.merged.g.vcf.gz
│ ├── H005-00ML.germline.merged.g.vcf.gz.tbi
│ └── logs
│ ├── ApplyBQSR
│ ├── ApplyBQSR.0000-scattered.interval_list.out
│ ├── ApplyBQSR.0001-scattered.interval_list.out
│ ├── ...
│ ├── ...
│ ├── ApplyBQSR.0049-scattered.interval_list.out
│ ├── BaseRecalibrator
│ ├── BaseRecalibrator.0000-scattered.interval_list.out
│ ├── BaseRecalibrator.0001-scattered.interval_list.out
│ ├── ...
│ ├── ...
│ ├── BaseRecalibrator.0049-scattered.interval_list.out
│ ├── GatherBQSRReports.out
│ ├── GatherRecalBamFiles.out
│ ├── HaplotypeCaller
│ ├── HaplotypeCaller.0000-scattered.interval_list.out
│ ├── HaplotypeCaller.0001-scattered.interval_list.out
│ ├── ...
│ ├── ...
│ ├── HaplotypeCaller.0049-scattered.interval_list.out
│ ├── IndexBam.out
│ ├── MergeHaplotypeCaller.out
│ └── SortBam.out
├── rules
│ ├── BaseQualityScoreRecalibration.smk
│ ├── JointGenotyping.smk
│ ├── VEP.smk
│ └── VQSR.smk
├── Snakefile
├── snakemake_cluster_submit_log
│ ├── ApplyBQSR.24720887.out
│ ├── ApplyVQSR_snp.24777265.out
│ ├── BaseRecalibrator.24710227.out
│ ├── CombineGVCFs.24772984.out
│ ├── GatherBQSRReports.24715726.out
│ ├── GatherRecalBamFiles.24722478.out
│ ├── GenotypeGVCFs.24773026.out
│ ├── HaplotypeCaller.24769848.out
│ ├── IndexBam.24768728.out
│ ├── MergeCohortVCFs.24776018.out
│ ├── MergeHaplotypeCaller.24772183.out
│ ├── SelectVariants.24777733.out
│ ├── SortBam.24768066.out
│ ├── VEP.24777739.out
│ ├── VQSR_indel.24776035.out
│ └── VQSR_snp.24776036.out
For each analyzed patient, a seperate directory gets created. Along with the patient specific gvcf file, this directory contains log files for all the processing steps that were performed for that patient (log directory) as well as benchmarks for each rule, e.g. how long the step took or how much CPU/RAM was used (benchmark directory).
The cohort directory contains the multi-sample VCF file, which gets created after performing the joint variant calling. The cohort.recalibrated.vcf.gz is the product of GATKs Variant Quality Score Recalibration. The cohort.recalibrated.pass.vep.vcf.gz is the filtered and VEP annotated version of cohort.recalibrated.vcf.gz (only variants with PASS are kept).
For most applications, the cohort.recalibrated.pass.vep.vcf.gz file, is the file you want to continue working with.
2.5k Dec 28, 2022
7 Apr 05, 2022
227 Jan 05, 2023
16 Sep 15, 2022
4k Jan 05, 2023
1 Jan 31, 2022
1 Oct 07, 2021
249 Jan 06, 2023
16.7k Jan 08, 2023
1k Jan 01, 2023
1.3k Jan 02, 2023
22 Oct 31, 2022
1 Feb 05, 2022
2 Mar 29, 2022
4 Oct 12, 2022
2 Nov 04, 2021
27 Jul 09, 2022
1 Jun 26, 2022
2 Dec 13, 2021
3 Oct 24, 2019