Data Processing Pipeline question

[mboursnell]

Can anyone suggest an optimum data processing pipeline for analysing dog next gen sequencing data. We start with BAM files, and generally have about 5 cases and 5 controls. (10 samples = 10 BAM files). We don't want to ignore known SNPs since different breeds have different SNPs.

Is there a way to analyse all the BAM files in parallel so that information from all of them can be used in producing aligned cleaned deduped BAM files?

[Rocketknight]

There is indeed, if you're using GATK. Make sure every BAM file has a unique read group (you can use Picard's AddOrReplaceReadGroups function), mark duplicates with Picard, then merge all the BAM files into one (each read will retain its read group identifier, so you can distinguish them later).

After that, proceed with the standard GATK pipeline on this single merged BAM file. This has several advantages over single-sample processing: Firstly, novel indels from one sample can be used to help realignment in other samples. Secondly, you can call variants for all samples simultaneously with GATK's UnifiedGenotyper. You can then do VQSR on this multi-sample VCF file, which allows you to use population-level information (InbreedingCoeff) to find false-positive SNPs.

[mboursnell]

Thanks. Do I use picard/MergeSamFiles to do the merging?

[Rocketknight]

Yep, that will work. (Edit: Make sure you have SORT_ORDER=coordinate set when you merge, as GATK will expect your BAMs to be sorted)

[mboursnell]

Do you use the Queue.jar and the DataProcessingPipeline.scala file to run the standard GATK pipeline, or do you make your own pipeline (e.g. in PERL) to do the same thing?

[Rocketknight]

I tinkered with GATK-Queue, but I had a couple of problems (and I'm not too familiar with Java/Scala), so in the end I just went with a simple Python script to run everything. I used Python's multiprocessing module to run multiple samples at once in order to take advantage of multiple cores without having to split single samples by region and recombine, but this won't be possible if you're merging all your BAM files into one (unless you have several multi-sample BAM files you'd like to process concurrently).

[mboursnell]

Would it be possible to have a look at your Python script to help me setting up my PERL script? [email protected]

[Rocketknight]

Sure thing, just sent it there.

[angelinasusan]

Could I take a look at your script?? I badly need some help with a pipeline I am building and this would be very helpful. my id : [email protected]

[mrood]

script please?

Hi, would anyone be willing to send me their script to look at? I am new to programming and would love an example to build my own off of! [email protected]
Thanks in advance!

[Jeremy]

You could also use Samtools mpileup and vcftools, it treats each bam file as a separate sample.