[GenoMax]

Can you post the cluster density (K/mm^2) for these runs and the version of chemistry you are using? It is possible that you have an overloaded run this time and that may have resulted in Q-score drops. Once the clusters become fat the instrument has trouble distinguishing them apart.

What kind of data is this (we seem to be doing some of this backwards)?

[M4TTN]

Data in all cases is whole genome sequencing of S.cerevisiae genomic DNA (also containing mitochondiral DNA and an endogenous plasmid element). Prepared using NexteraXT using all standard conditions and sequenced with 2x300bp on MiSeq v3 chemistry. 24 samples multiplexed (and successfully demultiplexed).

The only change was the last run we used less Ampure beads during smaple prep than for first 3 runs (we used the conc recommended in the brochure for 2x250bp, whereas previously we'd used more Ampure beads) to size select larger amplicons.

We think this probably worked because we have less "duplicates" in the SAM file suggesting that fewer paired reads fully overlapped (which occurs when the inserts are shorter than 300bp). The last run was also sequenced at 350+250bp, however, so duplicates will only arise for fragments shorter than 250bp.

The cluster densities for the four runs (in order) are:

Run 1. 1399K/mm2; 90% pass filter; 33.4M clusters, 30.08 pass filter. Created lots of good quality data.
Run 2. 1212K/mm2; 92.3% pass filter; 29.15M clusters; 26.9M pass filter. Created decent data.
Run 3. 999K;mm2; 90.9% pass filter; 24.3M clusters; 22.1M pass filter. Created decent data.
Run 4. 740K/mm2; 77.8% pass filter; 18.4M clusters; 14.45 pass filter. Created data that aligns less well (75% versus 97% for other runs) and has clear quality issues and perfect read issues in PhiX library. It also looks like the bottom flowcell surface was particularly affected.

Could it be that the loading cluster density was too low, impacting clusters passing filter? 77.8% passing is very bad compared to other runs. (740K/mm2 doesn't seem that much below target loading to me though - indeed, the first run 1399, is well above target).

Yet even in those that passed, the data is crappy (in my opinion).

As an example, I attach an image of the PhiX error rate for the bottom surface for the four runs. To me, it looks like run 4 has a massive issue. The top surface is also not as good as the others in read1, but interestingly in read2 actually looks better (second image).

https://www.dropbox.com/s/uob4blluhctxbys/PhiX%20quality.pdf?dl=0

https://www.dropbox.com/s/zn2v3uce3sehhtl/PhiX%20error%20run%204%20top%20surface.png?dl=0

[joshquick]

When you decreased the SPRI ratio you increased the insert size and we found bowtie2 behaves weirdly above 500bp inserts with the default settings. You might want to change your bowtie2 settings or try bwa-mem:

http://nickloman.github.io/2013/05/02/use--x-with-bowtie2-to-set-minimum-and-maximum-insert-sizes-for-nextera-libraries/

[M4TTN]

When you decreased the SPRI ratio you increased the insert size and we found bowtie2 behaves weirdly above 500bp inserts with the default settings. You might want to change your bowtie2 settings or try bwa-mem:

http://nickloman.github.io/2013/05/0...era-libraries/

Actually this is a very good point that I knew about (from reading your post earlier) but recently forgot about. It turns out that part of the poor mapping of the recent run (aside from issues with quality that Illumina are investigating) are due to the difference between running Bowtie2 in Ugene (with default settings) and my coworker running Bowtie2 on a cluster with -X 1000.

Default settings gives about 75% alignment, whereas with X-1000 we achieved 94% (still not as good as some other libraries - 97% - , so we think that probably the low overall sequence quality in the last run is affecting mapping).