Wow - what do you think?

[yaximik]

Hi everyone,

I just found an intersting paper (attached), in which the three major platforms were compared in de novo NGS of a bacterial genome. To me, SOLiD failed miserably. I was considering this instrument for its unmatched accuracy! Your comments?

[gringer]

They are using bowtie to map, with --all, --best options [I presume]. Given that the Solid platform has vastly greater coverage (600x vs ~20x), I think these bowtie settings would be expected to over-emphasise any Solid false hits. Despite this, I think the paper demonstrates good performance for Solid. Better for Roche, but still favourable for Solid.

For example, look at Figure 2. In comparison to Illumina, Solid shows a higher true positive rate, and a lower false positive/negative rate at all positions on the graph.

Granted, the paper does mention that only half the Solid reads mapped to the genome, but I'm not convinced this is evidence that the platform is not appropriate. Colour-space mapping (I really hope they used the colour-space mapping feature of bowtie, because converting to base space then mapping would be bad) is tricky to evaluate, because it's harder to demonstrate that a miss is a result of sequence error, or mapping error. You can't just say, "well, the sequence in base space was completely wrong. there's no way that's true sequence", because colour-space has high accuracy between adjacent base pairs, rather than at the sequence level.

Of course, that's just my thoughts from the theory of colour-space. I don't have any practical experience yet.

[konrad98]

It would also be helpful if the authors could post which versions of reagents were used for sequencing - especially with the Illumina results. 9 million reads from 2 lanes is indicative of very old chemistry. In fact I'm almost convinced that they have used the old Illumina GA1 data for this.

[yaximik]

Most likely konrad98 is correct, but that is not the issue. Illumina 's results were not falling out, but SOLiD... In reply to ginger I wonder if lack of mapping, whether due to sequence errors or due to mapping errors, still may create a difficult problem if one has to go without a suitably close reference to map against.

I am also not expert in all these, so this is why opinions of experts here are would be very valuable.

[krobison]

You might also look at various Internet-published analyses of assembly of the German E.coli outbreak strain done on various platforms, as these include PacBio and Ion Torrent, as well as the latest Illumina reagents (HiSeq/MiSeq). No SOLiD though on that score that I know of.

[pmiguel]

It would also be helpful if the authors could post which versions of reagents were used for sequencing - especially with the Illumina results. 9 million reads from 2 lanes is indicative of very old chemistry. In fact I'm almost convinced that they have used the old Illumina GA1 data for this.

Yeah, this is a big issue for me. But the authors of this work did give some information in that regard. The Illumina instrument is listed as "an Illumina Genome Analyzer" and the 25 nt ME SOLiD run was from a SOLiD2 and the others from a SOLiD3. So 2 sub-generations back from where we stand now. Sadly, data from 18 months ago is just not a good indication of the present capabilities of next gen sequencers.

--
Phillip

[SeqAA]

SOLiD 2 chemistry still had higher accuracy than GAIIx.
The data is old but the trend is quite familiar. Lower false positives and higher accuracy with solid.

The low mapping is also the same story. Illumina reads are filtered before mapping and solid is not. I am surprised people still do not pick up on this so many years later.

[dottomarco]

Can someone provide links or pdfs of any other paper were the 2 or 3 technologies are compared especially regarding error rates, false <0 and <0, etc. ??
Looking at GA and SOLiD 2 is not that useful since we are using the new HiSeq and the new V3 chemistry...

[pmiguel]

Yes, what we really want is Illumina v3 chemistry vs SOLiD 5500XL. The 5500XL also has ECC chemistry that should lower the error rate further. (But it only is usable with the normal reads, not the reverse reads used in PE sequencing.)

By the time manuscripts doing those comparisons show up, there will be a new iterations of the chemistry/instruments that will make that data obsolete...

--

[pmiguel]

SOLiD 2 chemistry still had higher accuracy than GAIIx.
The data is old but the trend is quite familiar. Lower false positives and higher accuracy with solid.

The low mapping is also the same story. Illumina reads are filtered before mapping and solid is not. I am surprised people still do not pick up on this so many years later.

Well the install base of SOLiD is a fraction of that of Illumina sequencers. And Applied Biosystems has always had a tin ear towards this sort of thing. I remember there was a period of a couple of years where phred gave aberrantly high quality values to Megabase data. Near as I can tell this issue was completely ignored by Applied Biosystems, well, and nearly everyone else.

The issue is that the SOLiD does an entire imaging set on beads with just the dye-conjugated primer annealed. As a result they pretty much map every single bead on that slide -- including ones that have basically no chance of producing a base of usable sequence. Illumina does a much more slipshod job of cluster calling -- but the result is that many of the clusters that were never going to give useful data, are never identified as a cluster.

The other factor that gets ignored with facile "average quality" metrics is that quality values are assigned to the base calls. To the extent the quality values correctly reflect the accuracy of the call, the low quality bases can be ignored downstream.

Still, Applied Biosystems should use a "pass filter" to weed out the 10-20% of reads in a run that are of poor quality by default. That would probably make them look a lot better.

--
Phillip

[SeqAA]

well there is Stringency on v4 and a filter with 5500 i think.