Ion Torrent PGM vs Illumina MiSeq

[nickloman]

Still, in case you do another one, my (unsolicited) input: I personally would like to see a dollar-for-dollar constrained test and an hour-for-hour constrained test of these instrument systems.

Yep, I think that's something we'd like to do in an ideal world. I think more likely we can improve on this test by turning it into more of a community effort - and not just for E. coli but for more complex genomes, high/low GC genomes, amplicons, exomes etc which is obviously important to people.

I'll post some ideas up when I've had time to formulate the idea a bit more.

If I could switch the topic somewhat. In figure S2, in this pdf, it looks like the Torrent and the MiSeq both slightly undervalue the quality of their base calls for most quality values and the 454 Jr slightly overvalues the quality of their base calls. Am I reading the chart correctly? I knew the Torrent tended towards "modesty" in these matters. But I had the vague impression that Illumina tended towards "bluster".

Yes, you are reading that chart correctly and your suspicions are generally correct.

That plays out when you calculate measured quality for Ion Torrent PGM (actual quality better than expected) and 454 Jr (actual quality lower than expected) (Figure 1). I guess what was surprising to me when I did this was that the PGM quality caller is actually better than many had thought - it had been reported on blogs that it was seriously underestimating accuracy, but turns out that was because only nucleotide substitutions were counted in the recalculation process (something to watch out for with GATK).

For MiSeq the actual quality is a bit down on expected. But bearing in mind Phred's log-scale it turns out its hardly noticeable when plotted on a linear scale of % accuracy, whereas the effect is much more obvious for the other two platforms. So I guess I am less trustful of the top end of the quality values being precisely correct from MiSeq.

[pmiguel]

Part of the idea of the paper was to be practical, indeed these data were used for outbreak epidemiology at the time. One E. coli strain per two 454 Jr runs, one strain per 316 chip and 7 strains per MiSeq run is the kind of experimental design you would envisage in a real lab.

Not to back away from what I wrote -- it looks from my vantage point like you and your colleagues let your sense of fair play prevent the 454 Jr from getting trounced. That said, I could second-guess your experimental design all day, but you and your colleagues actually did the experiment. I am just sitting here taking pot shots. Still, in case you do another one, my (unsolicited) input: I personally would like to see a dollar-for-dollar constrained test and an hour-for-hour constrained test of these instrument systems.

If I could switch the topic somewhat. In figure S2, in this pdf, it looks like the Torrent and the MiSeq both slightly undervalue the quality of their base calls for most quality values and the 454 Jr slightly overvalues the quality of their base calls. Am I reading the chart correctly? I knew the Torrent tended towards "modesty" in these matters. But I had the vague impression that Illumina tended towards "bluster".

That is, it looks to me from the chart that the MiSeq bin of bases with a quality score of 20 (1 error in 100) actually had 1 error in 1000. Indicating that they really were Q30 bases, rather than Q20 bases?

--
Phillip

[nickloman]

About Nick's paper. Anyone else think that the MiSeq got sandbagged? Especially in comparison to the GS-Jr. Only 10% of a MiSeq run was used for the assembly. (And, mysteriously, 1/2 of that ended up mapping to one of the plasmids.) With ABySS I have seen improvements in contig/scaffold lengths up to 100x coverage.

--
Phillip

had

Hi Philip,

A fair point!

The MiSeq run we had multiplexed seven E. coli strains on a single run, and the comparison strain 280 came out about 15% of the run, pretty much what was expected. And yes, we had very high plasmid coverage (we hypothesise why in the paper) so we lost out on chromosomal coverage a bit.

But yes I think you are right, if we'd had more coverage we may have got better MiSeq assemblies. But I was still quite impressed with their performance and you saw they were the most effective at reconstructing gene space. Note we were able to use MIRA on all the data which is an Overlap-Layout-Consensus assembler and doesn't need quite as high coverage as de Bruijn graphs to work well (indeed too high is a problem).

Part of the idea of the paper was to be practical, indeed these data were used for outbreak epidemiology at the time. One E. coli strain per two 454 Jr runs, one strain per 316 chip and 7 strains per MiSeq run is the kind of experimental design you would envisage in a real lab.

Hope that's helpful

Nick

[scbaker]

The platform comparison is old enough to be of mainly historical interest. Dating back to the end of 2010 puts it before the Illumina's v3 HiSeq chemistry tripled the output of that instrument and put a stake through the SOLiD's heart. Also at that point it was not entirely obvious that Roche had abandoned the GS-FLX to tinker around with the their benchtop GS-Jr.

Cool though. Very comprehensive list of next gen sequencers.
--
Phillip

Yes, that page is horribly out of date. To be honest, it's (at least partially) because we were never that happy with the structure (e.g. lumping all of the Illumina platforms into a single bucket doesn't make a lot of sense). The site is migrating to a new backend system in a week or two that should give us more options for displaying info. In the meantime, most of the individual platform pages are up to date (well, at least better than the table).

We also need to add links to other great resources like SeqAnswers and the world map of NGS placements. So much to do, so little time... :-)

[pmiguel]

Feel free to check out BlueSEQ’s knowledge bank for more in depth, neutral assessments of the various sequencing platforms: http://blueseq.com/knowledgebank/home


Shawn

The platform comparison is old enough to be of mainly historical interest. Dating back to the end of 2010 puts it before the Illumina's v3 HiSeq chemistry tripled the output of that instrument and put a stake through the SOLiD's heart. Also at that point it was not entirely obvious that Roche had abandoned the GS-FLX to tinker around with the their benchtop GS-Jr.

Cool though. Very comprehensive list of next gen sequencers.
--
Phillip

[pmiguel]

About Nick's paper. Anyone else think that the MiSeq got sandbagged? Especially in comparison to the GS-Jr. Only 10% of a MiSeq run was used for the assembly. (And, mysteriously, 1/2 of that ended up mapping to one of the plasmids.) With ABySS I have seen improvements in contig/scaffold lengths up to 100x coverage.

--
Phillip

[snetmcom]

Can't resist responding to this...ease of preparation goes hands down, no contest, to the MiSeq.

Won't expand too much...but suffice to say that you can be sequencing your library on the MiSeq 15 minutes after it's quantitated. With the PGM you still have at least several hours of hands on manual labor (instrument prep, emulsion setup, enrichment, bead quantitation, chip prep/loading, etc), that involve two other pieces of instrumentation.

Not to mention you can set a MiSeq on a bench, plug it in and it works. Doesn't need uber-grade water, a constant supply of argon, or a giant server.

It's not quite several hours. More like an hour or so.

Can anyone comment on miseq analysis times. Does it do alignments? I haven't used one in many months and at the time the software was still dealing with windows issues.

[james hadfield]

You should read http://www.nature.com/nbt/journal/va.../nbt.2198.html
which compares the main platforms.

[MrGuy]

I noticed you aren't in a major metropolis that these sequencing companies pay attention to. Another thing to think about is if you can actually get the reagents, service, instrumentation, etc in a timely manner. It doesn't matter how great either machine is if you can't get the stuff to run it.

[scbaker]

Ramunas,

I saw a similar question to this on LinkedIn, and this is what I posted:

Points to consider about the MiSeq:
• Good choice if you already have access to a HiSeq (either your own or through a collaborator) as the workflow is very similar and the libraries will be cross compatible.
• This is generally considered the easier/simpler of the two platforms.
• Read length and output per run has been expanding, but not at the rate for the Ion Torrent platforms
• Runs are relatively quick (~24 hours), but not nearly as quick as for Ion Torrent (2-4 hours)


Points to consider about Ion Torrent:
• The Ion Torrent PGM is the “lower throughput” machine which is topping out at ~1Gb per run (much less than the MiSeq)
• The Ion Proton is the newer instrument which should have outputs of 10Gb (sometime this summer) and 100Gb (by early next year) – 100Gb is substantially more output than what Illumina has said the MiSeq will be capable of producing.
• The Ion Torrent machines have substantially quicker runtimes (~2-4 hours vs ~24 hours)
• Currently, both the PGM and the Proton use emulsion PCR for amplifying libraries prior to sequencing. While there are kits and instruments to help automate this, emulsion PCR is generally considered to be more of a hassle than the ‘bridge amplification’ that the MiSeq uses .

Feel free to check out BlueSEQ’s knowledge bank for more in depth, neutral assessments of the various sequencing platforms: http://blueseq.com/knowledgebank/home

Also definitely check out Nick Loman's paper.

Shawn

[low99]

I have to agree with ECO, for ease of use and data quality, the MiSeq is the best choice by far.
Nick Loman's comparison paper also highlights the systemic homopolymer issue that's inherent to the Ion Torrent chemistry

[ECO]

Can't resist responding to this...ease of preparation goes hands down, no contest, to the MiSeq.

Won't expand too much...but suffice to say that you can be sequencing your library on the MiSeq 15 minutes after it's quantitated. With the PGM you still have at least several hours of hands on manual labor (instrument prep, emulsion setup, enrichment, bead quantitation, chip prep/loading, etc), that involve two other pieces of instrumentation.

Not to mention you can set a MiSeq on a bench, plug it in and it works. Doesn't need uber-grade water, a constant supply of argon, or a giant server.

[snetmcom]

tough choice really. The paper was informative but already outdated.

[wraithnot]

A paper that came out yesterday in Nature Biotechnology may provide some of the answers you are seeking: http://www.nature.com/nbt/journal/va...f/nbt.2198.pdf

Hello all users,
recently i've received a grant and plan to purchase small scale NGS platform for targeted sequencing of human genome (germline mutations in hereditary cancer genes, in particular).
Now hesitating between the Ion Torrent PGM and Illumina MiSeq.
Since i imagine the price would be similar for both instruments (with additional accessories) what would be your suggestions regarding the advantages/disadvantages of these platforms?
The critical factors for me would be the quality of reads, ease of preparation and analysis.

Thank you in advance,

Ramunas

[ramujana]

Hello all users,
recently i've received a grant and plan to purchase small scale NGS platform for targeted sequencing of human genome (germline mutations in hereditary cancer genes, in particular).
Now hesitating between the Ion Torrent PGM and Illumina MiSeq.
Since i imagine the price would be similar for both instruments (with additional accessories) what would be your suggestions regarding the advantages/disadvantages of these platforms?
The critical factors for me would be the quality of reads, ease of preparation and analysis.

Thank you in advance,

Ramunas