[lcollado]

Hello,

I know that this is an old thread, but I'm curious to know how Swift compares vs the most recent SolexaPipeline versions.

Thank you!
Leonardo

[bioinfosm]

Are there any updates on SWIFT? data sizes, number of files generated, comparison with Illumina pipeline results..

[new300]

I'm quite interested in using open-source software for scientific work. We have recently acquired an Illumina GAII machine, and are trying to come up with data management solutions. Right now we are planning to throw away the images after the primary analysis (base-calling) is completed. We are saving the intensity and noise files, but not the images, which seems to be fairly common. However, it seems that this software requires the original images, which makes sense, but would limit our ability to use it on past experiments.

Are you using iPar to process the images and then mirroring off the intensity files? Swift will process from intensity files (as produced by the UNIX pipeline). I've heard the iPar intensity format is different from that used by the UNIX pipeline if someone wants to send me a sample file I'll write a parser for it.

Would it be feasible to use swift on the Firecrest output (intensity and noise)?

Yes it's feasible, I would hope the results would be comparable with the Illumina pipeline.

Do many labs actually save the image files?

It seems like an ideal initial setup would be to process the images with both the Illumina pipeline and Swift. Has anyone yet set this up?

As mentioned Sanger save the images while they do QC, the images are mirrored off as the run progresses and processed using the UNIX pipeline on a separate cluster.

If you're interested in trying out Swift drop me an email at new at sgenomics dot org. It's in ``active development'' at the moment and I'm happy to work with people on any issues that come up.

[clivey]

sanger have it set up - talk to Tom Skelley.

Images are still very diagnostic of any issue with your sample or sequencer (or run). Looking at images allowed sanger to optimise their pipeline. For example, when your flowcell quality goes down, or an operator gets oil on the flowcell etc., or your focusing is off and you suddenly get lots of strange new 'contaminants' in your output file as a result, or your base qualities all drop halfway through your project, youe data goes bad and you look and your clusters look wierd coz of an issue with your cluster station, or theres stuff growing in your reagents appearing as blobs on the images (but not visible to the naked eye), or your flowcell surface isnt there etc etc. You should keep them for QC - then throw them. Generally (but not in all cases) higher throughput labs with big projects indulge in some image retention for some period.

[lparsons]

I'm quite interested in using open-source software for scientific work. We have recently acquired an Illumina GAII machine, and are trying to come up with data management solutions. Right now we are planning to throw away the images after the primary analysis (base-calling) is completed. We are saving the intensity and noise files, but not the images, which seems to be fairly common. However, it seems that this software requires the original images, which makes sense, but would limit our ability to use it on past experiments.

Would it be feasible to use swift on the Firecrest output (intensity and noise)?

Do many labs actually save the image files?

It seems like an ideal initial setup would be to process the images with both the Illumina pipeline and Swift. Has anyone yet set this up?

[new300]

BTW - the link: http://swiftng.sourceforge.net appears to be broken.

The connection seems to be a problem only from my desktop at work (which is behind a US government firewall). From other locations i can get through OK.

Odd, you can try: http://sgenomics.org/swift/ which should also work.

[new300]

Is it normal to see different output when running the same binary version of swift on the same computer for multiple times and running it on different computers? I observed both. It looks like most of the differences in the fastq output is the quality scores.

Running on different computers it's quite likely that the output will vary slightly as they are likely to have different floating point implementations.

On the same computer is a little odd, how different are the results? If it's a small difference then this could be down to the FFTW implementation we are using which sometimes employs a non-deterministic algorithm.

[iris42]

Is it normal to see different output when running the same binary version of swift on the same computer for multiple times and running it on different computers? I observed both. It looks like most of the differences in the fastq output is the quality scores.

[cgb]

works for me

[timread]

BTW - the link: http://swiftng.sourceforge.net appears to be broken.

The connection seems to be a problem only from my desktop at work (which is behind a US government firewall). From other locations i can get through OK.

[new300]

In terms of memory usage we're trying to stay within a 2Gb limit. A 37Gb paired end peaks at around 1Gb.

[new300]

Could you maybe share some stats as to how Swift performs vs the current version of Bustard?
E.g. amount of data/reads mapped, error rate for the same lane analysed both ways.
thanks
david

I'm still in the process of validating it on non-phiX data. For the phiX data I've looked at, against the 1.0 pipeline I've seen 20% more PF reads at a similar error rate.

In terms of runtime, a GA1 single end takes around 10mins end to end. GA2 37 cycles paired end takes around an hour end to end.

[dvh]

Could you maybe share some stats as to how Swift performs vs the current version of Bustard?
E.g. amount of data/reads mapped, error rate for the same lane analysed both ways.
thanks
david

[new300]

i wonder if if can be put onto a boot DVD and run on the iPar computers - data mirrored in real time using the sanger mirroring scripts ?

Yes, this absolutely should be possible and is something we'd like to look in to. Users interested in doing this are encouraged to make contract.

[cgb]

cool

i wonder if if can be put onto a boot DVD and run on the iPar computers - data mirrored in real time using the sanger mirroring scripts ?