Thanks seb567 for the detailed response.
One point I missed earlier is the approx coverage of data recommended by Ray for Solexa data. I know velvet recommends ~40x, and is not efficient with more or less coverage.
Also, I heard MIRA is an assembler that combines various read technologies data..
-
SOLiD will require some thought as errors, variants and combinations thereof manifest differently with respect to each other. Hopefully you will embrace SOLiD data as many groups/labs are clamoring for an easy and powerful assembler for SOLiD data.Originally posted by seb567 View PostLeave a comment:
-
Ray is based on the OpenAssembler algorithm, and we did compare OpenAssembler with ABySS, EULER, Newbler, and Velvet for our submitted paper on OpenAssembler (submitted on 15 October 2009).
We compared on SRA001125, an Illumina paired-end reads dataset, and SRA003611, a mix of 454 and Illumina reads, as well as many simulated datasets (with and without randomly incorporated errors). The conclusion was that only OpenAssembler can mix technologies, and that OpenAssembler is the best on Illumina paired and non-paired data.
EULER was the worst (the very worst!) in my benchmarks. Velvet was very good, and Newbler was the best on 454 (only Newbler worked with 454 in my benchmarks)
Because OpenAssembler auto-learns from the data instead of trying to figure out the statistics like in Velvet (they created VelvetOptimizer to alleviate that shortcoming!), we think the usability is better.
Virtually, if errors are incorporated randomly (Illumina, and SOLiD), and if the coverage is rather uniform (any technology, I think), then we have strong theoretical support to say that the conservative approach of OpenAssembler disallows any miassemblies, (chimeric contigs), but we observed some mismatches. This theoretical support is provided by a set of rules, heuristics, and some invariants.
Tired of waiting for the reviewing process, I decided to start Ray and release its source code as soon as possible!
Accuracy:
OpenAssembler does not produce chimeric contigs, but produces some mismatches when errors are present in reads (28 mismatches for SRA001125!). On SRA001125, Ray produces no chimeric contigs, and a few mismatches.
Time:
For SRA001125, with all the reads from NCBI SRA, it takes about 30 minutes on 31 MPI processes. Human chromosome 1, with 50-nt reads at 50 X takes about 2 hours on 400 MPI processes (Itanium, Infiniband).
Memory:
Ray is gentle with the memory usage. It uses SplayTree (who uses them anyway??). In a splay tree, the keys accessed often are near the root whereas keys accessed a few times will be in the leaves.
Ray distributes everything on MPI processes: reads, paired-end linkages, vertices, arcs, seeds, extensions, fusions, finished fusions. To communicate, Ray utilizes about 90 message types!, so Ray instances like to communicate!
If you want to know about memory usage, check Vertex.h. The coverage is stored on a uint8_t, edges are stored on a uint8_t, and there are some linked lists too.
In Ray, there is no tip cutting, and no bubble popping, which makes it a very different approach in comparison with Velvet/soapdenovo/ABySS/EULER.
But remember, a genome assembler is like an interpreter (python/perl/ruby), and its execution depends on the program (the reads) you give it, so you can't really summarise things that much.
Enjoy Ray!
***
The Ray Project Team
Leave a comment:
-
Have you had a chance to compare it to the existing assemblers? both for accuracy and time, and also memory?Leave a comment:
-
Ray -- questions & answers!
[Q] It says "if your sff file contains paired-end reads, you must first extract the information, and tell Ray to use them with LoadPairedEndReads". Do you mean we should extract as FASTA with sffinfo, remove the linker, and create a .fasta/.fastq file?
That is right. Ray don't create paired-end reads from SFF file.
[Q] Is "OpenAssembler" the same software as "Ray" ?
No, but Ray is a parallel implementation of the OpenAssembler algorithm. The paper describing OpenAssembler is still under review (submitted on 15 October 2009...), and one of its weaknesses is that it is not parallel, thus not scalable. So, I started coding Ray (started on 2010-01-21), and I decided to put it on the web to get feedbacks.
[Q] "OpenAssembler assembles Illumina reads or 454 + Illumina reads, or any combination without non-random error incorporation.". Can you explain what you mean by "random error incorporation" ?
When an error occurs, it should occur randomly. The 454 homopolymer errors are not randomly observed, they occur in homopolymer stretches more often. In the OpenAssembler paper (under review since 15 October 2009) we show however that Illumina's error incorporation is random, and that 454+Illumina also has random error incorporation. The take-home message is that randomly incorporated errors are easy to detect and fix, whereas reproducible errors are defective-by-design.
Illumina errors are distributed on all the read, with more observed errors at the end. 454 errors are mosty related to homopolymers, for instance you will observe both ATCTAGCAAAAATACGCAT and ATCTAGCAAAAAATACGCAT with the same abundance (notice the length of AAAAAs).
[Q] How critical are the values of "<fragmentLength>" and "<fragmentLengthStandardDeviation>" to the assembly? Are they just starting points for estimating the true value?
You should know the true values before running Ray. For instance, the SRA001125 dataset contains paired-end Illumina reads for E. coli K12 MG1655. Usually, if you have paired-end data, you should know the geometry (fragmentLength+deviation) of your reads.
an example of that:
[boiseb01@ls30 SRA001125]$ echo "LoadPairedEndReads 200xSRR001665_1.fastq 200xSRR001665_2.fastq 215 20
LoadPairedEndReads 200xSRR001666_1.fastq 200xSRR001666_2.fastq 215 20" > input
[boiseb01@ls30 SRA001125]$ /home/boiseb01/software/ompi-1.4.1-gcc/bin/mpirun -np 31 /home/boiseb01/Ray/trunk/Ray ./input |tee Log
[boiseb01@ls30 SRA001125]$ ls -l Contigs.fasta
-rw-rw-r--. 1 boiseb01 boiseb01 4710363 2010-03-09 17:01 Contigs.fasta
[boiseb01@ls30 SRA001125]$ grep '>' Contigs.fasta |wc -l
224
As such, we get 224 >=100-nt bits for this small bug.
If you provide paired-end reads, you need to provide accurate values for <fragmentLength> and <fragmentLengthStandardDeviation>.
[Q] Does "Ray" use the quality values in the FASTQ file for anything?
No, Ray auto-calibrates itself using abundance of k-mers.
[Q] What does "Ray" do if I provide it with really long sequences, such as contigs from another assembly?
Try it!, I don't know.
My benchmarks so far include:
* SRA001125 paired (E. coli k12 MG1655, Illumina data)
* S. pneumoniae R6 50-nt reads, 50 X
* S. pneumoniae R6 50-nt reads, 50 X, 1% random mismatches
* E. coli k12 MG1655, 400-nt reads, 50 X
* Human chromosome 1, 50-nt reads
* Pseudomonas aeruginosa, 50-nt reads, 50 X
[Q] Does it support colorspace?
Currently, only fasta, fastq, and SFF.
As I understand, there is a bijection between strings from {A,T,C,G} and reads from {0,1,2,3}, with each color corresponding to a nucleotide given the previous one. I have not look into that yet, but I don't think the algorithm is going to change a lots with that taken into consideration.
My guest is that one could do the assembly in color-space (the alphabet size is 4 too), and then convert the color-space contigs to nucleotide-space.
I hope it helps!
***
The Ray Project Team
Leave a comment:
-
The README.txt is confusing in some of the sections. I hope you can help me clarify them.Originally posted by seb567 View Post
[Q] It says "if your sff file contains paired-end reads, you must first extract the information, and tell Ray to use them with LoadPairedEndReads". Do you mean we should extract as FASTA with sffinfo, remove the linker, and create a .fasta/.fastq file?
[Q] Is "OpenAssembler" the same software as "Ray" ?
[Q] "OpenAssembler assembles Illumina reads or 454 + Illumina reads, or any combination without non-random error incorporation.". Can you explain what you mean by "random error incorporation" ?
[Q] How critical are the values of "<fragmentLength>" and "<fragmentLengthStandardDeviation>" to the assembly? Are they just starting points for estimating the true value?
[Q] Does "Ray" use the quality values in the FASTQ file for anything?
[Q] What does "Ray" do if I provide it with really long sequences, such as contigs from another assembly?
Thank you for your time,
TorstenLeave a comment:
-
Ray: a NEW MPI-based 100% parallel genome assembler
Dear SeqAnswers:
The Ray Project Team gives you a 100% parallel MPI-based assembler called Ray. Ray is NOW available at http://sourceforge.net/projects/denovoassembler/files/. It supports Illumina paired-end reads. It is 100% parallel, and it is a single executable (no pesky perl scripts!). The source code is licensed with the GPL-v3.
Try it, and give us your comments, bugs, suggestions, and concerns on our mailing list.
Ray-0.0.3: a NEW MPI-based parallel genome assembler
***
The Ray Project Team
-
Researchers using sequencing and genomics tools often have to make trade-offs. They can choose between speed or scale, short reads or long-range information, or targeted panels or a view of the whole transcriptome. New technologies that have been released this year are built to address those tough choices.
We asked six companies the same four questions to learn about their latest products. The new technologies bring a lot to the table, including rethinking sequencing...
Leave a comment: