[jinghanna]

Ondov, thanks for the notes!

[Haneko]

Hi ondovb,

Many thanks for the notes! That was actually my concern.. My team had a discussion over the running time as we were not looking at Arabidopsis samples.

Which brings me to something else I just thought of: What is the difference between running on multiple threads and multiple nodes? I currently put threads=1 while total nodes=10..

On a side note, I realised that my processes that have been split into 10 nodes are all going into sleep mode.. Could this be because I didn't allocate enough RAM?

[ondovb]

Threads: should be the number of cores you want to use on each node. You mentioned you have 8 cores per node, so you'll want 8 threads to use them all.

Running time: will be linear with respect to genome length. Our data took 480 cpu hours, so yours (assuming a similar # of reads) should take 480 * 30 = 14400 cpu hours. If you use all 40 * 8 cores on your cluster, you're looking at about 45 hours.

Sleeping: if you remembered to include the -p flag, I'm not sure what else could cause this. Have you tried running it locally with the same settings and watching the output?

[Haneko]

Hi ondovb,

Yes, Im running it locally but it seems to be stuck at the aligning stage:

Round 1 / 4 (2101986 reads):

Sensitivity 4:

EDIT: I have used strace on the process and found it to be at the following state:

futex(0x40dd79d0, FUTEX_WAIT, 25312, NULL

[ondovb]

I think each instance might appear to be sleeping to the OS because the parent thread just sits and waits for the child threads to finish their computation (even if only one thread is chosen). What does the CPU usage look like?

Sensitivity 4 will take a pretty long time (even on your cluster), which could make it appear to be stuck. I wouldn't recommend going higher than 3. If you set the trim to at least 3, that should get rid of a lot of the errors and you should still be able to align a lot of reads.

[volks]

Running time: will be linear with respect to genome length. Our data took 480 cpu hours, so yours (assuming a similar # of reads) should take 480 * 30 = 14400 cpu hours.

can you estimate how the running time behaves in respect to # of reads?

[ondovb]

Running time is also approximately linear with respect to # of reads, and exponential with respect to sensitivity.

[sci_guy]

In fact I'm not aware of anybody who are doing bisulfite sequencing with SOLiD as yet.

I also also another unfortunate soul dealing with SOLiD bisulfite reads. I also know that Thomas Preiss' group in Sydney is working on RNA methylation using SOLiD.

[fwessely]

I have aligned a subset of the reads on my machine and have some questions.

I received several warnings (e.g. '5719579 substrings of chr1.fa ignored due to 5718003 character(s) other than [ACGTacgt]'). The Ns in the reference file(s) cause this problem and I don't know the impact of the warnings on the overall analysis.

At the end of the aligning part is says 'computing error frequencies'. What does this mean?

Does SOCS-B run faster, if all reference files would be merged into one multiFASTA reference file?

I struggle to understand the difference between the mismatch sensitivity (s) and the tolerance (t). Could you briefly explain these two parameters? Can I set them independently?

[ondovb]

I received several warnings (e.g. '5719579 substrings of chr1.fa ignored due to 5718003 character(s) other than [ACGTacgt]'). The Ns in the reference file(s) cause this problem and I don't know the impact of the warnings on the overall analysis.

These are just to keep you informed. If you were expecting that many Ns, you can ignore them. The only way they will affect your results is that you can expect coverage dips within a read's length of any Ns in the reference, since SOCS will not map to any substrings that contain an N.

At the end of the aligning part is says 'computing error frequencies'. What does this mean?

It is outputting the observed frequency of color-space errors for each position in the read length. The output should be in the stats folder.

Does SOCS-B run faster, if all reference files would be merged into one multiFASTA reference file?

The speed shouldn’t be affected by separate files. My only suggestion for efficiency in large genomes is to limit the number of ambiguous matches to keep (assuming you don't need all of them). Each read could map to thousands of places in the whole genome, which affects RAM estimation and can cause multiple "rounds" of alignment when you give it bigger chunks of reads.

I struggle to understand the difference between the mismatch sensitivity (s) and the tolerance (t). Could you briefly explain these two parameters? Can I set them independently?

I admit sensitivity and tolerance are confusing...here's an example: if the sensitivity (-s) is 3, you are guaranteed to find the best alignment in the genome with 3 or fewer color space mismatches (ignoring bisulfite changes). However, a lot of alignments will also be found by chance that have 4 or more mismatches. If a read only has alignments with 4 or more, you may or may not want to report the best one that was found, since it is not guaranteed to be the best in the whole genome. The threshold for reporting these is set by the tolerance (-t), and this should always be at least as high as sensitivity.