Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #16
    Ondov, thanks for the notes!

    Comment


    • #17
      Hi ondovb,

      Many thanks for the notes! That was actually my concern.. My team had a discussion over the running time as we were not looking at Arabidopsis samples.

      Which brings me to something else I just thought of: What is the difference between running on multiple threads and multiple nodes? I currently put threads=1 while total nodes=10..

      On a side note, I realised that my processes that have been split into 10 nodes are all going into sleep mode.. Could this be because I didn't allocate enough RAM?

      Comment


      • #18
        Threads: should be the number of cores you want to use on each node. You mentioned you have 8 cores per node, so you'll want 8 threads to use them all.

        Running time: will be linear with respect to genome length. Our data took 480 cpu hours, so yours (assuming a similar # of reads) should take 480 * 30 = 14400 cpu hours. If you use all 40 * 8 cores on your cluster, you're looking at about 45 hours.

        Sleeping: if you remembered to include the -p flag, I'm not sure what else could cause this. Have you tried running it locally with the same settings and watching the output?

        Comment


        • #19
          Hi ondovb,

          Yes, Im running it locally but it seems to be stuck at the aligning stage:

          Round 1 / 4 (2101986 reads):

          Sensitivity 4:

          EDIT: I have used strace on the process and found it to be at the following state:

          futex(0x40dd79d0, FUTEX_WAIT, 25312, NULL
          Last edited by Haneko; 07-01-2010, 07:06 PM.

          Comment


          • #20
            I think each instance might appear to be sleeping to the OS because the parent thread just sits and waits for the child threads to finish their computation (even if only one thread is chosen). What does the CPU usage look like?

            Sensitivity 4 will take a pretty long time (even on your cluster), which could make it appear to be stuck. I wouldn't recommend going higher than 3. If you set the trim to at least 3, that should get rid of a lot of the errors and you should still be able to align a lot of reads.

            Comment


            • #21
              Originally posted by ondovb View Post
              Running time: will be linear with respect to genome length. Our data took 480 cpu hours, so yours (assuming a similar # of reads) should take 480 * 30 = 14400 cpu hours.
              can you estimate how the running time behaves in respect to # of reads?

              Comment


              • #22
                Running time is also approximately linear with respect to # of reads, and exponential with respect to sensitivity.

                Comment


                • #23
                  Originally posted by zee View Post
                  In fact I'm not aware of anybody who are doing bisulfite sequencing with SOLiD as yet.
                  I also also another unfortunate soul dealing with SOLiD bisulfite reads. I also know that Thomas Preiss' group in Sydney is working on RNA methylation using SOLiD.

                  Comment


                  • #24
                    I have aligned a subset of the reads on my machine and have some questions.

                    I received several warnings (e.g. '5719579 substrings of chr1.fa ignored due to 5718003 character(s) other than [ACGTacgt]'). The Ns in the reference file(s) cause this problem and I don't know the impact of the warnings on the overall analysis.

                    At the end of the aligning part is says 'computing error frequencies'. What does this mean?

                    Does SOCS-B run faster, if all reference files would be merged into one multiFASTA reference file?

                    I struggle to understand the difference between the mismatch sensitivity (s) and the tolerance (t). Could you briefly explain these two parameters? Can I set them independently?

                    Comment


                    • #25
                      Originally posted by fwessely View Post
                      I received several warnings (e.g. '5719579 substrings of chr1.fa ignored due to 5718003 character(s) other than [ACGTacgt]'). The Ns in the reference file(s) cause this problem and I don't know the impact of the warnings on the overall analysis.
                      These are just to keep you informed. If you were expecting that many Ns, you can ignore them. The only way they will affect your results is that you can expect coverage dips within a read's length of any Ns in the reference, since SOCS will not map to any substrings that contain an N.

                      Originally posted by fwessely View Post
                      At the end of the aligning part is says 'computing error frequencies'. What does this mean?
                      It is outputting the observed frequency of color-space errors for each position in the read length. The output should be in the stats folder.

                      Originally posted by fwessely View Post
                      Does SOCS-B run faster, if all reference files would be merged into one multiFASTA reference file?
                      The speed shouldn’t be affected by separate files. My only suggestion for efficiency in large genomes is to limit the number of ambiguous matches to keep (assuming you don't need all of them). Each read could map to thousands of places in the whole genome, which affects RAM estimation and can cause multiple "rounds" of alignment when you give it bigger chunks of reads.

                      Originally posted by fwessely View Post
                      I struggle to understand the difference between the mismatch sensitivity (s) and the tolerance (t). Could you briefly explain these two parameters? Can I set them independently?
                      I admit sensitivity and tolerance are confusing...here's an example: if the sensitivity (-s) is 3, you are guaranteed to find the best alignment in the genome with 3 or fewer color space mismatches (ignoring bisulfite changes). However, a lot of alignments will also be found by chance that have 4 or more mismatches. If a read only has alignments with 4 or more, you may or may not want to report the best one that was found, since it is not guaranteed to be the best in the whole genome. The threshold for reporting these is set by the tolerance (-t), and this should always be at least as high as sensitivity.

                      Comment

                      Latest Articles

                      Collapse

                      • seqadmin
                        Best Practices for Single-Cell Sequencing Analysis
                        by seqadmin



                        While isolating and preparing single cells for sequencing was historically the bottleneck, recent technological advancements have shifted the challenge to data analysis. This highlights the rapidly evolving nature of single-cell sequencing. The inherent complexity of single-cell analysis has intensified with the surge in data volume and the incorporation of diverse and more complex datasets. This article explores the challenges in analysis, examines common pitfalls, offers...
                        06-06-2024, 07:15 AM
                      • seqadmin
                        Latest Developments in Precision Medicine
                        by seqadmin



                        Technological advances have led to drastic improvements in the field of precision medicine, enabling more personalized approaches to treatment. This article explores four leading groups that are overcoming many of the challenges of genomic profiling and precision medicine through their innovative platforms and technologies.

                        Somatic Genomics
                        “We have such a tremendous amount of genetic diversity that exists within each of us, and not just between us as individuals,”...
                        05-24-2024, 01:16 PM

                      ad_right_rmr

                      Collapse

                      News

                      Collapse

                      Topics Statistics Last Post
                      Started by seqadmin, Today, 07:23 AM
                      0 responses
                      8 views
                      0 likes
                      Last Post seqadmin  
                      Started by seqadmin, 06-17-2024, 06:54 AM
                      0 responses
                      11 views
                      0 likes
                      Last Post seqadmin  
                      Started by seqadmin, 06-14-2024, 07:24 AM
                      0 responses
                      24 views
                      0 likes
                      Last Post seqadmin  
                      Started by seqadmin, 06-13-2024, 08:58 AM
                      0 responses
                      17 views
                      0 likes
                      Last Post seqadmin  
                      Working...
                      X