Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • alignment speed on workstation vs desktop

    Hi all,

    Recently our lab purchased a workstation with 64 GB RAM, xeon processor (which has 64 cores) running on 64bit linux. I am connected to it through a desktop (linux 64bit) which has 4GB RAM, i5 processor (4 cores). I previously used to work on this desktop and now that I am connected to workstation, I thought of comparing the alignment speed on both machines.

    Data I used is paired end, with each 1.3 GB.
    I am using bowtie2 to align. (reference genome is yeast)

    On workstation I had set threads (-p) argument to 58 cores, and on desktop I used 3 cores out of available 4.

    Alignment results are same on both machines but the time taken by them is drastically different.

    Desktop machine took 8:39 secs, whereas
    Workstation took 18:18 secs !!!!!!!!

    What might be the problem ??

    I am sorry if my question is not relevent to the forum.

    Thank you !

  • #2
    guess

    perhaps time costs for parallelisation (splitting data and collecting results) are greater than time savings for mapping.

    better use a few cores per data set and process different datasets simultaneously (e.g. using &).

    only a guess...

    Comment


    • #3
      Number of cores matters little if you can't efficiently feed them data (as you may have discovered).

      You should experiment with ramping up the number of cores starting with a smaller value on the workstation and checking to see where you finally encounter a system-level bottleneck (specially in terms of I/O or memory bandwidth). You may end up finding that using 24 (or a smaller/bigger number for that matter) of cores gives you the fastest execution times.

      You may be able to improve the I/O to some extent by using fast SSD drives (you did not say if you already have them) but other limits (memory bandwidth) would be properties of hardware you have in the workstation and can't be changed.

      Comment


      • #4
        8:39 you mean 8 mins 39 secs?

        I/O in this case shoudn't be the bottleneck, only a 1.3GB file, cost nothing to read.

        Someone familiar with how bowtie2 does the multithreading?

        If you like try with BWA ALN, report the number here.

        Best,

        dong

        Comment


        • #5
          @GenoMax :

          You were right.. I used different number of cores this is what I found (attachment). At -p 30, it took about only 1 min 30 seconds, after that it went on increasing. I think this is where bowtie2 encountered system-level bottleneck.. May be I will stick with this number for a while.. Thank you all for the suggestions..
          Attached Files

          Comment

          Latest Articles

          Collapse

          • seqadmin
            The Impact of AI in Genomic Medicine
            by seqadmin



            Artificial intelligence (AI) has evolved from a futuristic vision to a mainstream technology, highlighted by the introduction of tools like OpenAI's ChatGPT and Google's Gemini. In recent years, AI has become increasingly integrated into the field of genomics. This integration has enabled new scientific discoveries while simultaneously raising important ethical questions1. Interviews with two researchers at the center of this intersection provide insightful perspectives into...
            02-26-2024, 02:07 PM
          • seqadmin
            Multiomics Techniques Advancing Disease Research
            by seqadmin


            New and advanced multiomics tools and technologies have opened new avenues of research and markedly enhanced various disciplines such as disease research and precision medicine1. The practice of merging diverse data from various ‘omes increasingly provides a more holistic understanding of biological systems. As Maddison Masaeli, Co-Founder and CEO at Deepcell, aptly noted, “You can't explain biology in its complex form with one modality.”

            A major leap in the field has
            ...
            02-08-2024, 06:33 AM

          ad_right_rmr

          Collapse

          News

          Collapse

          Topics Statistics Last Post
          Started by seqadmin, 02-28-2024, 06:12 AM
          0 responses
          21 views
          0 likes
          Last Post seqadmin  
          Started by seqadmin, 02-23-2024, 04:11 PM
          0 responses
          70 views
          0 likes
          Last Post seqadmin  
          Started by seqadmin, 02-21-2024, 08:52 AM
          0 responses
          77 views
          0 likes
          Last Post seqadmin  
          Started by seqadmin, 02-20-2024, 08:57 AM
          0 responses
          67 views
          0 likes
          Last Post seqadmin  
          Working...
          X