Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Parallel Bioinformatics tools

    Can we list all the available ( open source and commercial) parallel (written using MPI / Map Reduce etc) bioinformatics tools. It can include tools that cover both secondary and tertiary analysis i,e from Assembly to Annotation to SNP discover etc. Thank you.

  • #2
    Here is a read mapper using MapReduce/Hadoop:

    Schatz. CloudBurst: highly sensitive read mapping with MapReduce. Bioinformatics (2009) vol. 25 (11) pp. 1363-9

    Comment


    • #3
      Originally posted by geschickten View Post
      Can we list all the available ( open source and commercial) parallel (written using MPI / Map Reduce etc) bioinformatics tools. It can include tools that cover both secondary and tertiary analysis i,e from Assembly to Annotation to SNP discover etc. Thank you.
      Where do I start with this Map Reduce fad and is it as bad as Web 2.0, which is as exciting as my grandmother finding out about the "any key" concept?

      Map Reduce is a simple model that has been implicit in the design and workflow of parallel processes some time. Most short read aligners can be used in a Map Reduce model, and should be included. For example, you split the reads in to N chunks (usually while converting the Illumina/Solid/ect. input files to FASTQ), you run for each chunk in parallel using an instance of the mapping tool (typically implemented in a multithreaded fashion to take advantage of shared memory), and then merge the results (using maq merge or samtools merge etc.). This is clearly obeying the map reduce model. In fact, programs like BFAST actually come with pipeline scripts to operate in such a model using SGE/PBS schedulers.

      Specific computer clusters like hadoop have APIs (say in Java) that help with the map reduce model, but is lacking high performance computing since most code is written in C.

      My 1.5 cents (recession).

      Comment


      • #4
        Yes the MapReduce model in and of itself is nothing new. But the availability of cloud services like AWS EC2 and software like Hadoop have made it that much more accessible to those who aren't C gurus and don't want to deal with the likes of sockets, locks, and other hassles of multithreaded processing. Hadoop can work on much less expensive hardware and can scale a lot more easily (as far as the programmer/end user is concerned). And in the end, if tuned correctly (Hadoop MR is much easier to tune than C code), the ability to scale will outweigh the supposed performance penalty of Java vs. C (which, I might add is a dinosaur of a concept and doesn't hold as true anymore).

        Comment


        • #5
          Have you never heard of Hadoop streaming? Of course Hadoop can interact with programs written in other language including C, Perl, etc.

          Comment


          • #6
            BTW, the SEQAnswers software wiki has categories for both MapReduce & Hadoop, so a good way to find tools is to browse there

            Comment

            Latest Articles

            Collapse

            • seqadmin
              Best Practices for Single-Cell Sequencing Analysis
              by seqadmin



              While isolating and preparing single cells for sequencing was historically the bottleneck, recent technological advancements have shifted the challenge to data analysis. This highlights the rapidly evolving nature of single-cell sequencing. The inherent complexity of single-cell analysis has intensified with the surge in data volume and the incorporation of diverse and more complex datasets. This article explores the challenges in analysis, examines common pitfalls, offers...
              06-06-2024, 07:15 AM
            • seqadmin
              Latest Developments in Precision Medicine
              by seqadmin



              Technological advances have led to drastic improvements in the field of precision medicine, enabling more personalized approaches to treatment. This article explores four leading groups that are overcoming many of the challenges of genomic profiling and precision medicine through their innovative platforms and technologies.

              Somatic Genomics
              “We have such a tremendous amount of genetic diversity that exists within each of us, and not just between us as individuals,”...
              05-24-2024, 01:16 PM

            ad_right_rmr

            Collapse

            News

            Collapse

            Topics Statistics Last Post
            Started by seqadmin, Today, 07:23 AM
            0 responses
            8 views
            0 likes
            Last Post seqadmin  
            Started by seqadmin, 06-17-2024, 06:54 AM
            0 responses
            11 views
            0 likes
            Last Post seqadmin  
            Started by seqadmin, 06-14-2024, 07:24 AM
            0 responses
            24 views
            0 likes
            Last Post seqadmin  
            Started by seqadmin, 06-13-2024, 08:58 AM
            0 responses
            17 views
            0 likes
            Last Post seqadmin  
            Working...
            X