Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • yet another framework?

    Anyone who follows the web technologies is very familiar with the # of web frameworks there are, and the ever increasing number of them.

    I was pretty excited when I saw the announcement, but was quickly disappointed to find out this doesn't do much, if anything, beyond what Picard and GATK already provide.

    What are the advantages of using this framework over using Picard and GATK with BioJava? Development time will be faster in that scenario, there is more documentation, and they have a more liberal license (for now) for commercial users.

  • #2
    This is a good point. How does openge make it easier us to run good genomics workflows. Perhaps an easyrun function for exome, RNASeq,etc, analysis will be a great thing to have in this system.

    desaila, I think the current version of openge will provide some enhanced functions for doing thing with alignments, etc, since it is written in C.

    Anybody else care to comment?

    Comment


    • #3
      @desaila, i'd like to chime in on your thread here, which is a good distinguishment question-

      While there is overlap in the ultimate goals of Picard/GATK and the openge framework, as zee mentioned there are great differences in functionality. In terms of HPC, the openge binaries that were released are much more suited for parallelized performance (both CPU and GPU) in all of its available tools, and can also stream data similar to samtools and many our other favourite programs. Not mention that as a C program, the hassle of matching java versions is gone (for example, between your cluster nodes and your workspace), and can be opened on all 'nix distros with minimal worry.

      Aside from performance and command line usability, there is also the workflow creation aspect of openge. Although still in development, an example of its use would be that a user could potentially put together a workflow for a specific type of genomic data that he has, and share it via some kind of devblog or SEQanswers thread with others who may need the exact same thing - saving this subset of analysts many hours. The openge 'wrapped' workflow would then be able to execute on these different machines without trouble. Not to mention advantages of using the openge wrappers, such as automated error handling, and other features of an appealing workflow manager.

      Hopefully this can clear up your separation of the Picard/GATK and openge. You are also correct that there is more documentation, etc. but openge should be given its chance to grow as well It is a concept that has its own HPC functionality, but also a big potential for carrying a community.

      Comment


      • #4
        @oiiio
        What I don't understand, in your reply is how is OpenGE differentiating itself in it's parallelization model? GATK uses map/reduce and scales with the # of cpu's you have. It looks like this does the same thing by actually porting that functionality from GATK.

        Let me clarify and say that I'm not trying to be pessimistic, and that I'm completely in support of this framework -- especially if they open up the commercial side of their license. However, it realistically does nothing that GATK and Picard don't already do. You can create workflows and things in GATK/Picard too.

        Here's where I'm optimistic: the fact that Nvidia is touting this as a compute solution is implying that there will be cuda/openCL elements in the future and _that_ is certainly something GATK will not have.

        Comment


        • #5
          Originally posted by desaila View Post
          @oiiio
          What I don't understand, in your reply is how is OpenGE differentiating itself in it's parallelization model? GATK uses map/reduce and scales with the # of cpu's you have. It looks like this does the same thing by actually porting that functionality from GATK.
          I would encourage you to take a look at the code. It appears you have posted only twice to SEQanswers prior to posting this thread. So, I have no way to gauge your technical level but I think you are misunderstanding the implementation. OpenGE implements multithreading that takes advantage of newer CPUs (and GPUs) that have lots of cores. This is not something that is ported from GATK and it is not map/reduce.

          The comparisons to GATK and other tools are inevitable. It's an easy question to ask. We have no interest in replacing or competing with GATK or other tools though. We are looking to complement existing tools, but our primary focus or interest is:

          (1) High performance code that maximizes compute power than be utilized on commodity hardware, compute nodes, etc.

          (2) A workflow to integrate OpenGE with other tools (and vice versa). Here I can't say too much more (because I do not want to sell you on tales of the future) but the idea is to build graphical and sharable analysis pipelines. And I do not mean scripts. I mean fully managed workflows that track progress, detect and correct errors, etc.

          I again encourage you to check out the code, follow the forum and give the code a chance if you want. Furthermore, we would welcome your thoughts and everyone else's on how to decide upon and prioritize future features.

          Comment


          • #6
            Originally posted by genericforms View Post
            I would encourage you to take a look at the code. It appears you have posted only twice to SEQanswers prior to posting this thread. So, I have no way to gauge your technical level but I think you are misunderstanding the implementation. OpenGE implements multithreading that takes advantage of newer CPUs (and GPUs) that have lots of cores. This is not something that is ported from GATK and it is not map/reduce.
            https://github.com/adaptivegenome/op...thm_module.cpp and the .h

            Correct me if I'm wrong, but inheriting classes of AlgorithmModule are simply asynchronous threads, or they might use a ThreadPool -- but browsing through the code it seems like the only thing that uses the ThreadPools is the read_sorter.{h, cpp}

            So, asynchronous threads on a single box is the breakthrough here? If that's the case, GATK scales to the same levels as OpenGE does right now. Clearly that will change once this starts using CUDA.

            Comment


            • #7
              Originally posted by desaila View Post
              So, asynchronous threads on a single box is the breakthrough here? If that's the case, GATK scales to the same levels as OpenGE does right now. Clearly that will change once this starts using CUDA.
              I hope that future releases impress you more

              Comment

              Latest Articles

              Collapse

              • seqadmin
                Techniques and Challenges in Conservation Genomics
                by seqadmin



                The field of conservation genomics centers on applying genomics technologies in support of conservation efforts and the preservation of biodiversity. This article features interviews with two researchers who showcase their innovative work and highlight the current state and future of conservation genomics.

                Avian Conservation
                Matthew DeSaix, a recent doctoral graduate from Kristen Ruegg’s lab at The University of Colorado, shared that most of his research...
                03-08-2024, 10:41 AM
              • seqadmin
                The Impact of AI in Genomic Medicine
                by seqadmin



                Artificial intelligence (AI) has evolved from a futuristic vision to a mainstream technology, highlighted by the introduction of tools like OpenAI's ChatGPT and Google's Gemini. In recent years, AI has become increasingly integrated into the field of genomics. This integration has enabled new scientific discoveries while simultaneously raising important ethical questions1. Interviews with two researchers at the center of this intersection provide insightful perspectives into...
                02-26-2024, 02:07 PM

              ad_right_rmr

              Collapse

              News

              Collapse

              Topics Statistics Last Post
              Started by seqadmin, 03-14-2024, 06:13 AM
              0 responses
              32 views
              0 likes
              Last Post seqadmin  
              Started by seqadmin, 03-08-2024, 08:03 AM
              0 responses
              72 views
              0 likes
              Last Post seqadmin  
              Started by seqadmin, 03-07-2024, 08:13 AM
              0 responses
              80 views
              0 likes
              Last Post seqadmin  
              Started by seqadmin, 03-06-2024, 09:51 AM
              0 responses
              68 views
              0 likes
              Last Post seqadmin  
              Working...
              X