Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • seq_crumbs memory issues

    Hey all,

    I've been loving the seq_crumbs package for trimming and filtering my Illumina data, and was excited to download the latest update that can handle paired end data. Unfortunately, when I tried to use the tools I noticed that the memory requirements had increased dramatically. I expected a modest increase in processing paired end data compared to single end files, but it was jumping from 8G for single end to 80G for pe. Has anyone else encountered this issue or am I doing something totally wrong?

    Also, does anyone else know of tools for quality filtering and complexity filtering that maintain read pairs? I'm aware of trimmomatic and trim_galore for quality trimming, but I didn't think they do filtering based on the quality/complexity of the read overall.

    Thanks!

  • #2
    Originally posted by DrWorm View Post
    Hey all,

    I've been loving the seq_crumbs package for trimming and filtering my Illumina data, and was excited to download the latest update that can handle paired end data. Unfortunately, when I tried to use the tools I noticed that the memory requirements had increased dramatically. I expected a modest increase in processing paired end data compared to single end files, but it was jumping from 8G for single end to 80G for pe. Has anyone else encountered this issue or am I doing something totally wrong?

    Also, does anyone else know of tools for quality filtering and complexity filtering that maintain read pairs? I'm aware of trimmomatic and trim_galore for quality trimming, but I didn't think they do filtering based on the quality/complexity of the read overall.

    Thanks!
    Hi DrWorm,

    I am one of the developers of the seq_crumbs package. We're interested in the memory hogging issue.
    Versions prior to 0.1.8 had memory related bug that we solved in the 0.1.8 version.

    seq_crumb uses python's standar library for multiprocessing and seems that versions of python <=2.7.3 are affected by a bug related with memory consumption.

    With python 2.7.4 in the latest ubuntu the memory use stays stable but with latest debian and python 2.7.3 memory use grows untill the process is finished.

    Which version of python are you using?
    p.

    Comment


    • #3
      I believe we're using Python 2.6.5. I'll request an update and see if that makes a difference.

      Thanks for the advice!!

      Comment

      Latest Articles

      Collapse

      • seqadmin
        Strategies for Sequencing Challenging Samples
        by seqadmin


        Despite advancements in sequencing platforms and related sample preparation technologies, certain sample types continue to present significant challenges that can compromise sequencing results. Pedro Echave, Senior Manager of the Global Business Segment at Revvity, explained that the success of a sequencing experiment ultimately depends on the amount and integrity of the nucleic acid template (RNA or DNA) obtained from a sample. “The better the quality of the nucleic acid isolated...
        03-22-2024, 06:39 AM
      • seqadmin
        Techniques and Challenges in Conservation Genomics
        by seqadmin



        The field of conservation genomics centers on applying genomics technologies in support of conservation efforts and the preservation of biodiversity. This article features interviews with two researchers who showcase their innovative work and highlight the current state and future of conservation genomics.

        Avian Conservation
        Matthew DeSaix, a recent doctoral graduate from Kristen Ruegg’s lab at The University of Colorado, shared that most of his research...
        03-08-2024, 10:41 AM

      ad_right_rmr

      Collapse

      News

      Collapse

      Topics Statistics Last Post
      Started by seqadmin, 03-27-2024, 06:37 PM
      0 responses
      12 views
      0 likes
      Last Post seqadmin  
      Started by seqadmin, 03-27-2024, 06:07 PM
      0 responses
      11 views
      0 likes
      Last Post seqadmin  
      Started by seqadmin, 03-22-2024, 10:03 AM
      0 responses
      53 views
      0 likes
      Last Post seqadmin  
      Started by seqadmin, 03-21-2024, 07:32 AM
      0 responses
      69 views
      0 likes
      Last Post seqadmin  
      Working...
      X