Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • seq_crumbs memory issues

    Hey all,

    I've been loving the seq_crumbs package for trimming and filtering my Illumina data, and was excited to download the latest update that can handle paired end data. Unfortunately, when I tried to use the tools I noticed that the memory requirements had increased dramatically. I expected a modest increase in processing paired end data compared to single end files, but it was jumping from 8G for single end to 80G for pe. Has anyone else encountered this issue or am I doing something totally wrong?

    Also, does anyone else know of tools for quality filtering and complexity filtering that maintain read pairs? I'm aware of trimmomatic and trim_galore for quality trimming, but I didn't think they do filtering based on the quality/complexity of the read overall.

    Thanks!

  • #2
    Originally posted by DrWorm View Post
    Hey all,

    I've been loving the seq_crumbs package for trimming and filtering my Illumina data, and was excited to download the latest update that can handle paired end data. Unfortunately, when I tried to use the tools I noticed that the memory requirements had increased dramatically. I expected a modest increase in processing paired end data compared to single end files, but it was jumping from 8G for single end to 80G for pe. Has anyone else encountered this issue or am I doing something totally wrong?

    Also, does anyone else know of tools for quality filtering and complexity filtering that maintain read pairs? I'm aware of trimmomatic and trim_galore for quality trimming, but I didn't think they do filtering based on the quality/complexity of the read overall.

    Thanks!
    Hi DrWorm,

    I am one of the developers of the seq_crumbs package. We're interested in the memory hogging issue.
    Versions prior to 0.1.8 had memory related bug that we solved in the 0.1.8 version.

    seq_crumb uses python's standar library for multiprocessing and seems that versions of python <=2.7.3 are affected by a bug related with memory consumption.

    With python 2.7.4 in the latest ubuntu the memory use stays stable but with latest debian and python 2.7.3 memory use grows untill the process is finished.

    Which version of python are you using?
    p.

    Comment


    • #3
      I believe we're using Python 2.6.5. I'll request an update and see if that makes a difference.

      Thanks for the advice!!

      Comment

      Latest Articles

      Collapse

      • seqadmin
        Current Approaches to Protein Sequencing
        by seqadmin


        Proteins are often described as the workhorses of the cell, and identifying their sequences is key to understanding their role in biological processes and disease. Currently, the most common technique used to determine protein sequences is mass spectrometry. While still a valuable tool, mass spectrometry faces several limitations and requires a highly experienced scientist familiar with the equipment to operate it. Additionally, other proteomic methods, like affinity assays, are constrained...
        04-04-2024, 04:25 PM
      • seqadmin
        Strategies for Sequencing Challenging Samples
        by seqadmin


        Despite advancements in sequencing platforms and related sample preparation technologies, certain sample types continue to present significant challenges that can compromise sequencing results. Pedro Echave, Senior Manager of the Global Business Segment at Revvity, explained that the success of a sequencing experiment ultimately depends on the amount and integrity of the nucleic acid template (RNA or DNA) obtained from a sample. “The better the quality of the nucleic acid isolated...
        03-22-2024, 06:39 AM

      ad_right_rmr

      Collapse

      News

      Collapse

      Topics Statistics Last Post
      Started by seqadmin, 04-11-2024, 12:08 PM
      0 responses
      14 views
      0 likes
      Last Post seqadmin  
      Started by seqadmin, 04-10-2024, 10:19 PM
      0 responses
      21 views
      0 likes
      Last Post seqadmin  
      Started by seqadmin, 04-10-2024, 09:21 AM
      0 responses
      16 views
      0 likes
      Last Post seqadmin  
      Started by seqadmin, 04-04-2024, 09:00 AM
      0 responses
      44 views
      0 likes
      Last Post seqadmin  
      Working...
      X