Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Speeding up .sam to .vcf creation

    I'm trying to get .vcf files from a .sam file via the following commands:

    Code:
    samtools view -bt ${fai} ${prefix}.sam > ${prefix}_unsorted.bam
    samtools sort ${prefix}_unsorted.bam ${prefix} 
    samtools mpileup -ugf ${fasta} ${prefix}.bam | bcftools view -bvcg -p - > ${prefix}.raw.bcf
    The sorted .bam file is ~130GB, and the final command I'm running seems like it will take forever to complete. It's been running for a couple of days, and the .raw.bcf file is only 85MB so far.

    Is this normal? Is there an option in samtools/bcftools for multithreading, or am I doing something incorrectly?

    Thanks!

  • #2
    You can get some small improvements by
    1) piping your commands together (view | sort | mpileup | bcftools view > output.bcf)
    2) specifying -m (use memory) parameters on sort. Use as much memory as you can get away with.

    There are "multi-thread bam sorters" but they don't offer much real speed up.

    It's not unusual for it to take several days to process a bam file in this fashion.

    You probably have multiple cores on your computer. You can break down the bam|mpileup>bcf part into chunks (by chromosome?) and run them in parallel. example: samtools view x.bam chr22 | samtools mpileup |bcftools > x22.bcf
    Be sure to index the bam file if you choose this method.
    Last edited by Richard Finney; 10-02-2012, 05:05 PM.

    Comment


    • #3
      Is there a reason you have a .sam file?

      whatever is making your .sam file, you should pipe that into samtools view -bS and convert it to a .bam on the fly. .sam files are so huge, you should never have your whole data set as one. You should always pipe straight into samtools view, and make .sam file of subsets of your data, if you have to eyeball it.

      You can take a peek at your .bcf by running bcftools view on it to convert it to vcf as it's being made. (It might complain about the last line having not enough columns, but that doesn't matter) Just see if it makes sense. You'd hate to wait forever and find that something is wrong.

      I used to pipe view into sort, and then one day, it spontaneously stopped working. It was making no temp files at all, so I stopped doing it. But if you are only doing one .bam file, you can probably pipe that into mpileup too.

      Comment

      Latest Articles

      Collapse

      • seqadmin
        Recent Advances in Sequencing Analysis Tools
        by seqadmin


        The sequencing world is rapidly changing due to declining costs, enhanced accuracies, and the advent of newer, cutting-edge instruments. Equally important to these developments are improvements in sequencing analysis, a process that converts vast amounts of raw data into a comprehensible and meaningful form. This complex task requires expertise and the right analysis tools. In this article, we highlight the progress and innovation in sequencing analysis by reviewing several of the...
        05-06-2024, 07:48 AM
      • seqadmin
        Essential Discoveries and Tools in Epitranscriptomics
        by seqadmin




        The field of epigenetics has traditionally concentrated more on DNA and how changes like methylation and phosphorylation of histones impact gene expression and regulation. However, our increased understanding of RNA modifications and their importance in cellular processes has led to a rise in epitranscriptomics research. “Epitranscriptomics brings together the concepts of epigenetics and gene expression,” explained Adrien Leger, PhD, Principal Research Scientist...
        04-22-2024, 07:01 AM

      ad_right_rmr

      Collapse

      News

      Collapse

      Topics Statistics Last Post
      Started by seqadmin, 05-14-2024, 07:03 AM
      0 responses
      20 views
      0 likes
      Last Post seqadmin  
      Started by seqadmin, 05-10-2024, 06:35 AM
      0 responses
      44 views
      0 likes
      Last Post seqadmin  
      Started by seqadmin, 05-09-2024, 02:46 PM
      0 responses
      54 views
      0 likes
      Last Post seqadmin  
      Started by seqadmin, 05-07-2024, 06:57 AM
      0 responses
      42 views
      0 likes
      Last Post seqadmin  
      Working...
      X