Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Speeding up .sam to .vcf creation

    I'm trying to get .vcf files from a .sam file via the following commands:

    Code:
    samtools view -bt ${fai} ${prefix}.sam > ${prefix}_unsorted.bam
    samtools sort ${prefix}_unsorted.bam ${prefix} 
    samtools mpileup -ugf ${fasta} ${prefix}.bam | bcftools view -bvcg -p - > ${prefix}.raw.bcf
    The sorted .bam file is ~130GB, and the final command I'm running seems like it will take forever to complete. It's been running for a couple of days, and the .raw.bcf file is only 85MB so far.

    Is this normal? Is there an option in samtools/bcftools for multithreading, or am I doing something incorrectly?

    Thanks!

  • #2
    You can get some small improvements by
    1) piping your commands together (view | sort | mpileup | bcftools view > output.bcf)
    2) specifying -m (use memory) parameters on sort. Use as much memory as you can get away with.

    There are "multi-thread bam sorters" but they don't offer much real speed up.

    It's not unusual for it to take several days to process a bam file in this fashion.

    You probably have multiple cores on your computer. You can break down the bam|mpileup>bcf part into chunks (by chromosome?) and run them in parallel. example: samtools view x.bam chr22 | samtools mpileup |bcftools > x22.bcf
    Be sure to index the bam file if you choose this method.
    Last edited by Richard Finney; 10-02-2012, 05:05 PM.

    Comment


    • #3
      Is there a reason you have a .sam file?

      whatever is making your .sam file, you should pipe that into samtools view -bS and convert it to a .bam on the fly. .sam files are so huge, you should never have your whole data set as one. You should always pipe straight into samtools view, and make .sam file of subsets of your data, if you have to eyeball it.

      You can take a peek at your .bcf by running bcftools view on it to convert it to vcf as it's being made. (It might complain about the last line having not enough columns, but that doesn't matter) Just see if it makes sense. You'd hate to wait forever and find that something is wrong.

      I used to pipe view into sort, and then one day, it spontaneously stopped working. It was making no temp files at all, so I stopped doing it. But if you are only doing one .bam file, you can probably pipe that into mpileup too.

      Comment

      Latest Articles

      Collapse

      • seqadmin
        Latest Developments in Precision Medicine
        by seqadmin



        Technological advances have led to drastic improvements in the field of precision medicine, enabling more personalized approaches to treatment. This article explores four leading groups that are overcoming many of the challenges of genomic profiling and precision medicine through their innovative platforms and technologies.

        Somatic Genomics
        “We have such a tremendous amount of genetic diversity that exists within each of us, and not just between us as individuals,”...
        05-24-2024, 01:16 PM
      • seqadmin
        Recent Advances in Sequencing Analysis Tools
        by seqadmin


        The sequencing world is rapidly changing due to declining costs, enhanced accuracies, and the advent of newer, cutting-edge instruments. Equally important to these developments are improvements in sequencing analysis, a process that converts vast amounts of raw data into a comprehensible and meaningful form. This complex task requires expertise and the right analysis tools. In this article, we highlight the progress and innovation in sequencing analysis by reviewing several of the...
        05-06-2024, 07:48 AM

      ad_right_rmr

      Collapse

      News

      Collapse

      Topics Statistics Last Post
      Started by seqadmin, Today, 01:32 PM
      0 responses
      4 views
      0 likes
      Last Post seqadmin  
      Started by seqadmin, 05-24-2024, 07:15 AM
      0 responses
      198 views
      0 likes
      Last Post seqadmin  
      Started by seqadmin, 05-23-2024, 10:28 AM
      0 responses
      220 views
      0 likes
      Last Post seqadmin  
      Started by seqadmin, 05-23-2024, 07:35 AM
      0 responses
      229 views
      0 likes
      Last Post seqadmin  
      Working...
      X