Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • swbarnes2
    replied
    Is there a reason you have a .sam file?

    whatever is making your .sam file, you should pipe that into samtools view -bS and convert it to a .bam on the fly. .sam files are so huge, you should never have your whole data set as one. You should always pipe straight into samtools view, and make .sam file of subsets of your data, if you have to eyeball it.

    You can take a peek at your .bcf by running bcftools view on it to convert it to vcf as it's being made. (It might complain about the last line having not enough columns, but that doesn't matter) Just see if it makes sense. You'd hate to wait forever and find that something is wrong.

    I used to pipe view into sort, and then one day, it spontaneously stopped working. It was making no temp files at all, so I stopped doing it. But if you are only doing one .bam file, you can probably pipe that into mpileup too.

    Leave a comment:


  • Richard Finney
    replied
    You can get some small improvements by
    1) piping your commands together (view | sort | mpileup | bcftools view > output.bcf)
    2) specifying -m (use memory) parameters on sort. Use as much memory as you can get away with.

    There are "multi-thread bam sorters" but they don't offer much real speed up.

    It's not unusual for it to take several days to process a bam file in this fashion.

    You probably have multiple cores on your computer. You can break down the bam|mpileup>bcf part into chunks (by chromosome?) and run them in parallel. example: samtools view x.bam chr22 | samtools mpileup |bcftools > x22.bcf
    Be sure to index the bam file if you choose this method.
    Last edited by Richard Finney; 10-02-2012, 05:05 PM.

    Leave a comment:


  • falarkys
    started a topic Speeding up .sam to .vcf creation

    Speeding up .sam to .vcf creation

    I'm trying to get .vcf files from a .sam file via the following commands:

    Code:
    samtools view -bt ${fai} ${prefix}.sam > ${prefix}_unsorted.bam
    samtools sort ${prefix}_unsorted.bam ${prefix} 
    samtools mpileup -ugf ${fasta} ${prefix}.bam | bcftools view -bvcg -p - > ${prefix}.raw.bcf
    The sorted .bam file is ~130GB, and the final command I'm running seems like it will take forever to complete. It's been running for a couple of days, and the .raw.bcf file is only 85MB so far.

    Is this normal? Is there an option in samtools/bcftools for multithreading, or am I doing something incorrectly?

    Thanks!

Latest Articles

Collapse

  • seqadmin
    Best Practices for Single-Cell Sequencing Analysis
    by seqadmin



    While isolating and preparing single cells for sequencing was historically the bottleneck, recent technological advancements have shifted the challenge to data analysis. This highlights the rapidly evolving nature of single-cell sequencing. The inherent complexity of single-cell analysis has intensified with the surge in data volume and the incorporation of diverse and more complex datasets. This article explores the challenges in analysis, examines common pitfalls, offers...
    06-06-2024, 07:15 AM
  • seqadmin
    Latest Developments in Precision Medicine
    by seqadmin



    Technological advances have led to drastic improvements in the field of precision medicine, enabling more personalized approaches to treatment. This article explores four leading groups that are overcoming many of the challenges of genomic profiling and precision medicine through their innovative platforms and technologies.

    Somatic Genomics
    “We have such a tremendous amount of genetic diversity that exists within each of us, and not just between us as individuals,”...
    05-24-2024, 01:16 PM

ad_right_rmr

Collapse

News

Collapse

Topics Statistics Last Post
Started by seqadmin, Yesterday, 07:24 AM
0 responses
11 views
0 likes
Last Post seqadmin  
Started by seqadmin, 06-13-2024, 08:58 AM
0 responses
11 views
0 likes
Last Post seqadmin  
Started by seqadmin, 06-12-2024, 02:20 PM
0 responses
16 views
0 likes
Last Post seqadmin  
Started by seqadmin, 06-07-2024, 06:58 AM
0 responses
184 views
0 likes
Last Post seqadmin  
Working...
X