Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Capturing bowtie stats / samtools pipe / file IO?

    Hi all,

    I've been working on tweaking our NGS analysis pipeline and came across this minor issue for which I could not find a solution.

    The current pipeline scripts do some I/O redirection (2>&1) to capture the stats from bowtie alignment (which are printed to stderr): This works fine if you simply issue a command to run bowtie.

    bowtie -q -S index fqsequence >> stats.txt 2>&1

    stats.txt
    # reads processed: 1679207
    # reads with at least one reported alignment: 1487157 (88.56%)
    # reads that failed to align: 155467 (9.26%)
    # reads with alignments suppressed due to -m: 36583 (2.18%)
    Reported 1487157 alignments to 1 output stream(s)

    As part of the upgrade, I want to pipe the bowtie output directly to SAMtools (instead of creating an intermediate SAM file) to produce the BAM file on the fly. No problem, this also works fine.

    bowtie -q -S index fqsequence | samtools view -bS -o testoutput.bam -

    [samopen] SAM header is present: 14543 sequences.
    # reads processed: 1679207
    # reads with at least one reported alignment: 1487157 (88.56%)
    # reads that failed to align: 155467 (9.26%)
    # reads with alignments suppressed due to -m: 36583 (2.18%)
    Reported 1487157 alignments to 1 output stream(s)

    I figured it would be easy to capture the stderr output from this pipe and send it to the stats file as before, but appending this command with >> stats.txt 2>&1 only shunts the SAMtools information to the file. The bowtie stats are still printed to the screen. I've played around with the i/o redirect commands and can't get it to work. Any suggestions?

    bowtie -q -S index fqsequence | samtools view -bS -o testoutput.bam - >> stats.txt 2>&1

    stats.txt
    [samopen] SAM header is present: 14543 sequences.

    screen output:
    # reads processed: 1679207
    # reads with at least one reported alignment: 1487157 (88.56%)
    # reads that failed to align: 155467 (9.26%)
    # reads with alignments suppressed due to -m: 36583 (2.18%)
    Reported 1487157 alignments to 1 output stream(s)

  • #2
    demo.pl
    Code:
    #! /usr/bin/perl
    print STDERR "error";
    print "out";
    Code:
    ./demo.pl 2> stderr.txt | cat > stdout.txt

    Comment


    • #3
      Thanks for the example... hadn't thought of piping the output to cat, but unfortunately, that syntax didn't work either.

      Comment


      • #4
        (bowtie genome --sam <(gunzip -c file.fastq.gz) | samtools view -bS -o results.bam - ) 2> bowtie.err

        This worked for me (with bash).

        There may also be some way to use tee here, but I don't know exactly.

        I know how to use tee to write the output (standard output) of aaa.sh to bbb.out, while still displaying it in the terminal: ./aaa.sh | tee bbb.out How would I now also write standard error to a f...
        Last edited by mgogol; 09-22-2011, 07:28 AM.

        Comment

        Latest Articles

        Collapse

        • seqadmin
          Quality Control Essentials for Next-Generation Sequencing Workflows
          by seqadmin




          Like all molecular biology applications, next-generation sequencing (NGS) workflows require diligent quality control (QC) measures to ensure accurate and reproducible results. Proper QC begins at nucleic acid extraction and continues all the way through to data analysis. This article outlines the key QC steps in an NGS workflow, along with the commonly used tools and techniques.

          Nucleic Acid Quality Control
          Preparing for NGS starts with isolating the...
          02-10-2025, 01:58 PM
        • seqadmin
          An Introduction to the Technologies Transforming Precision Medicine
          by seqadmin


          In recent years, precision medicine has become a major focus for researchers and healthcare professionals. This approach offers personalized treatment and wellness plans by utilizing insights from each person's unique biology and lifestyle to deliver more effective care. Its advancement relies on innovative technologies that enable a deeper understanding of individual variability. In a joint documentary with our colleagues at Biocompare, we examined the foundational principles of precision...
          01-27-2025, 07:46 AM

        ad_right_rmr

        Collapse

        News

        Collapse

        Topics Statistics Last Post
        Started by seqadmin, 02-07-2025, 09:30 AM
        0 responses
        68 views
        0 likes
        Last Post seqadmin  
        Started by seqadmin, 02-05-2025, 10:34 AM
        0 responses
        107 views
        0 likes
        Last Post seqadmin  
        Started by seqadmin, 02-03-2025, 09:07 AM
        0 responses
        83 views
        0 likes
        Last Post seqadmin  
        Started by seqadmin, 01-31-2025, 08:31 AM
        0 responses
        47 views
        0 likes
        Last Post seqadmin  
        Working...
        X