Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Capturing bowtie stats / samtools pipe / file IO?

    Hi all,

    I've been working on tweaking our NGS analysis pipeline and came across this minor issue for which I could not find a solution.

    The current pipeline scripts do some I/O redirection (2>&1) to capture the stats from bowtie alignment (which are printed to stderr): This works fine if you simply issue a command to run bowtie.

    bowtie -q -S index fqsequence >> stats.txt 2>&1

    stats.txt
    # reads processed: 1679207
    # reads with at least one reported alignment: 1487157 (88.56%)
    # reads that failed to align: 155467 (9.26%)
    # reads with alignments suppressed due to -m: 36583 (2.18%)
    Reported 1487157 alignments to 1 output stream(s)

    As part of the upgrade, I want to pipe the bowtie output directly to SAMtools (instead of creating an intermediate SAM file) to produce the BAM file on the fly. No problem, this also works fine.

    bowtie -q -S index fqsequence | samtools view -bS -o testoutput.bam -

    [samopen] SAM header is present: 14543 sequences.
    # reads processed: 1679207
    # reads with at least one reported alignment: 1487157 (88.56%)
    # reads that failed to align: 155467 (9.26%)
    # reads with alignments suppressed due to -m: 36583 (2.18%)
    Reported 1487157 alignments to 1 output stream(s)

    I figured it would be easy to capture the stderr output from this pipe and send it to the stats file as before, but appending this command with >> stats.txt 2>&1 only shunts the SAMtools information to the file. The bowtie stats are still printed to the screen. I've played around with the i/o redirect commands and can't get it to work. Any suggestions?

    bowtie -q -S index fqsequence | samtools view -bS -o testoutput.bam - >> stats.txt 2>&1

    stats.txt
    [samopen] SAM header is present: 14543 sequences.

    screen output:
    # reads processed: 1679207
    # reads with at least one reported alignment: 1487157 (88.56%)
    # reads that failed to align: 155467 (9.26%)
    # reads with alignments suppressed due to -m: 36583 (2.18%)
    Reported 1487157 alignments to 1 output stream(s)

  • #2
    demo.pl
    Code:
    #! /usr/bin/perl
    print STDERR "error";
    print "out";
    Code:
    ./demo.pl 2> stderr.txt | cat > stdout.txt

    Comment


    • #3
      Thanks for the example... hadn't thought of piping the output to cat, but unfortunately, that syntax didn't work either.

      Comment


      • #4
        (bowtie genome --sam <(gunzip -c file.fastq.gz) | samtools view -bS -o results.bam - ) 2> bowtie.err

        This worked for me (with bash).

        There may also be some way to use tee here, but I don't know exactly.

        I know how to use tee to write the output (standard output) of aaa.sh to bbb.out, while still displaying it in the terminal: ./aaa.sh | tee bbb.out How would I now also write standard error to a f...
        Last edited by mgogol; 09-22-2011, 07:28 AM.

        Comment

        Latest Articles

        Collapse

        • seqadmin
          Exploring the Dynamics of the Tumor Microenvironment
          by seqadmin




          The complexity of cancer is clearly demonstrated in the diverse ecosystem of the tumor microenvironment (TME). The TME is made up of numerous cell types and its development begins with the changes that happen during oncogenesis. “Genomic mutations, copy number changes, epigenetic alterations, and alternative gene expression occur to varying degrees within the affected tumor cells,” explained Andrea O’Hara, Ph.D., Strategic Technical Specialist at Azenta. “As...
          07-08-2024, 03:19 PM
        • seqadmin
          Exploring Human Diversity Through Large-Scale Omics
          by seqadmin


          In 2003, researchers from the Human Genome Project (HGP) announced the most comprehensive genome to date1. Although the genome wasn’t fully completed until nearly 20 years later2, numerous large-scale projects, such as the International HapMap Project and 1000 Genomes Project, continued the HGP's work, capturing extensive variation and genomic diversity within humans. Recently, newer initiatives have significantly increased in scale and expanded beyond genomics, offering a more detailed...
          06-25-2024, 06:43 AM

        ad_right_rmr

        Collapse

        News

        Collapse

        Topics Statistics Last Post
        Started by seqadmin, Yesterday, 05:49 AM
        0 responses
        13 views
        0 likes
        Last Post seqadmin  
        Started by seqadmin, 07-15-2024, 06:53 AM
        0 responses
        24 views
        0 likes
        Last Post seqadmin  
        Started by seqadmin, 07-10-2024, 07:30 AM
        0 responses
        37 views
        0 likes
        Last Post seqadmin  
        Started by seqadmin, 07-03-2024, 09:45 AM
        0 responses
        204 views
        0 likes
        Last Post seqadmin  
        Working...
        X