Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • mapping with bbmap anderror

    I am having a problem when mapping my short reads 150bp (paired-end reads) to the genome fasta.

    I would like to have all unique reads (no mismatching, minimum reads 150 bp) out that are mapped to the genome.

    Here I face the problem: I assigned 20 threads (20 cores) but my job ended up with error. Why it is happening and what is the solution?. Each file 23 GB big (R1 and R2)

    Code:
    job starting at 09:42:42
    java -ea -Xmx60001m -Xms60001m -cp /sw/bioinfo/bbmap/38.61b/rackham/current/ align2.BBMap build=1 overwrite=true fastareadlen=500 pairedonly=t ambiguous=toss killbadpairs=t pairlen=1000 trimq=20 mintrimlength=120 minaveragequality=20 rescuemismatches=1 outm=2008-27-201.aligned.sam outu=2008-27-201.unaligned.sam showprogress=10000 scafstats=2008_27.scafstats in=/crex/proj/snic2020-6-16/datasets/human_depleted/Ki-2008-27-201_unmapped_R1.fq.gz in2=/crex/proj/snic2020-6-16/datasets/human_depleted/Ki-2008-27-201_unmapped_R2.fq.gz threads=auto
    Executing align2.BBMap [build=1, overwrite=true, fastareadlen=500, pairedonly=t, ambiguous=toss, killbadpairs=t, pairlen=1000, trimq=20, mintrimlength=120, minaveragequality=20, rescuemismatches=1, outm=2008-27-201.aligned.sam, outu=2008-27-201.unaligned.sam, showprogress=10000, scafstats=2008_27.scafstats, in=/crex/proj/snic2020-6-16/datasets/human_depleted/Ki-2008-27-201_unmapped_R1.fq.gz, in2=/crex/proj/snic2020-6-16/datasets/human_depleted/Ki-2008-27-201_unmapped_R2.fq.gz, threads=auto]
    Version 38.61
    
    Scaffold statistics will be written to 2008_27.scafstats
    Set threads to 20
    Ambiguously mapped reads will be considered unmapped.
    Set genome to 1
    
    Loaded Reference:       2.025 seconds.
    Loading index for chunk 1-1, build 1
    Generated Index:        3.261 seconds.
    Analyzed Index:         2.517 seconds.
    Started output stream:  0.371 seconds.
    Started output stream:  0.023 seconds.
    Creating scaffold statistics table:     0.065 seconds.
    Cleared Memory:         0.516 seconds.
    Processing reads in paired-ended mode.
    Started read stream.
    Started 20 mapping threads.
    Exception in thread "Thread-6" java.lang.AssertionError: A00605:23:HHNVHDSXX:1:1101:13711:1783 1:N:0:TAATGCGC+AGGCGAAG  0       -1      +       -1      -1      00000000000000000000000 1       0       0       ATGTTGAATATGCTTTGTTTTATGTTATAATCAGGTGCTATTTAATTGTTTGACTGTTGGTGATTGCAGGTTTTGGAGAAAAGGAGGAGCTCATCCAGATCAACAAAGCTTGGACTTCTACAACAGCGTTTGGACAAAGGTAAGAACGAAA FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF .       .       .       .
            at align2.AbstractMapThread.rescue(AbstractMapThread.java:1200)
            at align2.BBMapThread.processReadPair(BBMapThread.java:1097)
            at align2.AbstractMapThread.run(AbstractMapThread.java:536)
    Detecting finished threads: 0Exception in thread "Thread-18" java.lang.AssertionError: A00605:23:HHNVHDSXX:1:1101:16107:1705 2:N:0:TAATGCGC+AGGCGAAG    601     -1      +       -1      -1      00000000001000000000000 1       0       0       CCGACCCTCCAAGAGTGTCTGGTATGTGTGGCTGAATTCTGGGTCGCTTTTGTAGAGAAGTGGCCAATCAGAAGTCTCGTGCCCGCAAGAGTTGAGCACGGTGGTTAGCGCCATGATCGGTGGTCGACTGAGGCAATCTGCAACATTATTG FFFFFFFFFFFFFFF:FFFFFFF,FFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF,FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFFFFFFFFF:FF,FFF .       .       .       .       1,1,28300816,28300966,0,00,14,14532,14556,14557,145561,1,28300816,28300966,0,00,14,14532,14556,14557,14556
            at align2.AbstractMapThread.rescue(AbstractMapThread.java:1200)
            at align2.BBMapThread.processReadPair(BBMapThread.java:1090)
            at align2.AbstractMapThread.run(AbstractMapThread.java:536)
    Exception in thread "Thread-24" java.lang.AssertionError: A00605:23:HHNVHDSXX:1:1101:8558:1626 1:N:0:TAATGCGC+AGGCGAAG  1002    -1      +       -1      -1      00000000000000000000000 1       0       0       GAAAACGGCTCCCACTTATTCTACACCTCTCAAGTCATTTCACAAAGTCGGACTAGAGTCAAGCTCAACAGGGTCTTCTTTCCCCGCTGATTCTGCCAAGCCCGTTCCCTTGGCTGTGGTTTCGCTAGATAGTAGATAGGGACAGTGGGAA FFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFF:FFFFFFF .       .       .       .       1,0,14943298,14943448,0,00,10,11843,11823,15526,118231,0,14943298,14943448,0,00,10,11843,11823,15526,11823      1,0,15184887,15185037,0,00,11,11685,11645,11646,116451,0,15184887,15185037,0,00,11,11685,11645,11646,11645
            at align2.AbstractMapThread.rescue(AbstractMapThread.java:1200)
            at align2.BBMapThread.processReadPair(BBMapThread.java:1097)
            at align2.AbstractMapThread.run(AbstractMapThread.java:536)
    Exception in thread "Thread-14" java.lang.AssertionError: A00605:23:HHNVHDSXX:1:1101:2293:1016 2:N:0:TAATGCGC+AGGCGAAG  1404    -1      +       -1      -1      00000000001000000000000 1       0       0       NATAGAGACAAGCTTTTATAGATAGAGAGGAAAGAGATAGGATCAACCACAATATTCATGGGATTGGCTTTATCACTTAGAAACTTCCATGAAAGGTGGCAAACCACTTCCCTTCTATGGAGATTCG !FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFFFFFFFFFFFF:FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFFFFFFFFFFFFFFFFFF .       .       .       .       1,0,37647506,37647634,0,00,6,7881,8211,0,82111,0,37647506,37647634,0,00,6,7881,8211,0,8211
            at align2.AbstractMapThread.rescue(AbstractMapThread.java:1200)
            at align2.BBMapThread.processReadPair(BBMapThread.java:1090)
            at align2.AbstractMapThread.run(AbstractMapThread.java:536)
    Exception in thread "Thread-8" java.lang.AssertionError: A00605:23:HHNVHDSXX:1:1101:3766:2785 2:N:0:TAATGCGC+AGGCGAAG   3802    -1      +       -1      -1      00000000001000000000000 1       0       0       GCAATGGATCGGTAGCATTACCCCAAATATGGATGGTATGGAGAGACCATGGTG  FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFFFFFFFFFFFFFF  .       .       .       .       1,0,21465899,21466010,0,00,1,1003,0,0,01,0,21465899,21466010,0,00,1,1003,0,0,0  1,0,27814628,27814681,0,00,4,4856,4856,4857,48561,0,27814628,27814681,0,00,4,4856,4856,4857,4856
            at align2.AbstractMapThread.rescue(AbstractMapThread.java:1200)
    
    
    -------------------
    ---------------------
    -----------------------------
    
    
    , 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19
    
    **************************************************************************
    Warning!  20 mapping threads did not terminate normally.
    Check the error log; the output may be corrupt or incomplete.
    Please submit the full stderr output as a bug report, not just this message.
    **************************************************************************
    
    
       ------------------   Results   ------------------
    
    Genome:                 1
    Key Length:             13
    Max Indel:              16000
    Minimum Score Ratio:    0.56
    Mapping Mode:           normal
    Reads Used:             212     (28878 bases)
    
    Mapping:                0.669 seconds.
    Reads/sec:              316.77
    kBases/sec:             43.15
    
    
    Pairing data:           pct pairs       num pairs       pct bases          num bases
    
    mated pairs:              2.8302%               3         1.9600%                566
    bad pairs:                0.0000%               0         0.0000%                  0
    insert size avg:           97.67
    
    
    Read 1 data:            pct reads       num reads       pct bases          num bases
    
    mapped:                   2.8302%               3         1.9552%                283
    unambiguous:              2.8302%               3         1.9552%                283
    ambiguous:                0.0000%               0         0.0000%                  0
    low-Q discards:           0.9434%               1         1.0432%                151
    
    perfect best site:        0.0000%               0         0.0000%                  0
    semiperfect site:         0.0000%               0         0.0000%                  0
    rescued:                  0.0000%               0
    
    Match Rate:                   NA               NA        86.9258%                246
    Error Rate:             100.0000%               3         9.1873%                 26
    Sub Rate:               100.0000%               3         9.1873%                 26
    Del Rate:                 0.0000%               0         0.0000%                  0
    Ins Rate:                 0.0000%               0         0.0000%                  0
    N Rate:                  33.3333%               1         3.8869%                 11
    
    
    Read 2 data:            pct reads       num reads       pct bases          num bases
    
    mapped:                   2.8302%               3         1.9647%                283
    unambiguous:              2.8302%               3         1.9647%                283
    ambiguous:                0.0000%               0         0.0000%                  0
    low-Q discards:           1.8868%               2         1.8189%                262
    
    perfect best site:        0.0000%               0         0.0000%                  0
    semiperfect site:         0.0000%               0         0.0000%                  0
    rescued:                  0.0000%               0
    
    Match Rate:                   NA               NA        89.0459%                252
    Error Rate:             100.0000%               3         9.5406%                 27
    Sub Rate:               100.0000%               3         9.5406%                 27
    Del Rate:                 0.0000%               0         0.0000%                  0
    Ins Rate:                 0.0000%               0         0.0000%                  0
    N Rate:                  33.3333%               1         1.4134%                  4
    Exception in thread "main" java.lang.AssertionError:
    The number of reads out does not add up to the number of reads in.
    This may indicate that a mapping thread crashed.
    If you submit a bug report, include the entire console output, not just this error message.
    3+0+0+82+1+0 = 86 != 106
            at align2.AbstractMapper.printOutputStats(AbstractMapper.java:1961)
            at align2.AbstractMapper.printOutput(AbstractMapper.java:1059)
            at align2.BBMap.testSpeed(BBMap.java:522)
            at align2.BBMap.main(BBMap.java:35)
    job finishing at 09:42:59
    Last edited by abubd; 10-25-2021, 12:54 AM.

  • #2
    Do you have 20 cores available on the machine for BBMap to use. What happens if you assign fewer cores? Starting with 8 would be a good number.

    Comment


    • #3
      Yes, cores are available as I did run it in a computer cluster. That is not a problem I think.

      Latest update: When I changed
      rescuemismatches 1 to 2
      . It started working but for a long time, 2 hours and the job is canceled due to the time limit that means I need more time to assign for this run with 20 cores (1 node). so far resulting files are:
      22G Oct 25 13:42 2008-27-201.aligned.sam
      120G Oct 25 13:42 2008-27-201.unaligned.sam
      But I want a unique reads that should not be so much!

      Comment


      • #4
        I would like to have all unique reads (no mismatching, minimum reads 150 bp) out that are mapped to the genome.
        If you need full length perfect alignments in your final file then I suggest that you try "perfectmode=t minid=1 mappedonly=t" options. If your reads are longer than 150 bp then you may need to modulate minid parameter. Add secondary=t if you want secondary alignments.

        Comment


        • #5
          I'm doing some research on this as well, but found little information

          Comment

          Latest Articles

          Collapse

          • seqadmin
            Current Approaches to Protein Sequencing
            by seqadmin


            Proteins are often described as the workhorses of the cell, and identifying their sequences is key to understanding their role in biological processes and disease. Currently, the most common technique used to determine protein sequences is mass spectrometry. While still a valuable tool, mass spectrometry faces several limitations and requires a highly experienced scientist familiar with the equipment to operate it. Additionally, other proteomic methods, like affinity assays, are constrained...
            04-04-2024, 04:25 PM
          • seqadmin
            Strategies for Sequencing Challenging Samples
            by seqadmin


            Despite advancements in sequencing platforms and related sample preparation technologies, certain sample types continue to present significant challenges that can compromise sequencing results. Pedro Echave, Senior Manager of the Global Business Segment at Revvity, explained that the success of a sequencing experiment ultimately depends on the amount and integrity of the nucleic acid template (RNA or DNA) obtained from a sample. “The better the quality of the nucleic acid isolated...
            03-22-2024, 06:39 AM

          ad_right_rmr

          Collapse

          News

          Collapse

          Topics Statistics Last Post
          Started by seqadmin, 04-11-2024, 12:08 PM
          0 responses
          18 views
          0 likes
          Last Post seqadmin  
          Started by seqadmin, 04-10-2024, 10:19 PM
          0 responses
          22 views
          0 likes
          Last Post seqadmin  
          Started by seqadmin, 04-10-2024, 09:21 AM
          0 responses
          17 views
          0 likes
          Last Post seqadmin  
          Started by seqadmin, 04-04-2024, 09:00 AM
          0 responses
          49 views
          0 likes
          Last Post seqadmin  
          Working...
          X