Announcement

Collapse

Welcome to the New Seqanswers!

Welcome to the new Seqanswers! We'd love your feedback, please post any you have to this topic: New Seqanswers Feedback.
See more
See less

Strange memory corruption error in BFAST match step

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Strange memory corruption error in BFAST match step

    Hi, I have a set of 300 million 50-bp long SOLiD reads, and I would like to use BFAST to map them to a reference genome. I've successfully created the color-space reference genome .brfg file, and the 10 color-space reference genome indexes recommended in the BFAST book.

    When trying 'bfast match -f reference_genome -A 1 -r reads.fastq > matches.bmf' in a 32 GB RAM machine, I got the memory corruption error copied below. Following BFAST book recommendations, I splitted the global fastq file in 10 filse with 30 million reads each. Since they also produced the same memory corruption error, I further splitted the reads in files with 3 million reads each. First three files were run successfully, but fourth file gave the same memory corruption error again. Then I splitted this file in 3 files with 1 million reads each. Again, first run was fine, but second run gave the same memory corruption error.

    The strange thing is that running the fifth 3 million reads file (which is a little bigger than fourth one) did not give the memory corruption error. So, I do not think that I need more RAM because some 3 million reads could be processed, whereas some 1 million could not.

    Any idea?

    Files size are 33 GB for the 300 million reads fastq file, and 3.6 GB for each of 10 indexes

    (...normal output here...)
    Reading reads.fastq into temp files.
    *** glibc detected *** bfast: malloc(): memory corruption: 0x000000000065cf30 ***
    ======= Backtrace: =========
    /lib/libc.so.6[0x7f8ad88d3a14]
    /lib/libc.so.6(__libc_malloc+0x90)[0x7f8ad88d5360]
    /lib/libc.so.6(vasprintf+0x3e)[0x7f8ad88ca66e]
    /lib/libc.so.6(asprintf+0x88)[0x7f8ad88ad098]
    /lib/libc.so.6(__assert_fail+0xb8)[0x7f8ad888a2a8]
    bfast[0x41770d]
    bfast[0x418927]
    bfast[0x423b3b]
    bfast[0x4259c5]
    bfast[0x42b2d2]
    /lib/libc.so.6(__libc_start_main+0xf4)[0x7f8ad887d1c4]
    bfast[0x401ce9]
    ======= Memory map: ========
    00400000-0044d000 r-xp 00000000 08:03 2711591 /usr/local/bin/bfast
    0064d000-0064f000 rw-p 0004d000 08:03 2711591 /usr/local/bin/bfast
    0064f000-010d8000 rw-p 0064f000 00:00 0 [heap]
    7f8abc000000-7f8abc021000 rw-p 7f8abc000000 00:00 0
    7f8abc021000-7f8ac0000000 ---p 7f8abc021000 00:00 0
    7f8ac1c0c000-7f8ac1c19000 r-xp 00000000 08:03 647184 /lib/libgcc_s.so.1
    7f8ac1c19000-7f8ac1e19000 ---p 0000d000 08:03 647184 /lib/libgcc_s.so.1
    7f8ac1e19000-7f8ac1e1a000 rw-p 0000d000 08:03 647184 /lib/libgcc_s.so.1
    7f8ac1e1a000-7f8ad885f000 rw-p 7f8ac1e1a000 00:00 0
    7f8ad885f000-7f8ad89b7000 r-xp 00000000 08:03 647441 /lib/libc-2.7.so
    7f8ad89b7000-7f8ad8bb7000 ---p 00158000 08:03 647441 /lib/libc-2.7.so
    7f8ad8bb7000-7f8ad8bba000 r--p 00158000 08:03 647441 /lib/libc-2.7.so
    7f8ad8bba000-7f8ad8bbc000 rw-p 0015b000 08:03 647441 /lib/libc-2.7.so
    7f8ad8bbc000-7f8ad8bc1000 rw-p 7f8ad8bbc000 00:00 0
    7f8ad8bc1000-7f8ad8bd7000 r-xp 00000000 08:03 647459 /lib/libpthread-2.7.so
    7f8ad8bd7000-7f8ad8dd7000 ---p 00016000 08:03 647459 /lib/libpthread-2.7.so
    7f8ad8dd7000-7f8ad8dd9000 rw-p 00016000 08:03 647459 /lib/libpthread-2.7.so
    7f8ad8dd9000-7f8ad8ddd000 rw-p 7f8ad8dd9000 00:00 0
    7f8ad8ddd000-7f8ad8dec000 r-xp 00000000 08:03 647294 /lib/libbz2.so.1.0.4
    7f8ad8dec000-7f8ad8feb000 ---p 0000f000 08:03 647294 /lib/libbz2.so.1.0.4
    7f8ad8feb000-7f8ad8fed000 rw-p 0000e000 08:03 647294 /lib/libbz2.so.1.0.4
    7f8ad8fed000-7f8ad9003000 r-xp 00000000 08:03 4089806 /usr/lib/libz.so.1.2.3.3
    7f8ad9003000-7f8ad9203000 ---p 00016000 08:03 4089806 /usr/lib/libz.so.1.2.3.3
    7f8ad9203000-7f8ad9204000 rw-p 00016000 08:03 4089806 /usr/lib/libz.so.1.2.3.3
    7f8ad9204000-7f8ad9284000 r-xp 00000000 08:03 647446 /lib/libm-2.7.so
    7f8ad9284000-7f8ad9483000 ---p 00080000 08:03 647446 /lib/libm-2.7.so
    7f8ad9483000-7f8ad9485000 rw-p 0007f000 08:03 647446 /lib/libm-2.7.so
    7f8ad9485000-7f8ad94a2000 r-xp 00000000 08:03 647435 /lib/ld-2.7.so
    7f8ad9543000-7f8ad9694000 rw-p 7f8ad9543000 00:00 0
    7f8ad969d000-7f8ad96a2000 rw-p 7f8ad969d000 00:00 0
    7f8ad96a2000-7f8ad96a4000 rw-p 0001d000 08:03 647435 /lib/ld-2.7.so
    7fff956b3000-7fff956c8000 rw-p 7ffffffea000 00:00 0 [stack]
    7fff957fe000-7fff95800000 r-xp 7fff957fe000 00:00 0 [vdso]
    ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]

  • #2
    Sounds like a bug in BFAST. Memory corruption errors are caused by incorrect use of malloc()/free() not running out of memory. Suggest you contact the BFAST authors. They might ask you to send an example file which has the problem.

    Comment


    • #3
      Originally posted by nickloman View Post
      Sounds like a bug in BFAST. Memory corruption errors are caused by incorrect use of malloc()/free() not running out of memory. Suggest you contact the BFAST authors. They might ask you to send an example file which has the problem.
      I think BFAST author follows closely this forum...

      BTW, it seems that BFAST does not like some read/s, so that memory corruption error only appears in subsets of reads having it/them. This would explain why it appears in some subsets and does not appear in others, irrespective of the file size or the reads number.

      Comment


      • #4
        Originally posted by javijevi View Post
        BTW, it seems that BFAST does not like some read/s, so that memory corruption error only appears in subsets of reads having it/them. This would explain why it appears in some subsets and does not appear in others, irrespective of the file size or the reads number.
        Maybe this is not true, since I have a 250,000 reads fastq file producing the memory corruption error, whereas the two 125,000 reads fastq files obtained by splitting it did not cause the error. Strange, very strange...

        Comment


        • #5
          Originally posted by javijevi View Post
          Maybe this is not true, since I have a 250,000 reads fastq file producing the memory corruption error, whereas the two 125,000 reads fastq files obtained by splitting it did not cause the error. Strange, very strange...
          Very strange indeed. Could you post the amount of RAM available, the full command used, the Operating System you are using? Also, if you post the reference and reads as well as the indexes you used, I can try to reproduce/debug.

          Hopefully we can figure this out!

          Nils

          Comment


          • #6
            Hi, has there been progress on debugging this issue? I am getting a similar error when trying to align a million Illumina reads on a Linux machine with 64G RAM. Any help you can give would be greatly appreciated!

            *** glibc detected *** bfast: malloc(): memory corruption: 0x000000000fd57f40 ***
            ======= Backtrace: =========
            /lib64/libc.so.6[0x384406faf1]
            /lib64/libc.so.6(__libc_malloc+0x7d)[0x3844070c9d]
            /lib64/libc.so.6(vasprintf+0x3e)[0x3844066e6e]
            /lib64/libc.so.6(asprintf+0x88)[0x384404bfa8]
            /lib64/libc.so.6(__assert_fail+0xbf)[0x384402971f]
            bfast[0x4167ad]
            bfast[0x417957]
            bfast[0x422646]
            bfast[0x4242df]
            bfast[0x42a172]
            /lib64/libc.so.6(__libc_start_main+0xf4)[0x384401d8a4]
            bfast[0x401b09]
            ======= Memory map: ========
            00400000-0044a000 r-xp 00000000 00:1a 443580418 /home/gfs08/nc276/programs/bfast-0.6.4d/bin/bfast
            0064a000-0064c000 rw-p 0004a000 00:1a 443580418 /home/gfs08/nc276/programs/bfast-0.6.4d/bin/bfast
            0fd4a000-15fea000 rw-p 0fd4a000 00:00 0 [heap]
            3843c00000-3843c1a000 r-xp 00000000 08:01 720069 /lib64/ld-2.5.so
            3843e19000-3843e1a000 r--p 00019000 08:01 720069 /lib64/ld-2.5.so
            3843e1a000-3843e1b000 rw-p 0001a000 08:01 720069 /lib64/ld-2.5.so
            3844000000-3844146000 r-xp 00000000 08:01 720070 /lib64/libc-2.5.so
            3844146000-3844346000 ---p 00146000 08:01 720070 /lib64/libc-2.5.so
            3844346000-384434a000 r--p 00146000 08:01 720070 /lib64/libc-2.5.so
            384434a000-384434b000 rw-p 0014a000 08:01 720070 /lib64/libc-2.5.so
            384434b000-3844350000 rw-p 384434b000 00:00 0
            3844400000-3844482000 r-xp 00000000 08:01 720071 /lib64/libm-2.5.so
            3844482000-3844681000 ---p 00082000 08:01 720071 /lib64/libm-2.5.so
            3844681000-3844682000 r--p 00081000 08:01 720071 /lib64/libm-2.5.so
            3844682000-3844683000 rw-p 00082000 08:01 720071 /lib64/libm-2.5.so
            3844c00000-3844c15000 r-xp 00000000 08:01 3580079 /lib64/libpthread-2.5.so
            3844c15000-3844e14000 ---p 00015000 08:01 3580079 /lib64/libpthread-2.5.so
            3844e14000-3844e15000 r--p 00014000 08:01 3580079 /lib64/libpthread-2.5.so
            3844e15000-3844e16000 rw-p 00015000 08:01 3580079 /lib64/libpthread-2.5.so
            3844e16000-3844e1a000 rw-p 3844e16000 00:00 0
            3845800000-3845814000 r-xp 00000000 08:01 1464653 /usr/lib64/libz.so.1.2.3
            3845814000-3845a13000 ---p 00014000 08:01 1464653 /usr/lib64/libz.so.1.2.3
            3845a13000-3845a14000 rw-p 00013000 08:01 1464653 /usr/lib64/libz.so.1.2.3
            3848c00000-3848c0f000 r-xp 00000000 08:01 1464655 /usr/lib64/libbz2.so.1.0.3
            3848c0f000-3848e0e000 ---p 0000f000 08:01 1464655 /usr/lib64/libbz2.so.1.0.3
            3848e0e000-3848e10000 rw-p 0000e000 08:01 1464655 /usr/lib64/libbz2.so.1.0.3
            2ac513b5f000-2ac513b62000 rw-p 2ac513b5f000 00:00 0
            2ac513b72000-2ac5328f8000 rw-p 2ac513b72000 00:00 0
            2ac53290a000-2ac532920000 r-xp 00000000 08:01 2703644 /opt/gcc-4.3.1/lib64/libgcc_s.so.1
            2ac532920000-2ac532b1f000 ---p 00016000 08:01 2703644 /opt/gcc-4.3.1/lib64/libgcc_s.so.1
            2ac532b1f000-2ac532b20000 rw-p 00015000 08:01 2703644 /opt/gcc-4.3.1/lib64/libgcc_s.so.1
            2ac534000000-2ac534021000 rw-p 2ac534000000 00:00 0
            2ac534021000-2ac538000000 ---p 2ac534021000 00:00 0
            7fff06fd5000-7fff06fea000 rw-p 7ffffffea000 00:00 0 [stack]
            ffffffffff600000-ffffffffffe00000 ---p 00000000 00:00 0 [vdso]

            Comment


            • #7
              Could you try the latest source via GIT to see if that resolves it? If not, could you try to find the offending read(s) so it is easier to debug?

              Comment


              • #8
                Hi,

                It's been a while since the last post in this thread, but i was wondering if someone found a solution to the problem yet? I have the exact same problem as described.

                Comment


                • #9
                  Originally posted by MaartenR View Post
                  Hi,

                  It's been a while since the last post in this thread, but i was wondering if someone found a solution to the problem yet? I have the exact same problem as described.
                  What version of bfast are you using?

                  Comment


                  • #10
                    0.6.4F, but theres a slim chance i am using 0.6.4E. I will have to check at work tomorrow

                    Comment


                    • #11
                      Does it occur every time? If it does, could you send me the test case so I can fix it?

                      Comment


                      • #12
                        No, it worked before. But when i tried doing it again it gave that error. Tried rebuilding the sample but it gives the same error.

                        Comment


                        • #13
                          Bfast memory corruption error on all 8 read files

                          Hello

                          I have 8 files with 50bp SE SOLiD reads. For each and single file I am getting the same error during match step:
                          *** glibc detected *** /mnt/fastfs/bin/bfast: malloc(): memory corruption: 0x000000000065cf30 *** (memory adresses are different of course).

                          I am using the bfast-0.6.5a and all standard parameters and masks, and I am mapping against the mRNA refseqs. Files with reads are about 5Gb and I have 32G memory.

                          Any ideas how to solve the problem

                          Thanks

                          K

                          Comment


                          • #14
                            If you could submit a test case, as detailed here, it would make it much easier for me to debug.

                            Comment


                            • #15
                              More details

                              Thank you.

                              I run the bfast match with -s 1 and -e 1 (the same is for -e 100) and the full error message is:
                              ************************************************************
                              Checking input parameters supplied by the user ...
                              Validating fastaFileName /mnt/fastfs/KW_Temp/bfast/refseq.fas.
                              Validating readsFileName /mnt/fastfs/KW_Temp/SSP2_16.single.fastq.
                              Validating tmpDir path ./.
                              **** Input arguments look good!
                              ************************************************************
                              ************************************************************
                              Printing Program Parameters:
                              programMode: [ExecuteProgram]
                              fastaFileName: /mnt/fastfs/KW_Temp/bfast/refseq.fas
                              mainIndexes [Auto-recognizing]
                              secondaryIndexes [Not Using]
                              readsFileName: /mnt/fastfs/KW_Temp/SSP2_16.single.fastq
                              offsets: [Using All]
                              loadAllIndexes: [Not Using]
                              compression: [Not Using]
                              space: [Color Space]
                              startReadNum: 1
                              endReadNum: 1
                              keySize: [Not Using]
                              maxKeyMatches: 8
                              maxNumMatches: 384
                              whichStrand: [Both Strands]
                              numThreads: 1
                              queueLength: 250000
                              tmpDir: ./
                              timing: [Not Using]
                              ************************************************************
                              Searching for main indexes...
                              Found 10 index (10 total files).
                              Not using secondary indexes.
                              ************************************************************
                              Reading in reference genome from /mnt/fastfs/KW_Temp/bfast/refseq.fas.cs.brg.
                              In total read 35860 contigs for a total of 91307366 bases
                              ************************************************************
                              Reading /mnt/fastfs/KW_Temp/SSP2_16.single.fastq into a temp file.
                              *** glibc detected *** /mnt/fastfs/bin/bfast: free(): invalid next size (fast): 0x000000000065cf10 ***
                              ======= Backtrace: =========
                              /lib/libc.so.6[0x2b30d30649a8]
                              /lib/libc.so.6(cfree+0x76)[0x2b30d3066ab6]
                              /mnt/fastfs/bin/bfast[0x415f9a]
                              /mnt/fastfs/bin/bfast[0x418af4]
                              /mnt/fastfs/bin/bfast[0x424124]
                              /mnt/fastfs/bin/bfast[0x425ffa]
                              /mnt/fastfs/bin/bfast[0x42d260]
                              /lib/libc.so.6(__libc_start_main+0xe6)[0x2b30d300f1a6]
                              /mnt/fastfs/bin/bfast[0x401e49]
                              ======= Memory map: ========
                              00400000-0044d000 r-xp 00000000 00:15 1003536386 /mnt/fastfs/bin/bfast
                              0064d000-0064f000 rw-p 0004d000 00:15 1003536386 /mnt/fastfs/bin/bfast
                              0064f000-0347c000 rw-p 0064f000 00:00 0 [heap]
                              2b30d250d000-2b30d2529000 r-xp 00000000 68:01 14196756 /lib/ld-2.7.so
                              2b30d2529000-2b30d252a000 r-xp 2b30d2529000 00:00 0 [vdso]
                              2b30d252a000-2b30d252d000 rw-p 2b30d252a000 00:00 0
                              2b30d254d000-2b30d2666000 rw-p 2b30d254d000 00:00 0
                              2b30d2728000-2b30d272a000 rw-p 0001b000 68:01 14196756 /lib/ld-2.7.so
                              2b30d272a000-2b30d27ac000 r-xp 00000000 68:01 14196741 /lib/libm-2.7.so
                              2b30d27ac000-2b30d29ab000 ---p 00082000 68:01 14196741 /lib/libm-2.7.so
                              2b30d29ab000-2b30d29ad000 rw-p 00081000 68:01 14196741 /lib/libm-2.7.so
                              2b30d29ad000-2b30d29bc000 r-xp 00000000 68:01 14197946 /lib/libbz2.so.1.0.4
                              2b30d29bc000-2b30d2bbb000 ---p 0000f000 68:01 14197946 /lib/libbz2.so.1.0.4
                              2b30d2bbb000-2b30d2bbd000 rw-p 0000e000 68:01 14197946 /lib/libbz2.so.1.0.4
                              2b30d2bbd000-2b30d2bd3000 r-xp 00000000 68:01 2328701 /usr/lib/libz.so.1.2.3.3
                              2b30d2bd3000-2b30d2dd3000 ---p 00016000 68:01 2328701 /usr/lib/libz.so.1.2.3.3
                              2b30d2dd3000-2b30d2dd4000 rw-p 00016000 68:01 2328701 /usr/lib/libz.so.1.2.3.3
                              2b30d2dd4000-2b30d2dd5000 rw-p 2b30d2dd4000 00:00 0
                              2b30d2dd5000-2b30d2deb000 r-xp 00000000 68:01 14196761 /lib/libpthread-2.7.so
                              2b30d2deb000-2b30d2feb000 ---p 00016000 68:01 14196761 /lib/libpthread-2.7.so
                              2b30d2feb000-2b30d2fed000 rw-p 00016000 68:01 14196761 /lib/libpthread-2.7.so
                              2b30d2fed000-2b30d2ff1000 rw-p 2b30d2fed000 00:00 0
                              2b30d2ff1000-2b30d313b000 r-xp 00000000 68:01 14196749 /lib/libc-2.7.so
                              2b30d313b000-2b30d333a000 ---p 0014a000 68:01 14196749 /lib/libc-2.7.so
                              2b30d333a000-2b30d333d000 r--p 00149000 68:01 14196749 /lib/libc-2.7.so
                              2b30d333d000-2b30d333f000 rw-p 0014c000 68:01 14196749 /lib/libc-2.7.so
                              2b30d333f000-2b30d3466000 rw-p 2b30d333f000 00:00 0
                              2b30d3472000-2b30d3488000 r-xp 00000000 68:01 14196739 /lib/libgcc_s.so.1
                              2b30d3488000-2b30d3688000 ---p 00016000 68:01 14196739 /lib/libgcc_s.so.1
                              2b30d3688000-2b30d3689000 rw-p 00016000 68:01 14196739 /lib/libgcc_s.so.1
                              2b30d4000000-2b30d4021000 rw-p 2b30d4000000 00:00 0
                              2b30d4021000-2b30d8000000 ---p 2b30d4021000 00:00 0
                              7ffffffea000-7ffffffff000 rw-p 7ffffffea000 00:00 0 [stack]
                              ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]

                              The first read in fastq file is:
                              @SSP2_16:559_41_123/1
                              TGNCCTNNGGGNNTGAGGTNTANNNGNCNNNNCNNCNGNNTNNNNACAN
                              +
                              [email protected],!-(<!!*<9!!698.3&!6,!!!3!8!!!!:!!%!)!!(!!!!&(%!

                              K
                              Last edited by kwicher; 09-03-2011, 05:56 PM. Reason: added info

                              Comment

                              Working...
                              X