BlastX core dump?

[Blaze9]

Hey guys,

I'm running into a bit of a problem with one of my BlastX instances.

I have a 50k sequence transcriptome split into 10 different files. I've ran 10 different blastX instances, but one of them has crashed a few times. I'm assuming this is because of the fasta file, but I'm not entirely sure.

Here's the core-dump and the fasta file, along with the commands I issued.

Can someone see why it's crashing? I ran both the precompiled versions of Blast as well as a self-compiled version. I'm running it on CentOS 6.5 on a computer w/ 32 threads and 128GB of RAM.

[root@NGS-Linux 1000 split]# blastx -show_gis -max_hsps 20 -num_threads 15 -db db1 -evalue 0.001 -word_size 3 -num_alignments 1 -outfmt 5 -out fasta5+7.fa.2.xml -query fasta5+7.fa.2
*** glibc detected *** blastx: munmap_chunk(): invalid pointer: 0x00007f59ada803db ***
======= Backtrace: =========
/lib64/libc.so.6[0x3a66a75e76]
blastx(__sfree+0xf)[0xb10b07]
blastx(BlastSequenceBlkClean+0x20)[0xb11b2c]
blastx(BlastSequenceBlkFree+0xf)[0xb11ad3]
blastx(BLAST_PreliminarySearchEngine+0x719)[0xad431f]
blastx(Blast_RunPreliminarySearchWithInterrupt+0x169)[0xad5ef7]
blastx(_ZN4ncbi5blast19CPrelimSearchThread4MainEv+0x148)[0xbad172]
blastx(_ZN4ncbi7CThread7WrapperEPv+0x2d0)[0x16c4498]
blastx[0x16c41c6]
/lib64/libpthread.so.0[0x3a672079d1]
/lib64/libc.so.6(clone+0x6d)[0x3a66ae886d]
======= Memory map: ========
00400000-01ec4000 r-xp 00000000 08:02 4733627                            /usr/bin/blastx
020c4000-021aa000 rwxp 01ac4000 08:02 4733627                            /usr/bin/blastx
021aa000-021c9000 rwxp 00000000 00:00 0 
023a9000-026ea000 rwxp 01ba9000 08:02 4733627                            /usr/bin/blastx
0316c000-04ca6000 rwxp 00000000 00:00 0                                  [heap]
3a66600000-3a66620000 r-xp 00000000 08:02 1442141                        /lib64/ld-2.12.so
3a6681f000-3a66820000 r-xp 0001f000 08:02 1442141                        /lib64/ld-2.12.so
3a66820000-3a66821000 rwxp 00020000 08:02 1442141                        /lib64/ld-2.12.so
3a66821000-3a66822000 rwxp 00000000 00:00 0 
3a66a00000-3a66b8a000 r-xp 00000000 08:02 1442142                        /lib64/libc-2.12.so
3a66b8a000-3a66d8a000 ---p 0018a000 08:02 1442142                        /lib64/libc-2.12.so
3a66d8a000-3a66d8e000 r-xp 0018a000 08:02 1442142                        /lib64/libc-2.12.so
3a66d8e000-3a66d8f000 rwxp 0018e000 08:02 1442142                        /lib64/libc-2.12.so
3a66d8f000-3a66d94000 rwxp 00000000 00:00 0 
3a66e00000-3a66e83000 r-xp 00000000 08:02 1442168                        /lib64/libm-2.12.so
3a66e83000-3a67082000 ---p 00083000 08:02 1442168                        /lib64/libm-2.12.so
3a67082000-3a67083000 r-xp 00082000 08:02 1442168                        /lib64/libm-2.12.so
3a67083000-3a67084000 rwxp 00083000 08:02 1442168                        /lib64/libm-2.12.so
3a67200000-3a67217000 r-xp 00000000 08:02 1442149                        /lib64/libpthread-2.12.so
3a67217000-3a67417000 ---p 00017000 08:02 1442149                        /lib64/libpthread-2.12.so
3a67417000-3a67418000 r-xp 00017000 08:02 1442149                        /lib64/libpthread-2.12.so
3a67418000-3a67419000 rwxp 00018000 08:02 1442149                        /lib64/libpthread-2.12.so
3a67419000-3a6741d000 rwxp 00000000 00:00 0 
3a67600000-3a67602000 r-xp 00000000 08:02 1442215                        /lib64/libdl-2.12.so
3a67602000-3a67802000 ---p 00002000 08:02 1442215                        /lib64/libdl-2.12.so
3a67802000-3a67803000 r-xp 00002000 08:02 1442215                        /lib64/libdl-2.12.so
3a67803000-3a67804000 rwxp 00003000 08:02 1442215                        /lib64/libdl-2.12.so
3a67a00000-3a67a15000 r-xp 00000000 08:02 1442214                        /lib64/libz.so.1.2.3
3a67a15000-3a67c14000 ---p 00015000 08:02 1442214                        /lib64/libz.so.1.2.3
3a67c14000-3a67c15000 r-xp 00014000 08:02 1442214                        /lib64/libz.so.1.2.3
3a67c15000-3a67c16000 rwxp 00015000 08:02 1442214                        /lib64/libz.so.1.2.3
3a67e00000-3a67e07000 r-xp 00000000 08:02 1442163                        /lib64/librt-2.12.so
3a67e07000-3a68006000 ---p 00007000 08:02 1442163                        /lib64/librt-2.12.so
3a68006000-3a68007000 r-xp 00006000 08:02 1442163                        /lib64/librt-2.12.so
3a68007000-3a68008000 rwxp 00007000 08:02 1442163                        /lib64/librt-2.12.so
3a6da00000-3a6da16000 r-xp 00000000 08:02 1442225                        /lib64/libgcc_s-4.4.7-20120601.so.1
3a6da16000-3a6dc15000 ---p 00016000 08:02 1442225                        /lib64/libgcc_s-4.4.7-20120601.so.1
3a6dc15000-3a6dc16000 rwxp 00015000 08:02 1442225                        /lib64/libgcc_s-4.4.7-20120601.so.1
3a70a00000-3a70ae8000 r-xp 00000000 08:02 4766536                        /usr/lib64/libstdc++.so.6.0.13
3a70ae8000-3a70ce8000 ---p 000e8000 08:02 4766536                        /usr/lib64/libstdc++.so.6.0.13
3a70ce8000-3a70cef000 r-xp 000e8000 08:02 4766536                        /usr/lib64/libstdc++.so.6.0.13
3a70cef000-3a70cf1000 rwxp 000ef000 08:02 4766536                        /usr/lib64/libstdc++.so.6.0.13
3a70cf1000-3a70d06000 rwxp 00000000 00:00 0 
3a76a00000-3a76a16000 r-xp 00000000 08:02 1442228                        /lib64/libnsl-2.12.so
3a76a16000-3a76c15000 ---p 00016000 08:02 1442228                        /lib64/libnsl-2.12.so
3a76c15000-3a76c16000 r-xp 00015000 08:02 1442228                        /lib64/libnsl-2.12.so
3a76c16000-3a76c17000 rwxp 00016000 08:02 1442228                        /lib64/libnsl-2.12.so
3a76c17000-3a76c19000 rwxp 00000000 00:00 0 
3a78a00000-3a78a10000 r-xp 00000000 08:02 1442240                        /lib64/libbz2.so.1.0.4
3a78a10000-3a78c0f000 ---p 00010000 08:02 1442240                        /lib64/libbz2.so.1.0.4
3a78c0f000-3a78c11000 rwxp 0000f000 08:02 1442240                        /lib64/libbz2.so.1.0.4
7f40729d0000-7f4cf29d1000 rwxp 00000000 00:00 0 
7f590e9d6000-7f5948c73000 r-xp 00000000 08:03 5767468                    /media/NGSSoftware/BLASTDB/db1.22.psq
7f5948c73000-7f5983161000 r-xp 00000000 08:03 5767455                    /media/NGSSoftware/BLASTDB/db1.21.psq
7f5984b22000-7f59c0463000 r-xp 00000000 08:03 5767494                    /media/NGSSoftware/BLASTDB/db1.24.psq
7f59c0463000-7f59fbce0000 r-xp 00000000 08:03 5767481                    /media/NGSSoftware/BLASTDB/db1.23.psq
7f5a6d230000-7f5a9a703000 r-xp 00000000 08:03 5767325                    /media/NGSSoftware/BLASTDB/db1.11.psq
7f5a9a703000-7f5ad493a000 r-xp 00000000 08:03 5767312                    /media/NGSSoftware/BLASTDB/db1.10.psq
7f5ad493a000-7f5b1079d000 r-xp 00000000 08:03 5767299                    /media/NGSSoftware/BLASTDB/db1.09.psq
7f5b1079d000-7f5b40735000 r-xp 00000000 08:03 5767273                    /media/NGSSoftware/BLASTDB/db1.07.psq
7f5b40735000-7f5b7934c000 r-xp 00000000 08:03 5767260                    /media/NGSSoftware/BLASTDB/db1.06.psq
7f5b7934c000-7f5bb06dd000 r-xp 00000000 08:03 5767247                    /media/NGSSoftware/BLASTDB/db1.05.psq
7f5bb06dd000-7f5be76f9000 r-xp 00000000 08:03 5767234                    /media/NGSSoftware/BLASTDB/db1.04.psq
7f5be76f9000-7f5c03ffd000 r-xp 00000000 08:03 5767221                    /media/NGSSoftware/BLASTDB/db1.03.psq
7f5c03ffd000-7f5c17f4e000 r-xp 00000000 08:03 5767208                    /media/NGSSoftware/BLASTDB/db1.02.psq
7f5c17f4e000-7f5c4adf3000 r-xp 00000000 08:03 5767195                    /media/NGSSoftware/BLASTDB/db1.01.psq
7f5c4e90d000-7f5c7001a000 r-xp 00000000 08:03 5767442                    /media/NGSSoftware/BLASTDB/db1.20.psqAborted (core dumped)

The "db1" DB is actually a replicate of the "nr" database. I thought that when running multiple blast instances, each should have their own database.

I also tried this same fasta file with the original nr database incase something went wrong with one of the duplicates. Also redownladed the nr db as well incase something corrupted it. The DBs are stored on a 1TB 840 EVO, and right now only one blast is running. The output/input fasta files are stored on a 7.2k 3TB storage drive. Blast is also stored on the same 1TB Evo along w/ my OS.

Here is the fasta file. Fasta on Dropbox I ran it through a few different cleaning scripts but their output was the same as the input.

Thank you for your help!

[bt27uk]

I took a quick look at your fasta file, and as you said, nothing jumps out as suspicious. If no-one replies to this thread with further ideas about what to do, then I would suggest that you contact the blast support folk at the NCBI to see if they have any suggestions. I believe they can be contacted by email using the address blast-help (at) ncbi.nlm.nih.gov. If you are not already running the latest version of blast, please try upgrading and see if that resolves the issue before reporting the problem. If you do report this issue, I'm sure the folks at the NCBI would appreciate knowing the blast version you are using, the version/date of the nr database you are working with, and information about the system(s) you have seen this problem on. Good luck.

[maubp]

Multiple BLAST jobs can access the same database at once, so there is no reason to copy/replicate the nr.* files. Perhaps there was an error in your copy which causes this crash?