Lastz help

[Esther]

Hi all,

I'm trying to get lastz to compare some sequences to the human reference genome. This is the command I'm using:

lastz /path/to/data/fastafile.fasta[multiple] /path/to/reference/reference_human.fa --format=difference --identity=90 --coverage=50 --output=/path/to/data/lastzout.fa

This gives the following error:

FAILURE: call to realloc failed to allocate 135087680 bytes, for add_segment

Does anyone know what this means and what I can do about it?

[david2]

Hi Esther,

I am getting the identical error message when trying to align 454 run with ~8000 reads on an 80kb reference sequence, using:

./lastz brca1.fasta 454run.fna --format=sam --step=10 --seed=match12 --notransition --exact=20 --noytrim --match=1,5 --ambiguous=n --coverage=90 --identity=95

Is there anyone out there who knows what we do wrong?

I would like to replicate the Lastz parameters used on the public Galaxy server, does anyone know where I can find which parameters they use?

Cheers,

David

[amitbik]

I am using lastz for piler and my command is

$lastz unigenes.fa[multiple][unmask] --format=maf > unilastz.maf
FAILURE: bad fasta character in unigenes.fa, >Contig10: Y

my contig10 is
>Contig10
TGAAAACTAATATATATATATGTAGCATACCTGGATTCAACTTTACTGGTGGTTCAACAT
GAATCGTTCAAACAATTCCGCCTGTGCTTTTAGTTTCTCCAATTCGCCATTAGTCTTCTT
ATTTTCCTCCTTTTGAGCCTCCAACTCCATCTTGAGGGTCTGAATTTCTTCTTTAGCACT
CTCAAGCTGAGTTTGAGTCTCATTTATCCCAAAATAGCTATAGCGACGGGGCACCTTCAC
CCCCCAACCCATACCTTTTTGATACCCTGATCTCACACCGAGAGTCTCAACCATAATCTC
CTCTTCAGTCTTCGAATTTTCTCCATCCATCATAGACGATTCCTCCATTTCAGCCATTTT
CTCCTGAAATTTATTATTAATATATGTGAAATTATGGTAAAAGAATAGTTCCAGAATTAT
GGTAAAAGAATTGTTGAGGTGTTGAAACACAAGAGGTTTGGTGCAAAATTCCCTATTTTG
TGTAGAGCAACTTACATATGTCATTTCAGCCTCTTCGTTCACGAATTTATTCACGAATTG
ACCCTTCCTGCGAACATGATTATCCTTCCACAACAGAATCCTACTACGATTAACCTATAA
GGAAAACATATACAATTACTAAGTTTCTTACCTCATCACAATATCCAGTATAGATATCTG
AATTCAAAAATAGAGCACAAATATAGAAATATTAGATTTCAAAAATTGAAAGTACATACC
ATGCTCCGGTAGCGAACAAGGTTTCTGGTCCTTCCTGATGTGTGGGGAAAAAGCATGCAT
TTTCTATTTGCTTTATTGATGGCAGACCTCATCTACATCATAAACAATGAACAACAAATC
AATTTTTTTTATTTAATAAGAACCTTTGGCCAAATATAATCCCAGCATCCATACAATTAT
CTTTTGTCGCAGAAAAAAAGGAGAAACAGAGAGAGTGTTAGAATAAAAAGGCATACCTGA
AAACGATCAGACTCAACGTGCTCACATAGTGCATGCCAACTCTCAACTGTGAGTTCTCTC
CGCGGAATTGAATTTCTAGGTTGAAGCCCCTTGTTCTTCATATTTGTGTGGATCTTGCCC
AACCTATATCTCCAATTTCTGTAATTTGTCTGGAGTGCTTGATCTATTATAGTCTGGATT
CGCCAACTTGACAGGTCTTCAATCTCGAACCATTGCTAGTAATCAAAATATCTCTATCAT
TACTTGTATATATAAAATAAAAAGAAAAACACTTGGCTTATTTGTTCGAGGAGAATAAAA
AACTCACAAACTAAGGTGATTAAAAAGTAAAAATTACCAATAAATTTTCTAGACAAGCTT
GCTTATTTGTACTAGAAACTTCCTTCCACTTTGACACGCCTTCAATATCCGCATGTGCTC
TTACAGTAACCCCAATAGCATCAGCAAACTCTCCGGTATATTGTGTTGGTATTGTCTAAC
AAGGATTAAAAACTAATTTCATCTTCCCATTCTGCGAAACAACATAATAATTAGAGAGTT
AAATTCATAAAACAAGATCTCCAATTTTCACAGATTAGAATATTGTACCCGTTTCACCAT
CTTATCGAGGTGCAGGCCTCTACCCGGACCGTGAGTATTCTGTTTAGGCTTTGATCCTAA
ATAAAAAGTTATTGAAATAGTGTTAATGTTCCCTAAAGATACTAAACTAAATAAATATGT
TCCGCAAAGAATAAAGAATAGAATTAAAAAAAAAAAATGAAACCAAACAACATACCAATG
GTTGTTCCTGCAGATCCTTCTAGAGTTGCTGAGACAGGAATAGATGGAGGCTGACGCACA
CTTAAATTAGGAGATGAAGCAGTGGACCTTAGAGTATTTCCAGCAGTGGTGGATGTGGAA
GTTGACATATTAGATGCTACATATTTGGATTTTGAAGCAGCTGGAATTGGTTGTTTATTA
ACCACTGAAGTGGAAGGCCTGGAAGTTGACGGCTGGAATTTCAAAGTTGGTGGAATTGAT
GCAACATTTCCAGACGTAACATGTACCTGGGGCTGTTTATTAACCACTGTAGTAGAAGGT
CTGGAAGTTGACGGTTGGAATTTCGAAGTTGGTGGAATTGATGCAACATTTCTAGTCGTA
ACATGTACTTGGGGTCGTACAAACCTCTTGAACTTTGGTCCTCCTGACATCTGAAAAAAT
AAAAGACAAAAAGATATCAGTATGCATGATATAAAGATGAAGAGTTAAAGTATATGCTAC
ATATTCAGTAAAGTGTATTACATTGAATAGGCCGAAAAACAGTTAAAAAAGAAAAACCAA
AGCTATAACAGTTCTTGCATATTCTCTTATAACTAATAAAGAATGAGTTAATAAAAATAA
ATCCTCCTCTTCTTCTTCTTCTTCTTGTTCATTTCTTCTTCTTCTTCTTCCTCCTCCTCC
TCTTCTTCTTCCTCCTCTTCTCCCTCCTCCTCATATGATTGTTCAGTATTTTGATTTCTG
GACTTTCTAATGCTTTCAAAGATCATCCTATCAAGCTCTTCAGGTTCTAAATCGTCCCTT
GCCAATTGCCGTCGACCATTCTGCTCACTTGCTTCTTCAATTGGATCAACACGAGTAGAT
TCATATTGCTGAACAACCTCTGCCACAAATTCTTGTTCATTAATGGTTGTTTTGTCCACA
ACATCATTTACAGCTGATTCACAAACATCCCACAAATGCCTATGTGAAACTCTCTGCACA
ACCTTCCAATTTTTACCCAGTTTTGTGTCATTCACATAAAACACTTGTTGTGCCTGGACA
GGTAATATAAATGGTTGCTCCTTGTACCACCTAGAGCTAACATCAACACTAATGAAATGT
TTATCCTCTTGGATAAATTTATTACTTCCAGAGTTGAACCACTCACAATCAAACAAAACC
ACTTTTCTACCACCAATATATGTTAACTCCACAACCTTTCTCAAGAAACCGAAAAACTCA
ACCGGCTTCCTCTTATGAAATCCTCCAACTACAAGGCCACTATTTTGAGTTTTCGTCGTC
TGTCACAATCCCTTGTATGAAACCGGACCTGATTCACCATGCATCCATTATAAAAATCTG
AGCTCAGCCTCGAAGGACCATTTGCTAAAGCCCACAGCTCAGTTGTAATTTTGTCTGCTT
TCTGAAAATGCAACAGACCCATCTACAAGATAATATAATAGTTAGAACAAAGTATTCTGC
TAAAATCATTTAAGTTCAGAGTATAAGTTATCAACTAAAGGAAATAAACTAACATGTTTT
TAAACCACTGTGGGAAGGTGTTTTTCTGCCTTGGGTCAAGTTCATCCATGTTACCCACTT
CTTCTACGGTAATTCCTTGCAATTCAGCTCTATGCATTCTGAAAAGGGCATAACCATTAG
ATTACAAATAAGTTTATGGTTGTCAAGTGTCAACAAAACACTATGTATTACCACAGTGAA
CGGGTACATCATACTGCCTCAATCCAAACATGGAATAACATCATAGTTACTTAAGTTTTA
CTATCAACAGTAACTATCTAAAATCAGCAGCAGGTATATGCCTAAAGATACTGGAACAAA
TCAGTAACTATCTAAAATCAGCAGCAGGTATATGCCCAAAGATACTGGTAGTCATAGTTC
AGACAAAACTCGTAGACATAAGATAACAATAGAGAGAGATAAGTATTTCTTACTTTAGAT
ATGGCTCAACTTCATCACAATTGCACATTATGTACCAATGTGCCTCATCTAATTCTCCTC
TAGTCAAAGAACACTTGGGGCTAACTTCCCCAAATGGACGAGTTCTCATTGAAAATACAG
AAATTCCTTCCTTTTGTTGACCATAGACACCATCAAAATTCCGTGGAAGGCAATTAAATT
TAGTTTCCATGTCCATAGGCAAATGCATAGAAAAAAATGTGATGTTCTCATGCACCAAGT
ATTGTTTAGCGATGGAAGCTTCAGGATGATTTCTATTTCGGATAAAGCCTGTGAGAGATC
CCAGAAACCTAACAATAAGAAGGAAAGTAAACACCGTAGAATCAGAATCATGATACAGTT
AAAATCAGAATCATTGCATCATATATATCTCAAAAATTTCACCTGGCAGAAGCCGCATAA
GATGTTCCTTAGACACAATATATATCTCAAAAATTAAGGCTTGAAATCAAGTAAAAATGC
AAAATAAATCAAGGCCAAATAAAATGTTACTCTGATTTCATACCACAAACAACACAACTT
GTATGAGTACAGATCAGCAGCCAACTTAATAGACCAGAACACAAACTTAATAGACCAGAA
CACAGAATCACAAACTTATTCGACAATCACACGAAGGGTCACAATAAAAACTAACCTTTC
AAAGGGATACATCCAGCGAAATTGCACGGGTCCTGCAATCTTAGCCTCTTGGGCCAAGTG
GACAGATAAATGTACCATAATTGTGAAAAAGTAAGGTGGAAATATCTTCTCCAACTTACA
TAAGATAATTGGTATATTTTCATCCAACCGATTCACATCTTCGTCATACAAGGTCTTGGA
ACATAATTGCTTAAAGTAATCTCCTAATTCACATAAAGCGTCCTTCACTTCCTTGGTCAA
GTAACCACGAATTCCAGCAGGAATTAATCTCTGGAGTAGAATATGACAGTCATGGGACTT
CAGTCGTGATAATTTATCCTTTTTCACACGTGCTGAGATGTTAGAAGCAAACCCATCAGG
AAACTTAACTAACTTCAGAAACTCCAAGACTTCTCCTCTTTCACTTGGGGATAAGGTAAA
ACATGCAGGTGGCCTGAATATTTTATTGCCATTATCTTGCAGGTGCAGCTCCTTCCTAAT
ACCCATATCGCGTAAATCTTCTCTTGCAACATTAGAGTCTTTTGTTTTCCCTTTAATATC
AAATAATGTACCAAGGACATTATCACATATATTCTTCTCAACGTGCATCGGATCGAGATT
GTGCCTTATCTTCAACATGTGCCAATATGGCAGCTGGTAGAAACCACTTATTTTATCCCA
ATTTACTTGATCCGAATCACGCTTTCTTTTACGATTTCCAGCTGCAATGAACACATCTTG
CAATGAATTAAGCCAATACAATATTTCTTCACCTGAAAGTTGTTTAGGTTTCTCTCTATG
CTCCTCTTTACCATTGAAAGTTAAACTTGTTCCTGCGGAATGGGTGTCTTTCGGGTAGAA
AACGACGATGTCCCATGTAACATATCTTTGTTCGTAATTGTGTAGAATCTGTCTCATCAA
GACAGCATGGGCATGCATAATAACCTGATGCTTGCCACCCTGACAAAAGACTATAACCAG
GCAAGTCGCTTATGGTCCACAACAACACAACACGCATTAGAAACTTCTCTTTTGTGAGAG
CATCATAAGTTTCGATCCCAACATCCCATAACTCCTTCAAATCTTCGATTAAGGGCTGCA
AATACACATCAATGTTTTTCCCAGGATAATTTGGGCCCGGTATAAGCAGCGACAACATAA
GAAATGGTTCACGCATACAATCATAAGAGGGCAGATTATAAACAACAAGAATCACGGGCC
AAATGGTATGCGAATTCCCGATGTCTGACCATGGAGTGAATCCATCACATGCTAAACCCA
ACCTAACATTCCGTGCATCCTTAGAAAACTCAGGAAATAGACTATCAAGGTGTTTCCAAG
CCAATGAATCGGCTGGATGTCTTGAGACTCCATCCTCAACTTTGCGTTTATCTTTGTGCC
AACGCATTTTTGCAGCCGTTTTGGTGAATGCAAAGAGTCTTTGCAACCGGGGCTTCAAGG
GGAAGTATCTCAATTTCTTATGAGCAACTCTCTTTATCTTACTACCAAGCGGCTGCCACC
TAGATTCTTTACAAACTGGACACTCGTTCATATTTTCATTCTCTTTCCGGAATAATACAC
AACCGTTCTTACATGCATCAATCTCTTCATACCCTAGTCCAAGTTCACGCAGCTCCTTCT
CAGCACCATATAATGACGATGGAAGAGTTTCACCTTCGGGTAAGATGCCTTTTATAAAGT
CCAGCAGCATACTGAAAGACTTGTCCGTCCATTTATCCTTAACTTTCAGATGTAGCAGTT
TTAGGGTAGCTAAAAGTGATGAAGTCTTACAGCCCGGATATAATTCATTTCTGGCATCAC
CCACCAACTTCTGGAATCTTTCTTCTTCTTCTTCTTCATAGTCGCTGTCCACATGTTGAC
CTTCAACCAATTCATCCAAGTACCCATTGATCGAACACATTAAATTCGACATCATCATTA
CCTTGATCATTTTCCTCATAATCATCAACACCCTCTGTTTCTGCTAGGAACTCATTTATC
TCGTCCTCGGATGGTAAACGAACATCATTATCAGTATCCTCATCATTTATATCAAGCTCT
TTCTCTCCATGATTGTTCCAAAGTACATATGTACGAGAAAATCCATTAACGTGTAAATCC
TTTTCAACCTCATCTATGGGCTTGTAGAATGAGTTGTTGCAAACCACACATGGACATTTA
ATCATGTTATCACTCCCCATCACATTATCAGCAGCAAACATAAGAAAACATTGCACACCA
TCCATATAAGAATTCGAAAACTTATTATGCTCTCTAATCCAACTTTTGTCCATTATCATT
ACGTTGTACCTAAGTAATAAATAGCTCAGGAAAAATATAACTCTGGCAAGTAGCTCAGGC
CTCAAATTAATAAATACCATATAAACATACACAAGTCAAATAATATCTACATAAGTAGAG
GAACAATCTGAAGCCAAATAATAAACATAGATACAAATATTCATCTAGAACTTATCAATA
TACACATACAGAAACGTGCCTAAATCATAAATTTTAGGAGAATTGAACTACATCAAAGTT
TGAAGTACATCATACCTTCTCTAACAAAGGACACAGCACAAACTTCGGATTTGGATAATG
ATTTGATCAAAGCTTGCTCCTCCAGATTCAAACGCAGCCACACTTCACAGTTGCTAGCCG
ATTCAAACGCGCCACCAGATTCAAACGCGCCACCAGATTGAAACACGATCCATCAGACGC
CTATCATTAACCTTCCTTTCCTGAATCTCCGATTGATAGTGTTGTACTGTTGTGTCGTGT
GATTAGATTTAGATCGATCGAAATTAGGGTTTTAGTAAAAGGGATTTTTGTTTGTGACTG
TGGGTTTCTGTCTAAGTAACGAAGAAGATGAAGTTATCTGATTCCCTTTTTAAAACTTAT
CCTTAATTGCAAAAAAGCCCAAAAGTGTTTCTTATTAACAAATATGCCGCCGAATTTTCT
TTCTTTCACAATTTTCAAATTTTATCATCTAAGAGCCTATTTTTCTATGATAATTTATTA
TATTTGTCATTCTAAAATATTTTTATTTGATAATTTAGTCTAAAAATGTCATCGTAAAGT
TTAATTGTGGTGACAAGTTTTAAGGTGCCACCTAATAGTTACTTTTATTTGATAATTTAY
CCTAATGTCATTTAAAACTAGTCATCTAACATCCTTTT

can anyone tell me why it is showing error and where it is wrong..??

Thanks....

[GenoMax]

As the error says you have the character "Y" in the second last line of the contig you have posted above.

You may have to specify option "--ambiguous=iupac" for lastz. That comes with a caveat as explained in the README so be sure to check the explanation for that specific option here: http://www.bx.psu.edu/~rsharris/last...z-1.02.23.html

[amitbik]

Thank U ... GenoMax

[amitbik]

Hi,
I am using lastz with this command but it is running from last 4 days and it is around 73.4 gb now. I am not able to understand why it is still running or i did wrong. Can any body guide me ?

lastz unigenes.fa[multiple] unigenes.fa --notrivial --format=maf --ambiguous=iupac > unilastz.maf

Thanks..

[GenoMax]

Are you searching the "unigene.fa" file against itself? If so how big is that file?

As indicated in the help page for lastz comparing entire chromosomes only takes a few hours. So 4 days seems long.

At this point in time you are too far into the computation to cancel and investigate other options like (--notransition or --step) which are supposed to lower sensitivity and run times. If you are using a cluster you can always start another job with these options and see if second job finishes before the first one.

[amitbik]

Thank U ... GenoMax

ya I am searching the "unigene.fa" file against itself and my file size is 334mb. I am running lastz on my system which is 16gb of ram and 8cpu.

[Bob-Harris]

Hi all,
FAILURE: call to realloc failed to allocate 135087680 bytes, for add_segment

Howdy, Ester,

Sorry for this late reply. I am new to seqanswers today.

Are you still having this problem? If so, please email me so I can figure out why this is happening (you can find my email address in the lastz readme).

That error report is a failsafe and probably means that memory fragmentation is happening. I've never actually seen that error manifest in my own use. My best guess as to the cause is very high repeat content. But that's just a guess.

Bob H
(lastz author)

[Bob-Harris]

I am getting the identical error message when trying to align 454 run with ~8000 reads on an 80kb reference sequence, using:

./lastz brca1.fasta 454run.fna --format=sam --step=10 --seed=match12 --notransition --exact=20 --noytrim --match=1,5 --ambiguous=n --coverage=90 --identity=95

Howdy, David,

I'm very surprised this could be happening with such a small problem. Could you email me directly (my address is in the lastz readme)? I'd like to run the same sequences here so I can figure out how that memory allocation condition is being triggered.

Bob H
(lastz author)

[Bob-Harris]

That error report is a failsafe and probably means that memory fragmentation is happening. I've never actually seen that error manifest in my own use. My best guess as to the cause is very high repeat content. But that's just a guess.

I'm expanding on my earlier answer, since other users may encounter this problem and google leads them to this page.

My best guess when this happens is that the input sequences contain a lot of unmasked repetitive sequence. The simplest (probable) solution is to split the target into several runs, for example, have a separate run for each of some subset of the target sequences. The other possibility would be to softmask repeats the sequences (assuming they aren't already), but this can be time consuming as well.

In detail. Internally, lastz processes one query at a time. It identifies HSPs (high scoring ungapped alignments) between the query and all the target sequences, collects these into a table, and then proceeds to the gapped alignments stage. There is one segment in the table for each HSP. The amount of memory allocated for the table grows as more segments are needed. If allocation fails, a message is generated indicating a memory allocation failure in add_segment.

Even if such a large memory request could be satisfied, this wouldn't really solve the problem. Most likely the program wouldn't finish in a timely fashion.

The cause of such a large number of HSPs is usually something like an un-masked microsatellite, a long sequence with a short repeat period, e.g. AGTAGTAGTAGTAGT. These are problematic since, in the HSP finding stage, every shift by a multiple of the period looks as good as any other. If your lastz parameters allow for high divergence then highly-diverged microsatellites will have a similar effect.

If you are running several queries, it is often just a few queries that contain the problematic element. You can find out which query is being processed at the point of failure by adding --progress to the command line.

Bob H
(lastz author)

[skothap]

Lastz output problem

Hi all,

I am graduate student in bioinformatics.I am working with Lastz, trying to compare two dna sequence files of sparrow genome.

my command is
lastz GCF_000385455.1_Zonotrichia_albicollis-1.0.1_genomic.fa[multiple] Capensis_both_Out.txt.scafSeq --format=difference --output=/net/home/skothapalli/result.fa

output was
lastz: No match.
what does this mean? Can someone help me out!

thanks in advance,
Sam

[Bob-Harris]

re: Lastz output problem

output was
lastz: No match.
what does this mean?

This is apparently a message from your shell. The string "no match" isn't something lastz ever outputs.

I've seen this reported a couple times before, but I've never found a machine that does it. There is more information about that in this thread (search down the page for "no match"):

http://seqanswers.com/forums/archive/index.php/t-20113.html

My second post in that thread (05-15-2014, 07:28 AM) contains a workaround.

my command is
lastz GCF_000385455.1_Zonotrichia_albicollis-1.0.1_genomic.fa[multiple] Capensis_both_Out.txt.scafSeq --format=difference --output=/net/home/skothapalli/result.fa

The command looks fine, but just FYI the output format won't be fasta.

Bob H (lastz author)

[skothap]

Hi Bob,

I ran ls -al 'which lastz' and my output was
skothapalli@bioinformatics:/net/common/data/sparrow_project$ ls -al `which lastz`
-rwxr-xr-x. 1 root root 435735 Jun 17 14:33 /usr/local/bin/lastz

and I am working in the following path:
/net/common/data/sparrow_project

So,do I have to change path somewhere?

thanks in advance,
Sam

[Topulaneus-Hattum]

Well I just typed a longer reply but when I submitted it my browser failed. If this post is successful I will try again.