[Brian Bushnell]

BBMap needs a fairly large amount of memory dependent on kmer size, regardless of how small the reference is. You can estimate it like this:

C:\temp\ecoli>java jgi.AssemblyStats2 in=ecoli_K12.fa k=13
A       C       G       T       N       IUPAC   Other   GC      GC_stdev
0.2462  0.2542  0.2537  0.2459  0.0000  0.0000  0.0000  0.5079  0.0000

Main genome scaffold total:             1
Main genome contig total:               1
Main genome scaffold sequence total:    4.640 MB
[I](stuff)...[/I]

[B]BBMap minimum memory estimate at k=13:     [COLOR="DarkRed"]-Xmx900m[/COLOR]     (at least 1000 MB physical RAM)[/B]

So, it looks like K=13 with a tiny genome like E.coli still needs 900MB! Let's try K=12:

C:\temp\ecoli>java jgi.AssemblyStats2 in=ecoli_K12.fa k=12
A       C       G       T       N       IUPAC   Other   GC      GC_stdev
0.2462  0.2542  0.2537  0.2459  0.0000  0.0000  0.0000  0.5079  0.0000

Main genome scaffold total:             1
Main genome contig total:               1
Main genome scaffold sequence total:    4.640 MB
[I](stuff)...[/I]

[B]BBMap minimum memory estimate at k=12:     [COLOR="DarkRed"]-Xmx480m[/COLOR]     (at least 540 MB physical RAM)[/B]

That's more reasonable; K=12 will work with -Xmx480m. Note that you must specify K both when building the index AND mapping.

[mhgseq]

out of memory error

even with a very small ref and input file, I got the out of memory error. Would you check what is happen?


bbmap.sh in=~/Documents/align/test.fa out=~/Documents/align/outputu.sam

java -Djava.library.path=/home/local/bbmap/jni/ -ea -Xmx700m -cp /home/local/bbmap/current/ align2.BBMap build=1 overwrite=true fastareadlen=500 in=/home/Documents/align/HLAtest.fa out=/home/Documents/align/outputu.sam
Executing align2.BBMap [build=1, overwrite=true, fastareadlen=500, in=/home/Documents/align/test.fa, out=/home/Documents/align/outputu.sam]


BBMap version 35.51
Retaining first best site only for ambiguous mappings.
Set genome to 1

Loaded Reference: 0.023 seconds.
Loading index for chunk 1-1, build 1
Generated Index: 0.817 seconds.
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at align2.BBIndex.analyzeIndex(BBIndex.java:106)
at align2.BBMap.loadIndex(BBMap.java:410)
at align2.BBMap.main(BBMap.java:32)

[Brian Bushnell]

You can't determine copy number of genes from RNA-seq data, as the expression is not really determined by the copy-number (and is highly variable anyway). The best way is probably to map WGS DNA reads to the genome and then use bedtools to extract coverage over coding regions, and look for regions with different-than-expected coverage. This will remove a lot of the bias caused by mapping DNA reads to a transcriptome. Also, note that exome-capture will not yield useful quantitative results, either, due to extreme bias, without extensive calibration for a given kit.

Alternately, it would probably be a lot cheaper to run a SNP chip, or qPCR, or a probe array, or something like that, rather than doing sequencing. I can't remember which one is best for this but at least one is designed to do copy-count quantification without sequencing, across a genome.

[JamesJoyce]

Update: Results still have high standard deviations, even after only keeping the longest isoforms of the sequences found in the reference transcript.

I really like all the features BBMap holds, and as such I would like to keep using BBMap for a foreseeable future. Is there any way I could use BBMap to do this sort of analysis (determining copy number of certain genes within an individual organism), but instead by using RNA-Seq read sequences? Thanks again for your time guys.

[JamesJoyce]

Thank you all for your replies. I will be trying some of the things you have all suggested.
As for running samtools depth with BBMap's outputs, I get 0 coverage and Standard Deviation values. I first convert BBMap's sam output to bam, and then proceed to sort it using samtools. It is then that I try using samtools depth.

Edit:
I tried using one read to verify if there was a problem with the random selection of sites for ambiguously-mapped paired reads. However, the results were very similar, leading me to think that I should be focusing on extracting the longest isoform per gene and seeing how that looks like. However, I am unable to produce coverage and STDV directly from BBMap, as it tells me that "hist=" is an unknown command. Thank you again Brian.

[GenoMax]

It is a strange analysis and with all those caveats going to be of limited real world use for eukaryotic genomes.

[Brian Bushnell]

Well... it is kind of a strange analysis... and a normal transcriptome file, with many isoforms per gene, will not very useful. But, if you had exactly one transcript per gene in your reference, then theoretically, genes with 2 copies should get double the coverage of genes with 1 copy, if you map uniform WGS data. Using the "local" flag would be helpful because wherever reads go into introns they will not match the transcriptome. Also, pseudogenes will cause confusion. So I'm not sure how much signal there will be compared to the noise, but with those assumptions (one transcript per gene, local alignments, fairly uniform coverage, no pseudogenes), it should work.

[GenoMax]

@Brian: How is this analysis is going to give information about number of copies of a gene to @DarthMapper/@JamesJoyce? Am I missing something obvious?

[Brian Bushnell]

First off, you should expect a high standard deviation in this scenario. If one gene has 10 isoforms and another has 1 isoform, the gene with 1 isoform will have (on average) 10x the coverage of the isoforms from the gene with 10, when ambiguously-mapped reads are placed randomly. You need to use exactly one isoform per gene (preferably the longest).

However, it's also possible that there's a problem with the random selection of sites for ambiguously-mapped paired reads. Would you mind trying this (mapping with read 1 only):

bbmap.sh build=1 in1=catreads/read_1.fastq ambig=random local=t out=mRNA.sam covstats=covstats.txt hist=histogram.txt

Note that bbmap can internally generate coverage information and does not need a second pass from pileup. As for why samtools depth is not working, I have no idea. Does it give some kind of error message?

[JamesJoyce]

Thanks for your reply Brian.
I am using BBMap to align WGS reads to a set of mRNA sequences extracted from the organism's NCBI transcriptome. This is being done with different organisms, such as mammals, reptiles, and fungi. So far, I have tested a few mammals, including Felix Catus (cat) and Rattus Norvegicus (Rat). Again, I was instructed to find the copy number of certain genes within a single organism. My PI told me to map WGS sequences to the transcriptome's mRNA sequences as reference before mapping to individual gene sequences, but we stopped once we saw the results.

The results for both mammals are very similar, so I will only post one example

Here is the commandline I am using and the results:

Index Build
bbmap.sh ref=mRNA.fa build=1

Alignment
bbmap.sh build=1 in1=catreads/read_1.fastq in2=catreads/read_2.fastq ambig=random local=t out=mRNA.sam

Coverage and Standard Deviation:
pileup.sh in=mRNA.sam out=covstats.txt hist=histogram.txt

P.S. I have tried to use samtools depth to calculate coverage and STDV with BBMap's sam outputs but it does not let me - even when I have converted the sam's to bam, and sorted the bam. How can this be achieved?

CoverageResults
Set USE_COVERAGE_ARRAYS to true
Set USE_BITSETS to false
Note: Coverage capped at 65535

Average coverage: 15.29
Standard deviation: 384.14
Percent scaffolds with any coverage: 99.81
Percent of reference bases covered: 69.32

Is there any way to lower that STDV?

[Brian Bushnell]

Hi James,

Can you give me the exact command line you are using?

Also, the complete output printed to the screen would be helpful, as well as the name of the organism.

[JamesJoyce]

Hello Brian;
I am trying to do something similar to what DarthMapper was suggesting.
My PI had instructed me to determine the copy number of certain genes in a single organism by mapping WGS reads to said gene sequences as reference. I then am supposed to compare coverage values of these individual genes with that obtained from mapping the WGS reads to the whole set of mRNA sequences present in the studied organism's transcriptome. The problem is that I am also getting strange standard deviations when using BBMap.
What is wrong with my methodology?

[GenoMax]

One would need BBMap to create the alignment file that can then be used with bedtools

[Brian Bushnell]

I think you can do that with Bedtools and a gene annotation file.

[DarthMapper]

Well, usually we map the WGS reads to the genome assembly for quality testing. But, how exactly would we extract the information corresponding to the transcriptome? Would I be able to use BBMap for this?