Help with parallel transcriptome assembly.

[matthew.christenson]

Hello,

I am trying to assemble an insect (a leafhopper) transcriptome. The raw data is 372,756,990 reads, 150 bp in length. Previously, my lab assembled a transcriptome, using trimmed reads (using prinseq), on a local server using ABySS-PE (k=25, n=8). This assembled transcriptome contains more than 45.9 million sequences with a contig mean 58 bp and N50 59 bp, and took about 7 days to complete its assembly.

Obviously we would like to improve our assembly but it is not reasonable continue assembling on our local server (because of the amount of time it takes for each assembly).

Our lab has access to a supercomputer known as Tusker (http://hcc.unl.edu/tusker/index.php and http://hcc.unl.edu/hcccreditedit/faq.php?tp=Tusker). Tusker requires users to submit jobs using Torque (a variant of PBS) but ABySS-PE is incompatible with Torque.

So, any suggestions on a parallel program that is compatible with Torque?

Thank you very much for your help.

-Matthew

[westerman]

I would like to stand your question on its head and ask, "why do you believe that ABySS-PE is incompatible with Torque"? I do not have problems running it on PBS so I am wondering if there is something about Torque that is different or that you do not understand.

[westerman]

My comment about ABySS-PE and Torque not withstanding, you could try Trinity for transcriptome assembly. Parts of that program are parallel-within-a-node, parts are single-cpu and parts are parallell-within-a-cluster.

Another option is to do 'digitial normalization' (program: diginorm) first in order to reduce the overall number of reads without reducing (much) the complexity of the transcriptome.

[westerman]

You asked, in private email, what commands I use to run ABySS on multiple machines. I am not a PBS expert and as far as I can tell every PBS installation seems different but here is a run I just did. Our installation uses openmpi. First the script file:

cd $PBS_O_WORKDIR

module load abyss/1.3.4-maxk96

abyss-pe np=48 mpirun=mpirun name=kmer80 s=202 n=10 k=80 lib='lib_A' lib_A='R1.fastq R2.fastq' mp='lib_A'

Which I then submitted using the following:

 qsub -V -l select=6:ncpus=8:mpiprocs=8,walltime=03:50:00 batch.file

The overall meaning of the above:
1) I tell abyss-pe to use MPI with 48 processors
2) I tell the queuing system to I wish use 8 processors on each of 6 nodes for up to 3 hours and 50 minutes.

Note that, as per the ABySS manual, I do not start up ABySS via mpirun but rather the reverse -- ABySS starts up mpirun for me.

The queuing system will make up a PBS_NODEFILE which ABySS will use to determine which nodes it will run on.

Hope this helps. Oh, the output I got from the above included:

lrwxrwxrwx 1 westerm hort 11 Aug 27 11:02 kmer80-scaffolds.fa -> kmer80-7.fa
-rw-r----- 1 westerm hort 34357518 Aug 27 11:02 kmer80-7.fa

[Melissa]

I can't help but notice that there's something strange about the transcriptome assembly. Contig mean 58 bp and N50 59 bp?