How to assembel more than two million ESTs sequnces?

[robertorun]

There is a project which need a referrence sequences, so i collect all the ESTs sequences and four 454 datasets from NCBI and modify to fasta format.

I tried to cluster and asemble with tgicl, but it's failure. My server have only one cpu with 4 core and 4GB RAM.

Who can tell me how to assemble these sequences? more detail more well!

best regards!

[Jose Blanca]

We had exactly the same problem and we developed a software to clean and prepare the assembly for mira. You could take a look at it. If you find it interesting it I recommend you to install the git version, it is much improved and we are close to release a new version.
If you have any questions we will be happy to help you.
Best regards.

[robertorun]

Thanks for Jose Blanca's answer!

I tried to insall MIRA ,but it's also failure. Becuase the OS is fedora7, and the latest version of python and boostlib under this OS is 2.5 and 1.3 respectively.

The infomation of install MIRA is following:

checking for boostlib >= 1.35.0... configure: error: We could not detect the boost libraries (version 1.35 or higher). If you have a staged boost library (still not installed) please specify $BOOST_ROOT in your environment and do not give a PATH to --with-boost option. If you are sure you have boost installed, then check your version number looking in <boost/version.hpp>. See http://randspringer.de/boost for more documentation.

[Jose Blanca]

Yes, you really requiere a more modern libboost. We have had that problem in the past. You might try to install the latest bost, talk to your sys admin.

[robertorun]

I have install MIRA 3 succesully and run it. But ,when I rum NGS_backbone , it's error.

python ./backbone_create_project.py -p temp001

Traceback (most recent call last):
File "./backbone_create_project.py", line 21, in <module>
from franklin.backbone.create_project import create_project
File "/usr/local/lib/python2.6/site-packages/franklin/backbone/create_project.py", line 26, in <module>
from configobj import ConfigObj
ImportError: No module named configobj

[Jose Blanca]

Hi, thanks for trying out ngs_backbone.

To be able to install it you need some python libraries, configobj is one of them. I've updated the documentation to clarify that point. Take a look at:

http://github.com/JoseBlanca/franklin/blob/master/doc/source/introduction.rst

Best regards

[robertorun]

I have installed the configobj successfully, but when i try to install pysam-0.2 and psubprocess-0.1.0, the same error is following

pysam-0.2]# python ./setup.py install
Traceback (most recent call last):
File "./setup.py", line 36, in <module>
from Pyrex.Distutils import build_ext
ImportError: No module named Pyrex.Distutils

********************

psubprocess-0.1.0]$ python ./setup.py install
Traceback (most recent call last):
File "./setup.py", line 60, in <module>
from setuptools import setup
ImportError: No module named setuptools

[robertorun]

But when i execute the 'python ./backbone_create_project.py -p temp001', it's successed!

[robertorun]

python ngs_backbone_analysis.py -a temp001
python: can't open file 'ngs_backbone_analysis.py': [Errno 2] No such file or directory
[root@cotton-dell scripts-2.6]# python backbone_analysis.py -a temp001
Traceback (most recent call last):
File "backbone_analysis.py", line 23, in <module>
from franklin.backbone.backbone_runner import (do_analysis,
File "/usr/local/lib/python2.6/site-packages/franklin/backbone/backbone_runner.py", line 29, in <module>
from franklin.backbone.annotation import DEFINITIONS as annot_defs
File "/usr/local/lib/python2.6/site-packages/franklin/backbone/annotation.py", line 25, in <module>
from franklin.backbone.analysis import Analyzer
File "/usr/local/lib/python2.6/site-packages/franklin/backbone/analysis.py", line 28, in <module>
from franklin.seq.readers import guess_seq_file_format
File "/usr/local/lib/python2.6/site-packages/franklin/seq/readers.py", line 28, in <module>
from Bio import SeqIO
ImportError: No module named Bio

[Jose Blanca]

You need some python packages to install it.

The Bio error is due to the lack of Biopython, you have to install it.

pysam requires the pyrexpackage.

psubprocess depends on a setup tool, but I have changed that now to avoid that requirement, just download the git version.

http://github.com/JoseBlanca/psubprocess

The easiest way to install everything is to use easy_install. In debian/ubuntu do:
sudo apt-get install easy_install
After that just do:
sudo easy_installl biopython
sudo easy_install pyrex
sudo easy_install psubprocess setup.py
And the rest.
Best regards.

[robertorun]

My OS is fedora 7, the biopython and pyrex is already intalled.

Jose ,would you like to send the modified psubprocess package to me? Because of there much lilltle files. My email is [email protected] .

[Jose Blanca]

Do you have python2.6 installed in FC7? If you have installed python2.6 outside the package manager and you use the package manager to install pyrex you are installing pyrex in the python provided by Fedora, not in python 2.6.
You can get the latest psubprocess here:

http://github.com/JoseBlanca/psubprocess/archives/master

[robertorun]

The version of Python is 2.6.5. It's downloaded from http://www.python.org/ and installed. The biopython is installed bu the following command:

yum install python-biopython

[ydj@cotton-dell estdemo2]$ python
Python 2.6.5 (r265:79063, May 24 2010, 16:24:13)
[GCC 4.1.2 20070502 (Red Hat 4.1.2-12)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

If you have installed python2.6 outside the package manager and you use the package manager to install pyrex you are installing pyrex in the python provided by Fedora, not in python 2.6.

I can't catch your means!

[Jose Blanca]

Yeah, you have the same problem with biopython. Every python version has it own libraries, so installing biopython (or pyrex) with yum install Biopython in your python2.5 copy, but not in python2.6.
The easiest way to solve that would be:
- Dowload the setuptools

http://pypi.python.org/pypi/setuptools

- run the python2.6 setup.py install for the setuptools. Mind the 2.6!!!

- Use easy_install to install everything.

sudo easy_installl biopython
sudo easy_install pyrex
sudo easy_install psubprocess setup.py
etc...

[robertorun]

I have installed the biopython and pyrex after downloading from web, then excute ngs_backbone, the following is thas,but there is backbone.conf file under the temp001 fold.

python ./backbone_analysis.py -a temp001
Usage: backbone_analysis.py [options]

backbone_analysis.py: error: Settings file path is mandatory if backbone.conf is not found