[sagarutturkar]

Installing SMRTanalysis

Hi All,

We are trying to install SMRTanalysis software from pacbio. We are getting error as below:

File "./smrtpipe.py", line 4, in <module>
import pkg_resources
File "/usr/local/lib64/python2.6/site-packages/distribute-0.6.24-py2.6.egg/pkg_resources.py", line 2707, in <module>
working_set.require(__requires__)
File "/usr/local/lib64/python2.6/site-packages/distribute-0.6.24-py2.6.egg/pkg_resources.py", line 686, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/local/lib64/python2.6/site-packages/distribute-0.6.24-py2.6.egg/pkg_resources.py", line 584, in resolve
raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: pbpy==0.1

Seems like something wrong with python. We updated to python version 2.7 and included correct path in .bashrc files. Still get the same error.

Any suggestions from previous experience?

Thanks

[krittika.sasmal]

Thanks for all your replies. I want to understand the SMRT pipe for running assembly. I understand the BLASR has to be run to align the longreads and CCS reads. And then make a consensus through the make-consensus from amos. However the input of the amos make-consensus is the TIG file. How do we go about genrating from the BLASR output? Please somebody help me on the pipeline.

[rghan]

Thanks for the information, sergek. I will try and take a look at this when I get to the lab in the morning.

[sergek]

I am the developer of pacBioToCA, happy to see interest in the pipeline. If you have only fasta files, the pacBioToCA wiki page includes a section on inputting PacBio RS sequences: http://sourceforge.net/apps/mediawik...o_RS_Sequences

We provide a java utility to convert the fasta data to fastq with uniform quality values (http://www.cbcb.umd.edu/~sergek/PacB...ToFastq.tar.gz). The instructions for using it are at the above link.

[rghan]

Replying to SillyPoint-

We were interested in employing the pacBioToCA script, but the pacBioToCA pipeline expects PacBio RS sequences in fastq format (with sanger (PHRED32) quality values). We were only given an assembled.fasta and a filtered_subreads.fasta.

[jbingham]

The quality scores are phred-style. Insertion QVs give the likelihood that the given base is itself an insertion. Deletion QVs refer to the immediately preceding base, and the most likely deleted base is stored as the DeletionTag in the HDF file. In PacBio data, insertions are the most common error. The QV in the FASTQ is predominated by the insertion QV.

[SillyPoint]

AFAIK, any ascii-encoded Q scores in fastq or SAM files will be encoded Q+33.

See last post for caveats about quality scores, however.

--TS

[krittika.sasmal]

Quality scores in the pacbio .fastq files

Hi, I wanted to know what kind of quality scores are there in a fastq file from pacbio? PHRED 32 /64? or is it Sanger type quality scores?

[SillyPoint]

Actually, the answer to rghan's "question regarding quality scores" is tougher than it looks. First off, it depends on what you want the fastq file to contain.

If you're after the circular consensus reads, that exists as Analysis_Results/<MovieName>.ccs.fastq.

But if it's the individual raw reads you're after, what do you want to see? All the bases from all the reads? Probably not: you can't feed that to an aligner, for example. You probably want the raw reads to be split up into subreads of contiguous sequence, with the adapters removed. And you probably want only productivity-1 reads. I.e., you want the fastq equivalent of the filtered_subreads.fasta file produced by secondary analysis.

pbh5tools won't give you that, I'm afraid. (Nor will my package ). "bash5tools.py --outType fastq --readType Raw" produces a fastq file containing all the bases from all the reads, unfiltered and un-split.

You could extract a fastq file from aligned_reads.sam. But that gives you just what it says: only the sub-reads which secondary analysis managed to align.

The next question is: What do those Q scores mean, anyway?

The bas.h5 file includes 4 separate probability scores for each basecall: substitution, insertion, deletion Q-probabilities, and an overall "QualityValue". The first three are easy to understand, but I've never been clear on what the 4th one represents. That's the score you see in the SAM and pbh5tools files.

I've heard it said that QualityValue is the Q-encoded combination of the first three probabilities. But looking at data, that doesn't appear to be true. (Can't read the code: it's part of primary analysis, not released by PacBio ).

And in any case, what do you make of the deletion probability? That's the prob that this basecall may have been followed (preceded?) by a missed base. That doesn't tell you anything about the validity of the basecall itself.

Perhaps some helpful PacBio person can shed a bit more light on all this.

--TS

[GenoMax]

Current default output of SMRTanalysis is fasta format files as you have noticed. "fastq" format sequence files would be produced as default by a future version of SMRT analysis package but in the mean time you can get quality values from the *.bas.h5 files by using the script PacBio posted here: https://github.com/PacificBiosciences/pbh5tools/

Tom Skelly from Sanger recently posted a set of useful scripts for PacBio here: https://github.com/TomSkelly/PacBioEDA

Apologies if this is a rather naive question, but http://oelemento.wordpress.com/2011/...uence-dataset/ mentioned that PacBio fastq files contain quality scores (c) for each nucleotide in each read. We are not seeing any quality scores in our initial analysis. Any help or suggestions would be greatly appreciated.

[rghan]

question regarding quality scores

Apologies if this is a rather naive question, but http://oelemento.wordpress.com/2011/...uence-dataset/ mentioned that PacBio fastq files contain quality scores (c) for each nucleotide in each read. We are not seeing any quality scores in our initial analysis. Any help or suggestions would be greatly appreciated.

[jbingham]

One more tip: there's also a C++ API to read PacBio HDF files. It's located in the SMRT Analysis source download in

cpp/common/data/hdf/HDFBasReader.h

[jbingham]

The pacbio.spec file is specific to Celera Assembler. PacBio's pipeline doesn't generate it. Examples are available for SGE

http://www.cbcb.umd.edu/~sergek/PacBio/data/sampleData/pacbio.SGE.spec

and for high memory instances

http://www.cbcb.umd.edu/~sergek/PacBio/data/sampleData/pacbio.spec

Once you've got a working spec file, you should be able to use it for all analyses.

If you use the error corrected reads as your starting point, you can run the SMRT Portal GUI directly, as @GenoMax suggested. You cannot yet do error correction through the GUI. You'll have to do it from the command-line.

[GenoMax]

Krittika,

We discovered that the CLI for the PacBio SMRT analysis software is not fully supported by PacBio (at least that was our experience). We were trying to use the CLI and ran into problems that only the developers could answer. But we never received satisfactory answers. You also need to use some settings xml files that are difficult to reproduce by hand so I would advise staying away from the CLI for the current version of SMRT analysis.

That said, SMRTanalysis software does work through the SMRTPortal web interface they provide (which has its own problems since there is no good security model but if you are the only user then it may not be an issue). So your best bet may be to install that and move forward.

You can set up some of the hybrid assembly through the SMRTportal interface (we are in the process of trying it now). They do recommend having a cluster to run this on so I hope you have access to one and are planning to do this work there.

Thank you for all your answers. Can anybody tell me the pipeline to be followed to generate ths error corrected CLR reads. I have downloaded the SMRT pipe.
Moreover how are the filtered reads generated? Help me out with the SMRT analysis. I downloaded E.coli raw reads from DevNet. However there seem to be several bas.h5 files. Do I combine them and proceed.
What are the parameters BLASR takes. Can anybody help?

[krittika.sasmal]

@jbingham- Thanks loads. I could find that there is a pacBio.spec file that is required. It is not there for any of the reads downloaded from PacBio DevNet. Is it always supplied with the data, as is written in the manual (infact I doubt it..).
can you shed some more light on it?