How to work with a PacBio sra file?

[ymc]

Dear all, I downloaded a 2GB sra file generated by RSII from

http://www.ncbi.nlm.nih.gov/sra/SRX852868[accn]

fastq-dump gave me a 500Mbp fastq file as expected. But I suppose there should be other useful data I can get from the sra file.

Can you tell me what I can do extract other info? I googled but looks like there aren't many discussions on PacBio sra...

Thanks a lot in advance!

[GenoMax]

Submitters may have submitted only the fastq files for the samples. Corresponding record at ENA: http://www.ebi.ac.uk/ena/data/search?query=SRX852868

[ymc]

Submitters may have submitted only the fastq files for the samples. Corresponding record at ENA: http://www.ebi.ac.uk/ena/data/search?query=SRX852868

Thanks for your reply.

But how come Illumina sra usually has file size that is a fraction of # of bases but here the PacBio data is ~4x the # of bases?

[GenoMax]

Interesting problem. There is no "pacbio-dump" program to dump the contents of a pacbio sra file though fastq-dump seems to extract fastq data.

I looked at one of the sra files from your link and used a program ("kar") that is included in SRA toolkit to "extract" a set of directories from the archive. I am not exactly sure what kind (and format) of data was extracted but there are at least 4 directories with these names "CONSENSUS PASSES SEQUENCE ZMW_METRICS" under a directory called "tbl". (Perhaps this is an exploded representation of the HDF file?)

Definitely worth emailing tech support at SRA to find out how to get all the data out in a format that is usable/recognizable (*.bax.h5 and other files). Please post an update here when you hear from them.

[GenoMax]

Just heard back from SRA support.

In cases where the data submitters provide original PacBio files (metadata.xml, *.bax.h5 and *.bas.h5 ) they can be found under the "Download" tab as a "tgz" archive (see an example from the accession that ymc had included above: http://trace.ncbi.nlm.nih.gov/Traces...run=SRR1772703).

Even though sra archive contains all of PacBio hdf5 data, it is not possible to reconstitute the original files since data formatting schema for PacBio hdf5 files is proprietary. As a result there is no "pacbio-dump" utility.

[ndelaney]

PacBio file formats are not proprietary, but are a simple variant of the well-known HDF5 format that is used in a large variety of fields for storing data in a compressed format. They provide significantly more information than an Illumina file so are expected to be larger.

For information about the file formats see:

http://pacificbiosciences.github.io/FileFormats/

For information about working with PacBio hdf5 files, please see:

http://www.pacb.com/devnet/

[GenoMax]

I was paraphrasing the response received from SRA support.

Thanks for links to PacBio file formats.

[ymc]

PacBio file formats are not proprietary, but are a simple variant of the well-known HDF5 format that is used in a large variety of fields for storing data in a compressed format. They provide significantly more information than an Illumina file so are expected to be larger.

For information about the file formats see:

http://pacificbiosciences.github.io/FileFormats/

For information about working with PacBio hdf5 files, please see:

http://www.pacb.com/devnet/

Thank you for your reply.

According to the PacBio doc, there should be two types of reads: 1) Continuous Long Read (CLR); 2) Circular Consensus Sequence (CCS) Read. Also, the CCS reads are called from subreads joined by adapter sequences.

My question is: Is it possible to get CLR reads and CCS reads as well as their subreads from the hdf5 files?

[GenoMax]

Long as you have access to SMRTportal (if you don't have one locally PacBio makes an amazon AMI image available) it is possible to reprocess the data using *.h5 and metadata.xml files.

[rhall]

Unfortunately the documentation is a little misleading, CCS reads used to be calculated automatically, but this hasn't been the case since 2.0. pbh5tools can be used to dump subreads (the same thing as CLR) from the bax.h5 file, but to calculate the CCS reads you need to run a protocol in SMRTportal as GenoMax pointed out.