Issues in running sff_extract

[yy01]

Hi, all,

I encountered an error when running sff_extract to convert an sff file (~1.5GB) to fasta, fasta.qual and xml. The error message is:

File "/usr/local/bin/sff_extract", line 1505, in ?
sys.exit(main())
File "/usr/local/bin/sff_extract", line 1492, in main
extract_reads_from_sff(config, args)
File "/usr/local/bin/sff_extract", line 1041, in extract_reads_from_sff
for seq_data in sequences(fileh=sff_fh, header=header_data):
File "/usr/local/bin/sff_extract", line 255, in sequences
fposition=fposition)
File "/usr/local/bin/sff_extract", line 184, in read_sequence
data=data, byte_padding=8)
File "/usr/local/bin/sff_extract", line 74, in read_bin_fragment
read = struct.unpack('>' + item[1], buffer)
MemoryError

Please, anyone knows what caused the error, and how to solve the issue?
Thanks!

[maubp]

My guess is the SFF file is corrupted. Try downloading it again from the sequencing center.

Have you tried looking at the SFF file with any other tools? e.g. The Roche software, or Biopython?

[yy01]

You are right, maubp. I checked the sff file via Biopython, and it seems to be a corrupted one.
Thanks!

[maubp]

The Biopython SFF code was originally based on Jose's sff_extract but reworked quite a bit - I'd hope it gave a more helpful error message. Does Bio.SeqIO.index() work on it? If so you may be able to recover most of the reads.

Alternatively, if you can't solve the problem by redownloading the file (which I strongly recommend trying!), you might try frecover from flower - it claims to be able to be able to recover corrupted SFF files. I've not tried it myself (maybe I have been lucky with my SFF files?)

http://blog.malde.org/index.php/flower/

http://blog.malde.org/index.php/2010/07/20/updated-software-versions-available/