removing N inserts

[yaximik]

I am trying to build a comprehensive database of prokaryotic (bacteria and archea) and fungal genomes to be used for screening ancient DNA reads for contamination. What I found unfortunately that many of genomes in NCBI or EMBL databases have a lot of poly-N inserts, which obviously need to be eliminated. This can be done either by removing inserts from each FASTA record, which may be difficult, or by splitting records at poly-N inserts and trimming Ns from the ends. Is there a tool/sctipt to do this? Alternatively, I may have to abandon genomes and just concatenate GenBank relevant records, but I first will have to extract FASTA from them. Any advice?

[swbarnes2]

It's not obvious to me that they have to be taken out. Reads just won't align there, that's all.

You could use sed to get rid of all the N's, or write a script in something to trim out sequences of Ns that are more than a certain length.

[yaximik]

Well, dowloaded data files need to be preprocessed by BWA-SW to make databases for local install of DeconSeq, and the author removed Ns by splitting, as BWA-SW replaces Ns with either of A, G, C, T at random (citing the paper). But it does not say in the paper how this was done...

[jimmybee]

Why do you need to remove the poly-N regions? These are gap regions that could be useful

[maasha]

I am also not convinced that you should remove N's, but if you must, you can with Biopieces (www.biopieces.org):

read_fasta -i in.fna | transliterate_seq -d 'nN' | write_fasta -o out.fna -x

[yaximik]

Wow, i did not think biopieces can do this. Gotta try!

[maasha]

@yaximik

For more finegrained control you can use substitute_vals to remove blocks of N's longer than 25:

read_fasta -i in.fna | substitute_vals -k SEQ -s 'N{25,}' -r '' -ig | write_fasta -o out.fna -x