Extract k-mers from a FASTA file

[Wizard50]

Hi everyone,

I'm looking for a library in C which would allow me to :

- Extract the first x k-mers of a FASTA file (put them in a buffer)
- Make some treatments on these k-mers
- Extract the next x k-mers of a FASTA file (put them in a buffer)
- Make some treatments on these k-mers
- And so on, until the end of the FASTA file.

Basically, the goal is to read a FASTA file by chunks of x k-mers, I don't want to load a complete sequence in memory (to extract the k-mers after) especially if this one is very long.
I could do this code by myself but I'm pretty sure this was already implemented somewhere so if you have a clue where I can find this, it would be very nice to tell me.

Thank you a lot for your help.

Best regards.

[TiborNagy]

Or you can use wordcount in Emboss.

[Wizard50]

First of all, thank you for your answer.

The thing with wordcount in EMBOSS, like Jellyfish or DSK, is that they extract all unique words of size k with their counts.
In my case, I don't need the counts and I don't want to read only unique k-mers. If the k-mer x appears y times in my FASTA file, I want to read it y times where it appears (the order is important).

Furthermore, I also want to use a buffer because I never want to use more memory for extracting the k-mers that the size of this buffer. In these tools for counting k-mers, I can parameterize the size of the memory (RAM or disk) I want to use (most of the time) but I cannot make some treatments on the k-mers extracted when the memory I had allocated is full.

[wokai001]

I have created an R package which can do a variety of k-mer counts on fasta and fastq. The C library is open, so you can insert specialized actions at specific points. The only drawback is that the library starts to get slow when k>10.

You can download it from R-forge:
install.packages("seqTools", repos="http://R-Forge.R-project.org")

Additionally, I have a manuscript in preparation because I found batch effects in fastq files by clustering k-mer counts:

http://arxiv.org/abs/1405.0114

The C code may be a bit difficult to understand because I keep two sequence arrays (due to work on compressed files and for skipping of newlines). It works sequentially, so memory consuption mainly depends on k. Just contact me if you have questions. Any feedback would be great.

Wolfgang