Tweet
A reference-genome guided compression stratigy for storing mapped reads
Recent days, I encounter space problem for storing mapped reads. Then, I came up with some new ideas.
As developing of sequencing more and more reads would be generated. A very significant property is that all reads came from a genome with very few errors. A wise method to store mapped reads could be
1. Use reference genome as guide.
2. Store the mapped position of every read and the error information
3. Use difference between quality scores to store quality score under the assumption that adjacent quality score would not change too much. This part could be lossy.
4. Discard or keep other useless fields
5. Sort the reads by genome position, then chromosome ID could be saved and the mapped coordinates could be saved by storing differences only.
If we just keep the mapped location and error information, each read may costs only several bytes because
1. chromosome IDs for each read costs 1bit, encoding whether current read has changed chromosome/strand if we fix an order beforehand
2. relative position for each read cost 1byte, allowing 128 difference (e.g. for DNA sequencing, suppose the whole genome are covered)
3. Error information: k mismatches need k * 4 bits if do not allow indels.
The above 3 information is actually most frequently used. Such a reference-genome guided compression strategy should be much effective than SAM/BAM format. If there is no effective reference genome, a genome based on reads could be constructed using Bruijn graph like idea.
How do you guys think?
As developing of sequencing more and more reads would be generated. A very significant property is that all reads came from a genome with very few errors. A wise method to store mapped reads could be
1. Use reference genome as guide.
2. Store the mapped position of every read and the error information
3. Use difference between quality scores to store quality score under the assumption that adjacent quality score would not change too much. This part could be lossy.
4. Discard or keep other useless fields
5. Sort the reads by genome position, then chromosome ID could be saved and the mapped coordinates could be saved by storing differences only.
If we just keep the mapped location and error information, each read may costs only several bytes because
1. chromosome IDs for each read costs 1bit, encoding whether current read has changed chromosome/strand if we fix an order beforehand
2. relative position for each read cost 1byte, allowing 128 difference (e.g. for DNA sequencing, suppose the whole genome are covered)
3. Error information: k mismatches need k * 4 bits if do not allow indels.
The above 3 information is actually most frequently used. Such a reference-genome guided compression strategy should be much effective than SAM/BAM format. If there is no effective reference genome, a genome based on reads could be constructed using Bruijn graph like idea.
How do you guys think?
Comment