[sklages]

I'm told (correct me if I'm wrong) that MIRA is one of few assemblers that even outputs ambiguity codes, and even then it can be told not to?
Iain

Yes, it is one of the assemblers that does ... but that's not a drawback but a feature, especially for denovo cDNAs assemblies without reference ...

thanks,
Sven

[imilne]

I was wondering if it would be possible to show IUPAC codes in consensus.
As I frequently use MIRA to assemble EST datasets, I'd prefer the correct IUPAC code over '?' in the consensus ;-)

In the short term, it's unlikely. One of Tablet's original optimizations was to create its own internal alphabet of supported characters. This alphabet was purposely designed to be tiny, mainly so it wouldn't use up much memory. All raw data is translated into a compressed bytestream version of this, allowing us to display data very very quickly. When we moved to caching data on disk, we thought maybe it wouldn't be needed any more, but it turns out it was still useful because the amount of data we need to read and write to disk is subsequently much smaller too (and therefore aids creation/retrieval time), meaning the overhead of using a hard drive over memory isn't that bad.

So for now, we're locked into the way we currently do things, with an alphabet that can't support much more than the basic set of A, C, G, T, and some others we use to help speed up rendering. Ultimately it's currently limited to 16 characters (2^4).

I'm told (correct me if I'm wrong) that MIRA is one of few assemblers that even outputs ambiguity codes, and even then it can be told not to?

Iain

[imilne]

Would it be possible to implement some sort of features track to make it easier to see pileups over exons, etc?

We actively working on that, but it's tricky and time consuming unfortunately, and there isn't much demand for it from people in my group. Tablet is an assembly viewer, and to turn it into a genome browser (which I think it would have to be to display the features properly) isn't something we can really commit to.

Attached, however, is a sneak preview of what we do have working in one of the builds. At best it'll be one feature "type" per track. If stuff overlaps then that'll be tough luck - for this version at least

Iain

[sklages]

I was wondering if it would be possible to show IUPAC codes in consensus.
As I frequently use MIRA to assemble EST datasets, I'd prefer the correct IUPAC code over '?' in the consensus ;-)

cheers,
Sven

[vamin]

Would it be possible to implement some sort of features track to make it easier to see pileups over exons, etc?

[imilne]

Another update today, mainly to fix a few bugs and make things a bit more robust when dealing with BAM files.

- NEW: Implemented a new, hopefully more robust, method of determining the version number for update purposes.
- NEW: Added an option to set Picard's SAMFileReader validation stringency to lenient to suppress SamValidation errors.
- NEW: Added a parse-time option force DNA ambiguity codes to be read as N.
- CHG: Changed the protein translator to treat any base with an N as ok (codon will translate to X) rather than having it skip the base.
- BUG: Fixed a problem copying any of the reverse strand protein translations to the clipboard.
- BUG: Fixed problems relating to Tablet not re-initialising its BAM reader after encountering SamValidation error.

Proper visual support for paired end data (might) be next. Please do let us know what you'd like to see...

Iain

[imilne]

Hi folks,

Thought I'd post a quick message to let you know that Tablet has been updated again. It's now at version 1.10.05.06 (special UK election day version or something ). Changes are as follows:

- NEW: Added a new colouring scheme based on read direction/orientation.
- NEW: The overview displays can now be "subsetted", forcing them to show only an overview of whatever region you define for them.
- NEW: Variant highlighting (in red) can now be turned on or off for the overviews or popups.
- NEW: Added the ability to search for sequence substrings within reads.
- NEW: Added a new graphical layer that performs "read shadowing", highlighting any reads under the a given column or mouse position.
- NEW: Example files (linked to on the web) can now be opened directly from within Tablet.
- NEW: The nucleotide text for each base can now be shown or hidden.
- NEW: Added (optional) support for trimming reads in ACE files based on the QA tags.
- CHG: Redesigned and added additional options to the control ribbon.
- CHG: The ACE parser now ignores base quality information, making it more compatible with badly formatted files.
- BUG: The SAM parser now looks for all possible header tags to determine if a file is SAM or not.
- BUG: Fixed various issues with searching for reads.
- BUG: Fixed some miscellaneous rendering issues when switching between contigs.

As usual, it should auto-update the next time you run it, or you can grab the download manually from http://bioinf.scri.ac.uk/tablet

Iain

[jeny]

Hi Iain,

you are right. The problem is that s_N_export.txt or s_N_sorted.txt files contain chromosome matched (or genome) and not contig name. So, in fact Tablet can't match chromosome (in export or sorted files) to contig names in reference file. I will have a look and return to you when I find a solution.
Any help appriciate !

Jeny

[imilne]

Hi all,

I am working on Illumina GAIIx single read data.
To test Tablet, I used NSTbioinformatics sorted2sam.pl script on my data (s_N_sorted.txt) to generate s_N_sorted.sam (SAM format). That worked fine.
But using s_N_sorted.sam as assembly file and phix.fa as reference file in Tablet, it displayed a window with only reference sequence and 0 read.
The trouble is that in s_N_sorted.sam there are 119,795 reads matched to the reference genome.
There is no info in output.log.
Could you give me some clue to solve this problem ?
Any help appreciate ?

Cheers,
Jennifer

0 reads in Tablet probably means that the contig names assigned to each read do not match the contig names that are in the reference file, ie, Tablet has (probably) seen the reads, but doesn't know what to associate them with.

Have a look at the first few lines of the sam file and see if the contig name for each line (3rd column) matches the name(s) of the any of the reference sequence(s) in the fasta file.

Iain

[jeny]

Hi all,

I am working on Illumina GAIIx single read data.
To test Tablet, I used NSTbioinformatics sorted2sam.pl script on my data (s_N_sorted.txt) to generate s_N_sorted.sam (SAM format). That worked fine.
But using s_N_sorted.sam as assembly file and phix.fa as reference file in Tablet, it displayed a window with only reference sequence and 0 read.
The trouble is that in s_N_sorted.sam there are 119,795 reads matched to the reference genome.
There is no info in output.log.
Could you give me some clue to solve this problem ?
Any help appreciate ?

Cheers,
Jennifer

[lh3]

Another feature you may consider is to add FTP support. Most human alignments are sitting on NCBI ftp. If I want to look at a small region for multiple samples, I would need to download alignments, which is a lot. The best way is to directly open BAMs over FTP without downloading the entire file, like what UCSC is doing now. IGV allows you to open alignments on the IGV server. I know someone is thinking to open BAM from amazon/google cloud.

I am just throwing ideas. Do not take them seriously.

[imilne]

It's update time again, and most importantly, Tablet now has proper, native support for BAM assemblies!

It's worth reading the new section in the help that specifically covers BAM as it obviously works a little differently to all of the other assembly types that Tablet supports (prepare to meet the "bambambar" )

As for the rest of the changes, they're as follows:

- NEW: Added support for indexed BAM assemblies.
- NEW: Added a new navigation bar for moving the viewing window around within a larger BAM assembly.
- NEW: Implemented background (multicore) thread support for calculating auxiliary display data, meaning large contigs can now be viewed instantly.
- NEW: Various implementation changes mean less memory is now used for large contigs.
- NEW: The Open Assembly dialog is now significantly quicker at detecting the assembly type.
- NEW: The page left/right controls now hide when the edges of the display are reached.
- NEW: Added options to the Preferences dialog to enable/disable disk caching (where supported).
- NEW: Loading a very large number of features is now much faster.
- NEW: The bp coordinates (data set/BAM window and current view) are now shown on the overview images.
- CHG: Updated the mismatch code to ignore regions of reads that extend beyond the left or right ends of the consensus.
- BUG: Fixed a problem with files being ignored when passed to Tablet on the command line.
- BUG: Fixed a bug which caused Tablet to not exit correctly when a filter had been applied in the Contigs Table.
- BUG: Fixed multiple issues with AFG parsing related to clr and gap tags.
- BUG: The unpadded read length was no longer being shown in the popup information box.

A few other points:

- The issues with CIGAR parsing remain (for now).

- We have an Eland parser ready to go, we're just waiting on confirmation of a few things before we release it. (We don't use Eland here, so we have no data files ya'see).

- I'd also like to take this opportunity to thank the Picard/samtools folks for their Java API. Without it, proper BAM support in Tablet would probably never have happened, so we (and hopefully lots of Tablet users too) are very grateful to the developers for making it available.

Iain

[GStephen]

Many thanks for your clarification on these points - and speedy response - lh3. This will be very helpful when it comes to expanding our current support for insertions in SAM/BAM.

Gordon.

[lh3]

It should be (i.e. the samtools paper gives the right view):

NNNNNNNNNNNNNN**NNNNNN
      TTAGATAAAGGATA*CTG
        AGATAA*GGATA

Samtools tview does not support padding (P) and thus fails to give the right view. Like tablet, IGV does not show inserted bases at the moment. It is being implemented so far as I know. I guess Gambit is the only viewer that supports padding and insertion (never tried, though). Gap5 should also work with these operations, but it does not directly work with BAM (am I right?).

As to reads standing out of the reference, we are inclined to claiming them as invalid. But it would be good for a tool to be robust to that.

[GStephen]

I'm the other Tablet programmer that imilne mentioned. As imilne said I had previously asked for clarification on how to handle inserts in SAM/BAM on the samtools mailing list, but I got no response there. It would be good if lh3, or somebody else who is relatively close to the SAM/BAM project could clarify the way in which CIGAR insertions should be handled.

Given the following example (adapted from the SAM/BAM format paper) what is the correct output? I've kept the example simple to save space.

ref  - AGCATGTTAGATAAGATAGCTGTGCTAGTAGGCAGTCAGCGCCATGGAT

SAM file:
@HD	VN:1.0	SO:Sorted
@SQ	SN:ref	LN:60
r001	0	ref	7	30	8M2I4M1D3M	*	0	0	TTAGATAAAGGATACTG	!!!!!!!!!!!!!!!!!
r002	0	ref	9	30	3S6M1P1I4M	*	0	0	AAAAGATAAGGATA	!!!!!!!!!!!!!!

The SAM format paper proposes that the result should look something like the following:

ref  - AGCATGTTAGATAA**GATAGCTGTGCTAGTAGGCAGTCAGCGCCATGGAT

r001 -       TTAGATAAAGGATA*CTG
r002 -      aaaAGATAA*GGATA

Whereas Samtools tview 0.1.7 on Windows produces (change marked in bold):

ref  - AGCATGTTAGATAA**GATAGCTGTGCTAGTAGGCAGTCAGCGCCATGGAT

r001 -       TTAGATAAAGGATA*CTG
r002 -      aaaAGATAA*[B]*[/B]GATA

And IGV produces:

ref  - AGCATGTTAGATAAGATAGCTGTGCTAGTAGGCAGTCAGCGCCATGGAT

r001 -       TTAGATAA[B]|[/B]GATA*CTG
r002 -      aaaAGATAA[B]|[/B]GATA

Where | represents an insertion marker. Note also that the reference has not had the two * characters inserted.

I could list a few more viewers as there seem to be subtle differences between the way CIGAR insertions are handled between all viewers.

Would it be possible for somebody to clarify which of the above is the intended outcome. We would really appreciate a clarification so that CIGAR insertions can be displayed as intended in the SAM/BAM format. As an interim measure we have a system in Tablet whereby we mark the insertions as features; which can be navigated to via our features table. Ideally we would like to display the insertions in whichever manner is deemed correct by those who work on the SAM/BAM format. We suspect the correct way to display CIGAR insertions is either the way seen in the SAM format paper, or the way seen in Samtools tview, we would really appreciate confrimation of this though.


On a related note, what is the position on reads which extend beyond the reference sequence at the start or end of the sequence. The Picard API in particular doesn't seem to like reads which either start before the reference sequence, or end after it. Are these reads valid in SAM/BAM or not?

Gordon.