[glados]

Yes it's a good idea to calculate your mean inner distance and I would not trust the flag "properly paired" tophat gives. See this thread.

[cedance]

EGrassi, About the cleanup, Yes! However, its just to remember the next time I look at the code that I am computing the coordinates of the junctions at open intervals .

I just dint want to introduce any variability by mapping and determining inner distance and standard deviations from transcriptome. I don't think picard estimates inner distance this way. I remember using picard initially and I was not satisfied with the mapping.

[EGrassi]

I see, thanks. I did this evaluation with picard (and separately with awk on the bam file, should be similar to your perl) but with alignments done on the transcriptome because it seemed the sensible thing to do to me...maybe I was utterly wrong...ASAP I will perform some tests and report back here.

(the $idist formula could be cleaned removing some -1/+1 I believe , as in $second_mate->start - $first_mate->end -1)

[cedance]

I map it on to the genome. I do not use picard. I wrote a perl script using Bio:B::Sam, with which its very easy to obtain all pairs and their inner distance. I take only uniquely mapped pairs and where quality is >= 20 to compute the -r and --mate-std-dev parameter. It goes something like this:

# $opt_i is your input bam file, bam.bai must also exist in the same name under the same directory
my $sam = Bio:B::Sam->new( -bam => $opt_i );
# get all chromosome ids
my @targets = $sam->seq_ids;
# for each chromosome id
foreach my $seqid (@targets ) {
my $segment = $sam->segment( -seq_id => $seqid ); # get all reads that match
# get iterator to loop over all pairs
my $iterator = $segment->features(-type => 'read_pair', -iterator => 1 );
while ( my $pair = $iterator->next_seq ) {
# fetch 1 pair at a time
my ( $first_mate, $second_mate ) = $pair->get_SeqFeatures;
# check conditions to skip to next pair or not
next if( !defined( $second_mate) );
next if( $first_mate->get_tag_values( "XT" ) ne "U" );
next if( $first_mate->qual < 20 );
# conditions cleared? get inner distance
my $idist = ($second_mate->start - 1) - ($first_mate->end + 1) + 1;

# here, you can just push it to an array and after you get out of the for-loop compute the quantiles and the IQ and the mean and SD.
....
....
}
}

# compute mean and SD by computing IQ and filtering those inner distances that are within Q1 - (Q3-Q1)*2 to Q3 + (Q3-Q1)*2, where Q3-Q1 is the IQ (inter-quartile range).

[arvid]

If you are curious about trying to estimate the mean inner distance and its deviation, then you can take a look at the first reply here. I wrote this quite a while ago on biostars. It is what I have been doing to estimate the -r parameter and --mean-std-dev for quite a while. And with samtools flagstats I get over 75-90% of the reads as properly paired with ALL the data sets I have worked so far.

Just for clarification: do you run the alignment with BWA on the genome or on the transcriptome files? If it's on the genome, isn't it messy to deal with the inner distances for read pairs that span introns? Or does Picard do this for you?

[cedance]

If you are curious about trying to estimate the mean inner distance and its deviation, then you can take a look at the first reply here. I wrote this quite a while ago on biostars. It is what I have been doing to estimate the -r parameter and --mean-std-dev for quite a while. And with samtools flagstats I get over 75-90% of the reads as properly paired with ALL the data sets I have worked so far.

[EGrassi]

It depends on what you are doing downstream, as I said before. Are you looking for new transcripts, or using a quantification scheme where the insert size is used? I'd be especially careful if you are looking for new transcripts/splice variants, for other applications I find this setting to be less critical.

I see, thank you. We would like to start from known transcripts and compare this results with microarray data (and we are in a kind of a hurry to see some first numbers, the dimensions of this dataset as in some way surprised us - and our servers ), only afterwards we will look for new transcripts/splice variants...I hope to be able to compare results with different -r values or at least re-do the alignments with a sensible -r by that time.

[EGrassi]

Maybe you have skewed fragment size distributions (many I've seen are) - do you have wet lab info on that?

Nope, I've got only the fragment size calculated from the bioanalyzer trace (btw I checked and the picard analysis on the alignments gave me -20 as a result which was in accordance with a fragment size of 300 [adapters are 122] which was the first value given to me, but then I checked the precise numbers and they vary between 365 and 327, therefore I am not certain of which should be this "real" -r...).

[arvid]

It depends on what you are doing downstream, as I said before. Are you looking for new transcripts, or using a quantification scheme where the insert size is used? I'd be especially careful if you are looking for new transcripts/splice variants, for other applications I find this setting to be less critical.

[EGrassi]

I got a SD of 37 on a subset of aligned reads so in order to reduce the number of tests I sticked with the default value (20). If I get some cpu time for other tests I will let you know the results...right now as long as I have an ETA of 3 weeks to complete the alignment on my data I will go without -r (as long as I have 1 week of work already done in this way...I know that this does not sound right but when I started to align I had read the other thread here about differences in results with negative values and I didn't have the lab data to confirm my extimated value and as long as someone stated that the -r value is less important in the recent releases of tophat I decided to start without an explicit -r [though I've not understood completely what "less important" means and I don't have the time to check in the source code...])

[arvid]

Maybe you have skewed fragment size distributions (many I've seen are) - do you have wet lab info on that? In that case, I'd definitely choose a bigger --mate-std-dev, I think the default 20 is on the low side for typical distributions, anyway.

[cedance]

Do you also choose a value for --mate-std-dev?

[EGrassi]

Just to report my experience.

without -r (that is with the default value of 50):
samtools flagstat S2_003.th/accepted_hits.bam
28257215 + 0 in total (QC-passed reads + QC-failed reads)
0 + 0 duplicates
28257215 + 0 mapped (100.00%:-nan%)
28257215 + 0 paired in sequencing
14154028 + 0 read1
14103187 + 0 read2
23419378 + 0 properly paired (82.88%:-nan%)
26873012 + 0 with itself and mate mapped
1384203 + 0 singletons (4.90%:-nan%)
0 + 0 with mate mapped to a different chr
0 + 0 with mate mapped to a different chr (mapQ>=5)

with -r -20 (value gotten from analysis of an alignment done with bowtie on a subset of reads and from the wet lab infos):
samtools flagstat r20_S2_003.th/accepted_hits.bam
28386256 + 0 in total (QC-passed reads + QC-failed reads)
0 + 0 duplicates
28386256 + 0 mapped (100.00%:-nan%)
28386256 + 0 paired in sequencing
14218550 + 0 read1
14167706 + 0 read2
17439266 + 0 properly paired (61.44%:-nan%)
27002056 + 0 with itself and mate mapped
1384200 + 0 singletons (4.88%:-nan%)
0 + 0 with mate mapped to a different chr
0 + 0 with mate mapped to a different chr (mapQ>=5)

It is true that in principle it seems better to give tophat the actual value but the difference in the properly paired percentages is relevant and with 100bp reads I doubt that all the properly paired reads in the first case that are missing in the second are wrongly aligned...

[arvid]

Yes, it is a mean, not a threshold, but to me it seems that the 'properly paired' flagging depends on fragments being smaller than the mean added by some multiple of the supplied mate-std-dev. I might be wrong, but it doesn't seem to be actually checking whether fragments are much *smaller* than expected, which would explain the behaviour in your case.

Still I believe it is less important to have a high % of 'properly paired' pairs, than using settings that are close to the actual fragment spectrum - this would lead to more cautious use of the outlier fragments for transcript assembly in the downstream analysis (if you are looking for previously unknown transcripts), or e.g. using quantification tools that model the insert size to assign weights for multi-mapping read pairs...

[vkartha]

So when I say best, I mean that I observed the highest % looking at the 'properly paired' flagstat output for +50.

I read that this 'properly paired' flag in the samtools flagstat output is a measure of how accurate your -r option was for that tophat run in some sense.

The -r option is mean insert size, so we have to provide Tophat with a rough estimate. It isn't really 'allowance' as you put it right? It isn't a threshold where everything less than that will get mapped. That is what I didn't get by your reply.