[arvid]

It depends on what you are doing downstream, as I said before. Are you looking for new transcripts, or using a quantification scheme where the insert size is used? I'd be especially careful if you are looking for new transcripts/splice variants, for other applications I find this setting to be less critical.

[EGrassi]

Maybe you have skewed fragment size distributions (many I've seen are) - do you have wet lab info on that?

Nope, I've got only the fragment size calculated from the bioanalyzer trace (btw I checked and the picard analysis on the alignments gave me -20 as a result which was in accordance with a fragment size of 300 [adapters are 122] which was the first value given to me, but then I checked the precise numbers and they vary between 365 and 327, therefore I am not certain of which should be this "real" -r...).

[EGrassi]

It depends on what you are doing downstream, as I said before. Are you looking for new transcripts, or using a quantification scheme where the insert size is used? I'd be especially careful if you are looking for new transcripts/splice variants, for other applications I find this setting to be less critical.

I see, thank you. We would like to start from known transcripts and compare this results with microarray data (and we are in a kind of a hurry to see some first numbers, the dimensions of this dataset as in some way surprised us - and our servers ), only afterwards we will look for new transcripts/splice variants...I hope to be able to compare results with different -r values or at least re-do the alignments with a sensible -r by that time.

[cedance]

If you are curious about trying to estimate the mean inner distance and its deviation, then you can take a look at the first reply here. I wrote this quite a while ago on biostars. It is what I have been doing to estimate the -r parameter and --mean-std-dev for quite a while. And with samtools flagstats I get over 75-90% of the reads as properly paired with ALL the data sets I have worked so far.

[arvid]

If you are curious about trying to estimate the mean inner distance and its deviation, then you can take a look at the first reply here. I wrote this quite a while ago on biostars. It is what I have been doing to estimate the -r parameter and --mean-std-dev for quite a while. And with samtools flagstats I get over 75-90% of the reads as properly paired with ALL the data sets I have worked so far.

Just for clarification: do you run the alignment with BWA on the genome or on the transcriptome files? If it's on the genome, isn't it messy to deal with the inner distances for read pairs that span introns? Or does Picard do this for you?

[cedance]

I map it on to the genome. I do not use picard. I wrote a perl script using Bio:B::Sam, with which its very easy to obtain all pairs and their inner distance. I take only uniquely mapped pairs and where quality is >= 20 to compute the -r and --mate-std-dev parameter. It goes something like this:

# $opt_i is your input bam file, bam.bai must also exist in the same name under the same directory
my $sam = Bio:B::Sam->new( -bam => $opt_i );
# get all chromosome ids
my @targets = $sam->seq_ids;
# for each chromosome id
foreach my $seqid (@targets ) {
my $segment = $sam->segment( -seq_id => $seqid ); # get all reads that match
# get iterator to loop over all pairs
my $iterator = $segment->features(-type => 'read_pair', -iterator => 1 );
while ( my $pair = $iterator->next_seq ) {
# fetch 1 pair at a time
my ( $first_mate, $second_mate ) = $pair->get_SeqFeatures;
# check conditions to skip to next pair or not
next if( !defined( $second_mate) );
next if( $first_mate->get_tag_values( "XT" ) ne "U" );
next if( $first_mate->qual < 20 );
# conditions cleared? get inner distance
my $idist = ($second_mate->start - 1) - ($first_mate->end + 1) + 1;

# here, you can just push it to an array and after you get out of the for-loop compute the quantiles and the IQ and the mean and SD.
....
....
}
}

# compute mean and SD by computing IQ and filtering those inner distances that are within Q1 - (Q3-Q1)*2 to Q3 + (Q3-Q1)*2, where Q3-Q1 is the IQ (inter-quartile range).

[EGrassi]

I see, thanks. I did this evaluation with picard (and separately with awk on the bam file, should be similar to your perl) but with alignments done on the transcriptome because it seemed the sensible thing to do to me...maybe I was utterly wrong...ASAP I will perform some tests and report back here.

(the $idist formula could be cleaned removing some -1/+1 I believe , as in $second_mate->start - $first_mate->end -1)

[cedance]

EGrassi, About the cleanup, Yes! However, its just to remember the next time I look at the code that I am computing the coordinates of the junctions at open intervals .

I just dint want to introduce any variability by mapping and determining inner distance and standard deviations from transcriptome. I don't think picard estimates inner distance this way. I remember using picard initially and I was not satisfied with the mapping.

[glados]

Yes it's a good idea to calculate your mean inner distance and I would not trust the flag "properly paired" tophat gives. See this thread.