gene overlap

[wrox]

Hello,
I'm a newbie to awk and trying to teach myself to do basic lookups in sequencing data files from command line. I have a file with two columns which correspond to start and stop positions of genes. I want to print the lines (genes) that have overlap with other genes in the file based on the start and stop positions (columns 1 and 2) so that I can simply see which genes positions have overlap. Are there any suggestions for an awk based approach to quickly accomplish this?

I tried the following with no luck

awk '{prev=$0; b1=$1; b2=$2; getline if ($1 >= b1 && $2 <= b2) print prev}' filename

Any insight or suggestions would be appreciated - I may have some syntax errors here to work through.

[dpryan]

The tldr version: use bedtools or R, they have functions for this sort of thing that are already debugged.

The syntax error is due to not having a semicolon after getline:

 awk '{prev=$0; b1=$1; b2=$2; getline if ($1 >= b1 && $2 <= b2) print prev}' filename

Now with that aside, there are a couple implementational issues. Firstly, you're processing lines in batches of two, so if an even numbered line overlaps a subsequent odd numbered line then you'll miss that:

1	10
11	20
12	19
21	30

You'd want that to either print the "11 20" or the "12 19" line, but neither will get printed since the lines won't be compared (you're comparing line 1&2, then 3&4, etc.). Also, if you have an odd number of lines, then the last line will get printed, since the last call to "getline" will return the same line as the one you're on.

Another issue is that you're only looking for regions where one is included completely in the other. If two regions actually just overlap, then they'll be missed. You can get around that with a more complicated comparison operation.

A final issue is that if an entry overlaps more than one other entry you'll miss that. Getting around this with awk would be somewhat complicated.

For this sort of thing you'd be best off not using awk (though if you're trying to learn what's possible then this is probably a good exercise).

[wrox]

Thanks. I was afraid I was comparing only pairs of consecutive lines. Would a loop be best in this case? As part of this task, I'm basically trying to learn to do comparisons across all lines in the file, as opposed to column based comparisons which are more starlight forward in awk it seems. For instance, to assess non coding regions one could subtract the stop position for a given line in column 2, from the start position in column 1 of the next line (assuming the file is sorted based on position).

Anyway, thanks for your reply. It was helpful.

[dpryan]

A loop might be the only way to do this with awk. Keep in mind that a loop is going to be quite slow. If you ever want to do this for more than a learning experience then use bedtools or similar. It implements these sorts of operations in a much better way than is reasonably doable with awk.