[jkbonfield]

I would like to search for unpadded positions in gap5 as none of my variant positions match up in gap5 as the search position box within edit contig view includes pads.

How do I search for an unpadded position in gap5?

Unfortunately right now this isn't possible. I know it's a serious omission and one that I did start to resolve, although that particular solution was driving me stircrazy and I backed out. I do have other plans to address it though. (It's a *lot* harder to implement than it first appears - due to gap5 being an editor rather than just a viewer the mapping of padded to unpadded coordinates is fluid.)

How large are your contigs? The Gap4 approach to this problem is just to compute a consensus and count the number of pads in it, however that's slow on large data.

My apologies for that.

[alig]

Searching unpadded position in Gap5

Hi James,

That's a shame, but I imagine it's not an easy task.

Unfortunately my contig is the length of the X chromosome. But I am only interested in a small interval at the beginning so I'll try to break the contig & open in Gap4 to find the unpadded positions.

thanks for your quick reply

alig

[alig]

breaking contig in gap5

Hi,

Okay, I can't seem to successfully break a contig in gap5, & I cannot open my .aux file in gap4 to break a contig and search for unpadded positions.

I tried to break a contig in gap5 by putting the base position & it seemed to just duplicate my original contig so I had 2 contigs of the same length!

Is it possible to open my .aux file in gap4? I converted it from an ace file using tg_index

Thanks

alig

[jkbonfield]

Ok so first of all, I realise this was a mistake and it's been correct already for the next release - gap5 and gap4 both use the same file extensions, but for completely different file formats. (Or more precisely it's the same underlying mini-database code - hence the matching filenames - but the data that is stored is incompatible.) In the future I'll be using .g5d and .g5x for gap5 database names, with backwards compatibility of course.

So in short, gap4 cannot open the gap5 database. However in Gap5 you should be able to export it in CAF format and then use caf2gap to generate a gap4 database from that. See http://www.sanger.ac.uk/resources/software/caf/

For me if I break a contig in two I then get two smaller contigs. For a while I think it would sometimes split the data in two, but not correctly update the contig start/end locations so it appeared you have two large contigs (while scrolling around would show the data has actually been split). That bug has been fixed already, but maybe that was after the last release.

[jkbonfield]

The latest Staden Package (including Gap5) has been released on sourceforge at

https://sourceforge.net/projects/staden/

This version, 2.0.0b8, has prebuilt binaries for 32-bit and 64-bit linux and 32-bit windows as well as the usual source + doc-src gzipped tar balls. The code also builds fine on MacOS X 10.5 and 10.6, but I've been suffering library version hell with incompatible binaries even between 10.5 machines, so I'm unsure of whether I'll release any Mac prebuilt binaries yet.

On MacOSX though it's now far easier to compile from source. I used MacPorts versions of tcl, tk and X11 which seems to ease things. (You'll need to download and compile io_lib-1.12.5 first though too, which is available from the same site.)

James

PS Changes since the last release:

Staden 2.0.0b8, 8th Feb 2011
=============================

Gap5/tg_index 1.2.11

General changes
---------------

* Microsoft Windows version. The code should now compile and run
under MinGW/Msys on windows.

* MacOS X updates: improved compilation and fixed a few Mac
specific display glitches.

* Various code tidyups - less compiler warnings, more portable
for linux variants (eg Centos).

* Removed dependency on samtools. (This also avoids a few
samtools related bugs, but hopefully not creating new ones to
replace them.)

* Removed dependency on IncrTcl, IncrTk, IncrWidgets. These can
still be used for one part of Gap4 (prefinish GUI) but the
code will build and execute without these now.

Gap5 changes
------------

* In-program help should now be working.

* A Disassemble Readings option has been added to the main Edit
menu. (This performs the same task as in Gap4.)

* Tg_index now supports the CAF, fasta and fastq
formats. When importing fasta/fastq we just create single-read
contigs, so it's simply an easy way of importing consensus
sequences or a *few* finishing reads. CAF support will import
entire assemblies though. Also improved the CAF exporting code.

* First draft of a Find Read Pairs function (as in Gap4). At
present read-pair orientation needs some work still, but the
basic plot and searching is working.

* Internal record numbers are now 64-bit instead of 32-bit,
removing the need for reserving sequence record IDs in
tg_index (-r option) and allowing for more than 2 billion
sequences in a gap5 db.

* Contig editor "tag macros" (from gap4) have been added. Use
Shift + function key to define a tag macro and then that
function key to apply it.

* A "readings" list now exists (as in Gap4) which is displayed
and can be populated using the contig editor "names"
panel. The copying (cut/paste) of read names binding has also
changed from left-click+drag to middle-click, now matching
Gap4.

* Quality value 100 now has a special meaning for the consensus
algorithm, as in Gap4. It forces the base to be considered as
true (as it's only possible via a manual edit).

* More contig editor key bindings: Page up/down (already existed
for scrolling in 1Kb) can be modified by Shift, Control or
Shift+Control to scroll in 10, 100 and 1000Kb steps; [,] keys
for setting base quality to 0 or 100; Shift Up/Down arrow and
Control Up/Down arrow for incrementing/decrementing base
qualities by 1 or 10.

* Editor Delete now only works on pad characters unless Control
Delete is used, in order to prevent accidental removal of real
data.

* Added contig editor "Save Settings" command.

* Internal changes to the template display and Tk canvas.

* Sped up contig editor when dealing with excessively deep
alignments.

* Reading names in the contig editor are now be selected for
copy and pasting using the left mouse button.

* Tg_index now has a -q option to limit the number of unpaired
reads held in memory (for the "-p" option). This significantly
reduces the memory usage on large data sets, at the expense of
longer running times.

Bug fixes
---------

* Various fixes to compilation: parallel makes, improved DESTDIR
support.

* Big tidy up on the cache refence counting system, fixing
multiple sporadic crashes and rarely even data corruptions.

* Fixes to consensus caching sometimes storing corrupted data.

* More robust when we run out of disc space.

* Import GFF should be more robust. This now also supports
"colour=<0-17>" tags to match those used in Artemis.

* The insert size in the Libraries panel is now displayed
correctly (when filled out) instead of at 1/100th of the size.

* Reduced excessive memory usage of tg_index -t on large data
sets. Also fixed a few memory leaks and reference count leaks
(appear like memory leaks).

* SAM export now works correctly when auxillary fields or
read-group data is present.

* Removed a small memory corruption when shutting down
plots. This sometimes caused crashes when quitting the contig
editor.

* Fixed occasional crash when joining contigs.

* Editor searching crash fixes.

[Gators]

Okay this is embarrassing, but I am a complete newbie to this. I am having trouble figuring out how to install gap5. My desire is this - I generated a caf file via MIRA3, I'd like to view it in some sort of program, and MIRA recommends gap4 (although through some reading I've decided gap5 is just a new version of gap5). First thing I did was use caf2gap to convert the file, but really I am pretty sure I did this wrong, simply because I didn't know what -version variable to use; does this just correspond to gapX, i.e. i should be using -version=5? Even beyond that I have no idea how to use gap5, and I don't see any obvious installation instructions...help! I am running Ubuntu, download and extract the files, but beyond that, I am confused. There are no make files that I can see, nor ./configure files. Do i just move the bin directory into my path? Help!

[Gators]

Okay this is embarrassing, but I am a complete newbie to this. I am having trouble figuring out how to install gap5. My desire is this - I generated a caf file via MIRA3, I'd like to view it in some sort of program, and MIRA recommends gap4 (although through some reading I've decided gap5 is just a new version of gap5). First thing I did was use caf2gap to convert the file, but really I am pretty sure I did this wrong, simply because I didn't know what -version variable to use; does this just correspond to gapX, i.e. i should be using -version=5? Even beyond that I have no idea how to use gap5, and I don't see any obvious installation instructions...help! I am running Ubuntu, download and extract the files, but beyond that, I am confused. There are no make files that I can see, nor ./configure files. Do i just move the bin directory into my path? Help!

Okay just realized it is a perl script...d'oh

[jkbonfield]

A few comments.

1) Firstly gap5 is the logical successor to gap4, but gap4 is feature rich and robust while gap5 is very much still a work in progress and so lacking some of the functionality from gap4 and is likely to have some annoying bugs still (although obviously I'm fixing them as I find them).

It's really a matter of scale though. Gap4 can cope with 100k readings, maybe a few million if you wish to be patient, so was very widely used in the days of capillary sequencing or more recently with small pathogens sequenced via 454. Gap5 can cope with a billion or more. My advice is simply to use gap4 for now if you know the size of the project will be suitable, otherwise use gap5. Note that the two are completely incompatible in binary formats.

2) Caf2gap is really CAF to GAP4 conversion. It has nothing to do with gap5 and the -version flag isn't for that. Rather Gap4 DBs consist of a name and version; eg FOO.0 with 0 being the version.

3) Gap5 can now cope with caf directly, so you can just use "tg_index bar.caf" followed by "gap5 bar.0 &".

So pick 2) or 3) above depending on data size or maybe just curiosity.

[fkolakowski]

Hi James

I have been trying to get a functional GAP5 on several different Centos machines (5.8).

I have no errors/issues either compiling or installing the complete Staden package.
The issue is that when I run any Staden GUI based programs, they can’t seem to find package Tk.

$ gap5
can’t find package Tk

I know from other work that I have done that the libraries are present and in the LD_LIBRARY_PATH.
I can run tcl and tk scripts that I have written and get the right responses. You seem to call a staden specific Tk package and I suspect that package is not loading the rest of the Tk library correctly.

Are there some fixes/suggestions that you can recommend? I have compiled many programs and written a few and have rarely had this kind of issue. I have seen a few of your other suggestions to folks, but these are things I have already tried (propoer environment variables and sourcing correct shell variables).

The version I have installed is Staden-2.0.0b9.x86_64 from both binary and source.

[archie.chauhan]

can't find package Tk

hi i am experiencing the same problem with gap5. I could successfully create gap5 database using tg_index. But agp5 always comes up with
can't find package Tk. Does anyone has the solution?

[jkbonfield]

Mutter - somehow RedHat based systems always break somewhere with this. I suspect it's something to do with /usr/lib/ containing 32-bit libraries even on 64-bit installs and having to explicitly look in /usr/lib64/ for 64-bit variants. (Debian takes a different policy where /usr/lib/ is always the native architecture.) Are you running a 32-bit or 64-bit OS?

Do you have Tk definitely installed on the system? I think though we even bundled some libraries with the binary release.

You could also try setting the STADEN_DEBUG environment variable and rerunning gap5. This will dump out various messages including library errors. (Some are expected and normal.) It may help to diagnose why it is failing to find the correct system dependencies.

[aforntacc]

error make install staden on ubuntu

dear all please i need some help, kindly assist me please

i am trying to install staden on my ubuntu 14.04 32bits

i downloaded the package staden-2.0.0b10-src.tar.gz with it dependencies io.lib
the configuration went well but when i tried to make install by typing
'make && sudo make install'
it returned the error below and i am new to linux so i didnt know what else to do than to look for help from people that know better.

thanks


/home/charles/tcl8.5.8/unix/tk8.5.8/generic/tkDecls.h:909:20: error: unknown type name ‘CONST84’
int * argcPtr, CONST84 char ** argv,
^
In file included from /home/charles/tcl8.5.8/unix/tk8.5.8/generic/tk.h:1537:0,
from /home/charles/staden/staden-2.0.0b10-src/build.tristy/../tk_utils/cli_arg.c:7:
/home/charles/tcl8.5.8/unix/tk8.5.8/generic/tkDecls.h:1738:103: error: unknown type name ‘CONST84’
int (*tk_ConfigureWidget) (Tcl_Interp * interp, Tk_Window tkwin, Tk_ConfigSpec * specs, int argc, CONST84 char ** argv, char * widgRec, int flags); /* 29 */
^
/home/charles/tcl8.5.8/unix/tk8.5.8/generic/tkDecls.h:1764:5: error: expected specifier-qualifier-list before ‘CONST84_RETURN’
CONST84_RETURN char * (*tk_DisplayName) (Tk_Window tkwin); /* 55 */
^
make[1]: *** [cli_arg.o] Error 1
make[1]: Leaving directory `/home/charles/staden/staden-2.0.0b10-src/build.tristy/tk_utils'
make: *** [tk_utils] Error 2

[Nathanguy14]

Gap5 missing package plotchart

Hi Everyone,
I was trying to use the Template Display feature of gap5 in windows 7, and when I click on "Filter" I get the following error message:

"Error: Can't find package plotchart"
The details of the error are listed below:
can't find package Plotchart
can't find package Plotchart
while executing
"package require Plotchart"
(file "c:/program files (x86)/staden package/share/staden/tcl/gap5/libraries.tcl" line 1)
invoked from within
"source {c:/program files (x86)/staden package/share/staden/tcl/gap5/libraries.tcl}"
(in namespace eval "::" script line 1)
invoked from within
"namespace eval :: $auto_index($name)"
(procedure "auto_load" line 13)
invoked from within
"auto_load $name [uplevel 1 {::namespace current}]"
(autoloading "ListLibrariesPopulate")
(procedure "::unknown" line 22)
invoked from within
"ListLibrariesPopulate [set ${w}(io)] $t.list"
(procedure "seq_seqs_filter" line 107)
invoked from within
"seq_seqs_filter .read_depth1 .read_depth1.track1"
invoked from within
".read_depth1.controls.filter invoke"
("uplevel" body line 1)
invoked from within
"uplevel #0[list $w invoke]"
(procedure "tk::ButtonUp" line 24)
invoked from within
"tk::ButtonUp .read_depth1.controls.filter"
(command bound to event)

I guess I wanted to know where I can find this package and where I should put it so gap5 can find it. Any help you all can give would be greatly appreciated. Thanks!

[jkbonfield]

I hardly ever run Windows locally so I'm afraid I don't tend to spot things like this, although we did test it I'm sure.

plotchart is part of "tklib" http://sourceforge.net/projects/tcllib/files/tklib/0.5/ and should have been installed along with the Staden Package. If not then it can be unpacked and installed too, although you'll need to modify the Tcl auto_path.

IIRC this is normally done at install time. Take a look at $STADENROOT/share/staden/etc/shlib.conf and see which directories it adds to look in for plotchart. Locally mine says:

foreach x[list "/usr/share/tcltk/tklib0.5" ""] {
if {![info exists _p($x)]} {
set _p($x) 1
lappend auto_path $x
}
}

Ie tklib0.5 (which contains a subdirectory named plotchart) is pulled from /usr/share/tcltk/ directory.

[Nathanguy14]

Thank you for the detailed response! I unzipped and installed tklib0.5 to $STADENROOT/lib. I wasn't sure how to update Tcl auto_path after the installation process, but this way clicking on "Filter" brings up a dialog, but immediately erases everything on the Template viewer! I tried many subsets of filter options, but nothing seems to bring any data back (except hitting "cancel" immediately, which brings back the original plot). Is there something obvious that I'm neglecting to provide to gap5? I can definitely try the linux version, but it would be cool to get the windows version to work for the sake of convenience.