[coldturkey]

Hello

I have not used gap 5 in a while. Has there been any new release or has this project been sidelined? The version I have 1.2.2 does still shutdown out of the blue.

[jkbonfield]

So I managed to get gap5 to build and run on a 64-bit Centos system. It did need a patch though to make itcl work (see attachment) due to a non-standard location for it. Sorry I haven't been able to test Gentoo yet.

Centos (and likely some other RedHat based OSes) seem to scatter bits of Tcl to the four corners of the universe, making --with-blah a necessity. The configure line I used was:

./configure --with-samtools=/nfs/team71/psg/jkb/ftp/samtools-0.1.5c --with-io_lib=/nfs/team71/psg/jkb/work/srf/dist --prefix=/tmp/inst --with-tcl=/usr/lib64 --with-tk=/usr/lib64 --with-tklib=/usr/share/tklib0.4 --with-iwidgets=/usr/share --with-itcl=/usr/lib64 --with-itk=/usr/lib64

(The samtools and io_lib bits are just local test installs rather than official locations.) I'm not sure why RedHat uses a mixture of /usr/lib64 and /usr/share for tcl components. Perhaps it's a distinction between packages solely containing interpreted language vs partly or wholey compiled libaries, but tcl doesn't really distinguish this in packages. It's like splitting perl5lib into lots of separate locations. Ugly.

I hope to release a new source tarball soon though with the more up to date configure system. I've put all of these tweaks into the standard configure script with the intention that the next release should automatically build on most standard systems without forcing the user to jump through too many hoops (except for non-standard components like io_lib and samtools).

James

[jkbonfield]

There is a file /usr/lib64/libtk8.4.so, but using

--with-tklib=/usr/lib64 or --with-tklib=/usr/lib64/libtk8.4.so

also gives the same error. Could this be a 64 bit issue or the file being named "libtk8.4.so" instead of "tk8.4.so"? I'll give your precompiled binaries a shot also.

Quite possibly it's a 64-bit thing as the VM instances I've been testing different operating systems on have been 32-bit. The libtk8.4.so is the correct name - I mistyped it previously.

I've been told that the paths the configure script looks in aren't right for some 64-bit systems though as they sometimes hide the libraries in /usr/lib64/ instead of /usr/lib. I recieved a patch to make this work out of the box without needing --with options (the --with-tklib is confusingly to specify the location of the "tklib" package, not the tk library (libtk8.4.so) - not my choice of words!).

So the next source release should hopefully build better, but I'll try and get a fedora 64-bit install somewhere first to test this. Meanwhile you can either stick with the prebuilt binaries, with their own set of oddities, or if feeling brave try rebuilding the configure script from the m4 files in the subversion repository (http://staden.svn.sourceforge.net/vi.../src/ac_stubs/). I'm happy to send you a locally made configure script too if you don't have autoconf installed.

Sorry about these building problems.

James

[greigite]

There is a file /usr/lib64/libtk8.4.so, but using

--with-tklib=/usr/lib64 or --with-tklib=/usr/lib64/libtk8.4.so

also gives the same error. Could this be a 64 bit issue or the file being named "libtk8.4.so" instead of "tk8.4.so"? I'll give your precompiled binaries a shot also.

Hmm, is there not a /usr/lib/tk8.4.so (or /usr/lib/tk8.5.so)? The version number is important, and appears to be how all systems package up their tcl/tk distributions - well until Gentoo perhaps! I'm just using the supplied tcl.m4 for my autoconf detection too, so I'm guessing other packages relying on tcl/tk will fail to compile cleanly on this system as well.

What Gentoo version is this? I may be able to find a vmplayer image and give it a test myself.

However have you tried the prebuilt binaries too? They're not as new, but mainly the change to 2.0 was a major coding / file layout reshuffle.

James

[maasha]

staden-1-7-b1 fails - libssl.so.0.9.7 not found

I get this error trying to run pregap4 from staden-1-7-1b:

staden-linux-x86_64-1-7-1b/linux-x86_64-bin$ ./pregap4
stash: error while loading shared libraries: libssl.so.0.9.7: cannot open shared object file: No such file or directory

I saw the same error running gap5 1.2.0 - but an upgrade to gap5 1.2.2 fixed that.

M

[jkbonfield]

I am having trouble installing gap5 on a 64 bit Gentoo server... configure is unable to use the installed tklib version, present at /usr/lib/libtk.so .

Hmm, is there not a /usr/lib/tk8.4.so (or /usr/lib/tk8.5.so)? The version number is important, and appears to be how all systems package up their tcl/tk distributions - well until Gentoo perhaps! I'm just using the supplied tcl.m4 for my autoconf detection too, so I'm guessing other packages relying on tcl/tk will fail to compile cleanly on this system as well.

What Gentoo version is this? I may be able to find a vmplayer image and give it a test myself.

However have you tried the prebuilt binaries too? They're not as new, but mainly the change to 2.0 was a major coding / file layout reshuffle.

James

[maasha]

I just installed the gap5-1.2.2-linux-x86_64.tar.gz and it works with no warnings and errors. Nice.

Martin

[greigite]

I am having trouble installing gap5 on a 64 bit Gentoo server... configure is unable to use the installed tklib version, present at /usr/lib/libtk.so . I tried

-with-tklib=/usr/lib
and
-with-tklib=/usr

as well as not using the --with-tklib option at all.

Both give the following error message:

checking for Tcl configuration... found /usr/lib/tclConfig.sh
checking for existence of /usr/lib/tclConfig.sh... loading
checking for Tk configuration... found /usr/lib/tkConfig.sh
checking for existence of /usr/lib/tkConfig.sh... loading
checking for Tcl public headers... /usr/include
checking for Tcl private include files... Using srcdir found in tclConfig.sh: /usr/lib64/tcl8.4/include
checking for Tk public headers... /usr/include
checking for Tk private include files... Using srcdir found in tkConfig.sh: /usr/lib64/tk8.4/include
checking tklib directory... no
configure: error: Abort: no tklib package found, use --with-tklib=DIR

[jkbonfield]

Odd that caftools.pm wasn't placed in the caftools package, but I found a copy in the sanger miniphrap2gap distribution. See:

ftp://ftp.sanger.ac.uk/pub/PRODUCTIO...iphrap2gap.tgz

For what it's worth though, Gap4 is typically better at editing and displaying gap4 databases than Gap5 as it's more complete (albeit slow).

James

PS. In new heights of complexity we recently managed to get a 1.9 billion read Gap5 database! It did require a bit of tinkering with the source to build it though, so as always there's a lot more to do. The main issue is with tg_index, which even with tweaks took about 16 hours to build the DB.

[john_doe]

Hi James,

sorry for replying that late.
I'm trying to convert a gap4 database to the gap5 format as described in the README. However, caf2baf seems to depend on a Caftools.pm module in the pipe:
gap2caf -project Database/Lauf1+2 -version A | caf2baf > test/Lauf1+2.baf

Sadly, I'm not able to find that module. Neither in the gap5 distribution nor in caftools-2.0.2. Any ideas where to look?

Thanks in advance,
Andreas

[john_doe]

Ok, great. Thanks for the fast reply.
I'll get back to you if we find bugs.

Andreas

[jkbonfield]

To be honest editing has taken a backseat recently as most people have been after a decent viewer.

However it's always been my intention to replace Gap4's editing capabilities, and so Gap5 already has basic editing, contig joining and contig breaking (both at O(logN) complexity instead of O(N) used in Gap4), however it appears in the recent updates I managed to totally bust joining/breaking so I'll start work on fixing those next week when I get back to the office.

James

[john_doe]

Hi James,

just stumbled over the thread today. We were eagerly awaiting the Gap5 release and will probably test it next week.
I just have one question. How far are the editing capabilities integrated yet and when will they be fully usable? As this is the most important feature for us.

Cheers,
Andreas

[jkbonfield]

Primarily to address the issues of unwanted library dependencies I've rebuilt Gap5, now version 1.2.2. I also fixed a few other bugs, but I'm unsure if this fixes the odd ACE file crash people have reported.

I also took the opportunity to remove some of the very common bundled libraries, which should remove some of the png warnings. So the third party requirements now should be: zlib, png, X11, libcurl, C/math library.

If you're not having problems with the current code it's not important to download this new version.

As before, binaries are at sourceforge:

https://sourceforge.net/project/showfiles.php?group_id=100316&package_id=317355

If there's a pressing need for other platforms let me know. I can't guarantee anything as it takes time, especially on the machines I don't own myself, but it's good to get feedback on user requirements.

James

Gap5 1.2.2, June 22nd 2009
===========================

Minor bug fix release.

Minor changes
-------------


* Added a Hide Annotation setting in the contig editor. This is
also bound to control-Q (as per Gap4). This allows us a quick
way to see the quality values underneath a tag.

* Annotion contents are now visible in the editor information
line, but the "Tag editor" itself still hasn't been ported
over from Gap4.

* Changed SAM reading of LB tags to use TG tags. In practice
libraries rarely seem to appear directly in the sequence
lines, but instead occur in read-groups. In theory we should
then check the main read-group header for LB links from there,
but for now treating a library as a group (eg a run or a lane)
is perhaps more useful than a genuine library grouping.

* Lots of internal code tweaking to support newer versions of
Tcl/Tk (tested with 8.6b1). For now we still ship with 8.4,
but this is the first stage of making the code more portable
and easier to build from source.

* Extra slider (there's too many I know - it's overdue a
redesign) in the template display. This governs the "stacking"
y-mode, controlling where the data gets binned into groups. Eg
every 1k so that templates 0-1000 are stacked together,
1000-2000bp are below, 2000-3000 below those, etc.

Bug fixes
---------

* Removed a memory corruption when displaying long sequence
tags (> 1Kb).

* Removed issues with diagonal lines appearing at the start
of the library insert-size distribution plot.

* Removed unneeded dependency on C++ (libstdc++.so.5) and
FORTRAN (libg2c0.so) libraries.

* The contig selector window now internally uses more 64-bit
integers. This fixes issues where the total contig length grew
beyond 2Gb.

* Initialised more elements in some of the data structures.
Previously some parts of range_t struct were
uninitialised. Hopefully this resolves some of the random ACE
related errors (unknown).

[jkbonfield]

Would it be possible to also put some test database(s) with sequences from various sequencing platforms? Something small just to play with?

I through together something *very* small - a sanger human BAC clone (capillary sequencing) and a thin slice from mitochondrial genome out of one of the 1000genome submissions.

-rw-r--r--  1 badger hsg 5402975 Jun 19 16:59 DJ105K17.baf
-rw-r--r--  1 badger hsg 1566608 Jun 19 16:59 DJ105K17_g5
-rw-r--r--  1 badger hsg    8512 Jun 19 16:59 DJ105K17_g5.aux
-rw-r--r--  1 badger hsg  348101 Jun 19 16:59 mt.bam
-rw-r--r--  1 badger hsg  227824 Jun 19 16:59 mt_g5
-rw-r--r--  1 badger hsg    1120 Jun 19 16:59 mt_g5.aux

The files are in ftp://ftp.sanger.ac.uk/pub/badger/tmp/

They're produced using:

tg_index -o DJ105K17_g5 -B DJ105K17.baf
tg_index -o mt_g5 -b mt.bam

Obviously I could put bigger data sets there, but it's hard to find something the "right size". (I was looking for S.Suis - a nice 2Mb organism, but failed dismally to find our original test set for this and I don't want the hassle of figuring out what's publically released and what isn't.)

If I get a more appropriate data set I can place that there too. It'd be good to get some standard "moderate sized" test sets for developers to experiment with and compare/contrast tools on. Preferably with snps, structural variations, etc.

James