best unix environment for Bioinformatics?

[BurlEarl]

Hi all,

I am new to bioinformatics and trying to get some of the packages I need installed. The environment I am using at the moment is cygwin and I am having trouble compiling BWA. The make command always comes up with an error like this:

$ make
gcc -c -g -Wall -O2 -DHAVE_PTHREAD bwa.c -o bwa.o
In file included from bwa.c:8:
bntseq.h:32:18: zlib.h: No such file or directory
In file included from bwa.c:8:
bntseq.h:73: error: parse error before "fp_fa"
Makefile:22: recipe for target `bwa.o' failed
make: *** [bwa.o] Error 1


Also I obtained bowtie2, which is already compiled I think (no makefile), and it fails at line 69 every time regardless of the arguments provided.

$ bowtie2 -x index/lambda_virus -1 reads-1.fq -2 reads-2.fq
/usr/bin/bowtie2-align: /usr/bin/bowtie2-align: cannot execute binary file
Died at /usr/bin/bowtie2 line 69.

$ bowtie2 --help
/usr/bin/bowtie2-align: /usr/bin/bowtie2-align: cannot execute binary file
Died at /usr/bin/bowtie2 line 69.

So the question is: am I dumb or am I missing some package for Cygwin? Also, there is no reason for these to be mutually exclusive. I have heard that Cygwin is a pain in the neck for bioinformatics. Should I bite the bullet and install a full Linux OS?

Any help would be greatly appreciated.

Thanks
Earl

[Richard Finney]

Install zlib using the cygwin setup program for the BWA problem.
You may have to edit the "DFLAGS" line in the makefile to this :
DFLAGS= -DHAVE_PTHREAD -D_NO_SSE2 #-D_FILE_OFFSET_BITS=64

As for bowtie, check permission to execute program at "line 69".
You might try compiling bowtie yourself.
Often you can install Ubuntu as a second operating system and dual boot.
Ubuntu and/or Mint linux are free and will handle most bioinformatics apps.

[dnewkirk]

I think you will be far better off using a Linux environment such as Ubuntu, which has a large, accessible community and level of documentation. They also have BWA, Bowtie, and many other tools already in their repositories for easy installation. It has a bit of a learning curve, but long term it will make your life easier. I've taken the cygwin/Windows route in the past, and found Linux to be a simpler solution. If you have some time to get up to speed, and intend to do this type of analysis a lot in the future, then linux is the way to go. Cygwin will work just fine in the meantime.

[shyam_la]

I think you will be far better off using a Linux environment such as Ubuntu, which has a large, accessible community and level of documentation. They also have BWA, Bowtie, and many other tools already in their repositories for easy installation. It has a bit of a learning curve, but long term it will make your life easier. I've taken the cygwin/Windows route in the past, and found Linux to be a simpler solution. If you have some time to get up to speed, and intend to do this type of analysis a lot in the future, then linux is the way to go. Cygwin will work just fine in the meantime.

I second dnewkirk.. Every word is true!
You can also install Ubuntu in VirtualBox if you are reluctant going full out linux right now..
Meanwhile, http://bow.codeplex.com/ . Check this out. Bioinformatics on Windows. They have BWA for 64 bit windows too..

PS: Disclaimer: I am not a bioinformatician, not affiliated to BOW, have not tried BWA for windows myself and do not know how it really fares compared to the real deal..

[BurlEarl]

Install zlib using the cygwin setup program for the BWA problem.
You may have to edit the "DFLAGS" line in the makefile to this :
DFLAGS= -DHAVE_PTHREAD -D_NO_SSE2 #-D_FILE_OFFSET_BITS=64

As for bowtie, check permission to execute program at "line 69".
You might try compiling bowtie yourself.
Often you can install Ubuntu as a second operating system and dual boot.
Ubuntu and/or Mint linux are free and will handle most bioinformatics apps.

Thanks a heap. I had the zlib, but the "DFLAGS" bit did the trick(no idea what that does) GATK is still giving me an error. It looks like I need to make a java.library.path bwa. I have tried to follow instructions from https://getsatisfaction.com/gsa/topi...work_with_gatk

He says to modify the Makefile.am file...this is not in the untared bwa directory. I made the file myself from https://bio-bwa.svn.sourceforge.net/...bwa/trunk/bwa/ but when I run automake it asks for another file, configure.ac or configure.in, which I do not have either. Man I am confused.

I will work on compiling bowtie2 myself and hopefully it will not be this painful.

On another note. I am going to be doing this sort of computing for a very long time (just starting a thesis project using it), so ubuntu it is. They say that it works great with my thinkpad, so i think that will happen when I get home tonight.

Thanks again gents.

[adaptivegenome]

Ubuntu or MacOS are the best way to go. There are also other flavors of Linux...

[xied75]

Agree to above. I think you should either try to run Linux, or if you like/must stay in Windows world like me, better not use cygwin or MinGw, these two are ok for lightweight work, but for serious work, there are many hidden surprises (I know that because of my recent porting Linux -> Windows work).

You can download BWA and SAMTOOLS from my http://bow.codeplex.com, both run on Windows x64, with identical output and similar performance as Linux. Source code is on GitHub (branched from the real thing) so you can check/compile.

There is also .NET Bio http://bio.codeplex.com a large framework ready built you can call from C#, VB, Python, or PowerShell, many commercial companies using that for their own pipeline etc. (Because the Apache 2.0 License I guess, that means you don't need to contribute back. )

[pag]

I personally use Ubuntu, mostly with bash shell. Some scripts are in tsh (or perl). You do have to do a fair amount of package downloads and installations, but most do not require tinkering with configure and make, but are rather available via apt-get.

I don't use bowtie personally though.

A related question for my fellow cross-platform (windows/linux) folks: is there a way to install something like cygwin and have it able to access my linux (LVM) partition for purposes of getting at my data and/or running quick scripts and directing the file output to either windows or linux? Ideally, I'd like to be able to run consed from within a virtual environment from within windows to look at my data that is saved on the linux partition without having to reboot. Super-ideally this would act like a remote X-session via Xming such that I could have application windows spawn within the Windows 7 environment, rather than a bunch of windows contained within a single application.

[BurlEarl]

Well, set up Ubuntu last night. It is a piece of cake. Luckily I didnt destroy my harddrive in the proccess of repartitioning. I did it through Windows disk manager before I knew the dangers...oops.

Anywho, It is pretty awesome for open souce software....well its pretty sweet period.

The bad news. So I am still having install problems like crazy. I got the BWA from apt-get which loaded without a hitch except that it is not in Java.Library.Path for GATK. So I tried to compile it myself again. Now I am getting the zlib.h error again. installed z.lib with $sudo apt-get install z.lib. Same error. Got rid of [#] from #-D_NO_SSE2. Same error:

earl@Earl-Think:~/Bioinformaticus/bwa-0.6.2$ make
gcc -c -g -Wall -O2 -DHAVE_PTHREAD -D_NO_SSE2 bwa.c -o bwa.o
In file included from bwa.c:8:0:
bntseq.h:32:18: fatal error: zlib.h: No such file or directory
compilation terminated.
make: *** [bwa.o] Error 1


Bowtie2 also has an error:
earl@Earl-Think:~/Bioinformaticus/bowtie2-2.0.0-beta6$ make
/usr/bin/g++ -O3 -m64 -msse2 -funroll-loops -g3 -DCOMPILER_OPTIONS="\"-O3 -m64 -msse2 -funroll-loops -g3 \"" \
-fno-strict-aliasing -DBOWTIE2_VERSION="\"`cat VERSION`\"" -DBUILD_HOST="\"`hostname`\"" -DBUILD_TIME="\"`date`\"" -DCOMPILER_VERSION="\"`/usr/bin/g++ -v 2>&1 | tail -1`\"" -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -DBOWTIE_PTHREADS -DPREFETCH_LOCALITY=2 -DBOWTIE_MM -DBOWTIE2 -DNDEBUG -Wall \
\
-o bowtie2-build bt2_build.cpp \
ccnt_lut.cpp ref_read.cpp alphabet.cpp shmem.cpp edit.cpp bt2_idx.cpp bt2_io.cpp bt2_util.cpp reference.cpp ds.cpp multikey_qsort.cpp limit.cpp diff_sample.cpp bowtie_build_main.cpp \

ref_read.cpp: In function ‘void reverseRefRecords(const EList<RefRecord>&, EList<RefRecord>&, bool, bool)’:
ref_read.cpp:223:8: warning: variable ‘mergedLast’ set but not used [-Wunused-but-set-variable]
ref_read.cpp: In function ‘std:air<long unsigned int, long unsigned int> fastaRefReadSizes(EList<FileBuf*>&, EList<RefRecord>&, const RefReadInParams&, BitpairOutFileBuf*, int&)’:
ref_read.cpp:283:18: warning: variable ‘rpcp’ set but not used [-Wunused-but-set-variable]

Man I feel silly asking all these questions just to get something installed.
Is there a list of 'basic packages", like zlib and gcc, that would be helpful for a first timer?

Earl

[Richard Finney]

Is this a 32 bit machine ?

[BurlEarl]

Ubuntu Desktop 12.04 LTS 64 bit on a lenovo Think pad L512.

I also tried to compile velvet (even thouth I wont need it), and got this error:

earl@Earl-Think:~/bin/Bioinformaticus/velvet_1.2.06$ make
rm obj/*.o obj/dbg/*.o
rm: cannot remove `obj/dbg/*.o': No such file or directory
make: [cleanobj] Error 1 (ignored)
mkdir -p obj
gcc -Wall -O3 -D MAXKMERLENGTH=31 -D CATEGORIES=2 -c src/tightString.c -o obj/tightString.o
gcc -Wall -O3 -D MAXKMERLENGTH=31 -D CATEGORIES=2 -c src/run.c -o obj/run.o
gcc -Wall -O3 -D MAXKMERLENGTH=31 -D CATEGORIES=2 -c src/splay.c -o obj/splay.o
gcc -Wall -O3 -D MAXKMERLENGTH=31 -D CATEGORIES=2 -c src/splayTable.c -o obj/splayTable.o
src/splayTable.c: In function ‘inputSequenceArrayIntoSplayTableAndArchive’:
src/splayTable.c:1060:14: warning: variable ‘refElem’ set but not used [-Wunused-but-set-variable]

Dont know if that will help or if it is another problem entirely.

[BurlEarl]

I installed zlib1g and zlib1g-dev from apt-get. Now I get a message like:

earl@Earl-Think:~/Bioinformaticus/bwa-0.6.2$ make
gcc -c -g -Wall -O2 -DHAVE_PTHREAD -D_NO_SSE2 -D_FILE_OFFSET_BITS=64 bwa.c -o bwa.o
bwa.c: In function ‘bwa_idx_load’:
bwa.c:52:7: warning: ignoring return value of ‘fread’, declared with attribute warn_unused_result [-Wunused-result]
gcc -c -g -Wall -O2 -DHAVE_PTHREAD -D_NO_SSE2 -D_FILE_OFFSET_BITS=64 bamlite.c -o bamlite.o
gcc -c -g -Wall -O2 -DHAVE_PTHREAD -D_NO_SSE2 -D_FILE_OFFSET_BITS=64 utils.c -o utils.o
gcc -c -g -Wall -O2 -DHAVE_PTHREAD -D_NO_SSE2 -D_FILE_OFFSET_BITS=64 bwt.c -o bwt.o
gcc -c -g -Wall -O2 -DHAVE_PTHREAD -D_NO_SSE2 -D_FILE_OFFSET_BITS=64 bwtio.c -o bwtio.o
bwtio.c: In function ‘bwt_restore_bwt’:
bwtio.c:62:7: warning: ignoring return value of ‘fread’, declared with attribute warn_unused_result [-Wunused-result]

...and more of the same

simmilar [-Wunused-result] outputs happen for compiling bwa and bowtie2.



Anyway, thanks for your help so far guys. The precompiled bowtie2 is up and running (it took a long time to figure out the PATH stuff).

I am going to repost this with a more accurate title.

Thanks Again
Earl