[sklages]

Problem found and solved (for now). We were using glibc 2.15 on our systems. With this version of glibc memory allocation seems to fail (for unknown reasons). With glibc 2.19 issac-aligner works just fine. Took some time .. :-)

[GenoMax]

In my case it is a cluster running Rocks v.5.3.

[sklages]

Well, I tested it on my ubuntu notebook (freshly compiled isaac-align and ref sequence), and there it now runs just fine (both bcl and fastq input). No idea what went wrong before ..

But on our linux the problem still remains; even the bcl input mapping process gets stuck (very late). So I will investigate why it fails to properly run on our linux.

[GenoMax]

Fastq seems to be working as well. You have to sym link the fastq files as indicated on this help page: https://github.com/sequencing/test/b...down/manual.md. Check the fastq.gz examples.

Here is the command that is working for a single end fastq.

$ isaac-align --base-calls /path_to/Fastq --base-calls-format fastq-gz --default-adapters Standard -m 48 -j 6 --reference-genome /path_to/ISAAC_genomes/hg19/Homo_sapiens/UCSC/hg19/Sequence/IsaacIndex/sorted-reference.xml --verbosity 3 --input-parallel-load 6 --temp-parallel-load 6

[sklages]

This command seems to be working with latest iSAAC (no errors and there is regular progress). Full job completion could take 24 h (or more). Example tried is human data.

BTW: The -m default option now looks set to 0, which means no limit .

$ isaac-align --base-calls /path_to_Flowcell_ID/Data/Intensities/BaseCalls/ --base-calls-format bcl --default-adapters Standard -m 48 -j 6 --reference-genome /path_to/ISAAC_genomes/hg19/Homo_sapiens/UCSC/hg19/Sequence/IsaacIndex/sorted-reference.xml --verbosity 3 --input-parallel-load 6 --temp-parallel-load 6

Interesting :-) I just used a run folder as input, just as you did, and it seems to work ..
I have a PhiX run, shouldn't take too long though. So there is something fishy with the fastq input ...

I'll report back tomorrow, when the alignment has (hopefully) finished.
It is not really a soulution (for me) to use the BCLs but may server as a basis for a bug report ..

[GenoMax]

This command seems to be working with latest iSAAC (no errors and there is regular progress). Full job completion could take 24 h (or more). Example tried is human data.

BTW: The -m default option now looks set to 0, which means no limit .

$ isaac-align --base-calls /path_to_Flowcell_ID/Data/Intensities/BaseCalls/ --base-calls-format bcl --default-adapters Standard -m 48 -j 6 --reference-genome /path_to/ISAAC_genomes/hg19/Homo_sapiens/UCSC/hg19/Sequence/IsaacIndex/sorted-reference.xml --verbosity 3 --input-parallel-load 6 --temp-parallel-load 6

[sklages]

My initial call was:

isaac-align --base-calls $(pwd) --base-calls-format fastq-gz --default-adapters Standard --memory-limit 64 --reference-genome $(pwd)/../iSAACIndex.PhiX.20140901/sorted-reference.xml --verbosity 3

The parameter you mentioned have some reasonable default values .. nevertheless, specifying both on the command line just results in the same effect: stuck at "Determining memory capacity for Fastq data" :-(

Concerning the memory limit I have tried some values between 4 and 128GB ..

[GenoMax]

These options are from the previous version but I assume they are still there in the latest. In case they are not let us know.

Have you tried the -m option and specified a memory limit? This needs to be at least 48G for a human data set.

Also -j should probably be set to match the options below (6 or 8).

Also these options are important:

--input-parallel-load arg (=64) Maximum number of parallel file read operations for --base-calls
--temp-parallel-load arg (=8) Maximum number of parallel file read operations for --temp-directory

--input-parallel-load try setting this to 6 or 8 (n-2 cores based on what you have). With HAS this option turns out to be critical to prevent the app from completely overloading the server (default value of 64 is very high). Adjust the temp-parallel-load to match.

[sklages]

Well, it depends: on ubuntu I get an (boost) error, pthread_mutex_lock(m). strace tells me that this is probably related to memory allocation. On our server systems (home made linux), isaac-align hangs at "Determining memory capacity for Fastq data" and is waiting forever :-) Again, some problem with memory allocation!?

And it is always an issue that you never get "understandable & and error related error messages" with illumina (unix) software :-)

[GenoMax]

@Sven: Are you getting an error or does it seem to just hang?

I have not successfully run isaac-aligner as a standalone app in the past (going to upgrade to the latest version from git now, thanks for the heads-up) since there was no support available.

[sklages]

Is anyone running current git version? It still seems to be a problem running isaac-aligner on its own.
Currently it is always stuck here with memory issues (well, not necessarily the amount, I've tried to run on 512GB machines, nor limits).
isaac part of HAS is quite old.

[oiiio]

We had the command on this GCAT page work:

http://www.bioplanet.com/gcat/reports/530/variant-calls/illumina-100bp-pe-exome-150x/isaac-isaac/compare-156-adsqyilqlu-571/group-read-depth

[GenoMax]

I will have to look through my results from back last year. I have since used HAS several time though only for human data.

Before you start be aware that ISAAC/HAS was designed to take over an entire server (using *all* hardware resources on it). It is not going to run on a cluster.

[LVAndrews]

Has anyone had success with command-line isaac? I have a non-model organism I would like to push through this pipeline (ponderosa pine). If I make success, will try to post some commands here for future reference.

[GenoMax]

Hi GenoMax,

if you managed to run it under HAS, why don't you use the command line generated by HAS? That command line can be found either on the standard output, or at the beginning of the log file "Logging/Isaac.stderr.txt" that you will find in the HAS output directory.

Come

I could do that (I may give it a try) but the point was one should be able to use the aligner without having to worry about all other parts of HAS and options that need to go with it. I understand from our local FAS that isaac was developed with one aim and that was to finish alignments in the shortest time possible. I want to be able to point "isaac-align" to a set of sequence files (should not matter which flowcells they came from), an ISAAC reference genome and a few basic options to get aligned bam files.

If that is not possible (or is not in the scope of isaac package) then it would be best to have that spelled out clearly.

I am not sure if you are @Illumina/UK (part of the isaac development team?). If you are, then we could take this conversation temporarily offline to omit thread-clogging info that may be specific to problem I am noticing.