Velvet killed

[rndouglas]

I am trying to run velveth with paired-end, fastq files for a de novo genome assembly, but after running ~18 hours (16 CPU, 32 GB RAM) my runs end with just "Killed"

I've tried k-mers from 29 up to 81 (it's compiled for up to 99) and the same thing happens each time. An example of what I enter on the command line:

 velveth /home/data/v81 81 -fastq -shortPaired1 '/home/data_home/L008_R1.fastq' -fastq -shortPaired2 '/home/data_home/L008_R2.fastq'

The program reads both fastq files, then it says it's writing them into roadmap file. After that it starts inputting sequences.

Eventually, it kills. Here are the last few lines from the output:

[64018.420552] Inputting sequence 83000000 / 130691634
[64104.707519] Inputting sequence 84000000 / 130691634
[64191.472864] Inputting sequence 85000000 / 130691634
[64278.668702] Inputting sequence 86000000 / 130691634
[64366.189098] Inputting sequence 87000000 / 130691634
Killed

I'm at a loss for what to do. Does anyone know what might be causing this/what I might do to get around this issue?

[GenoMax]

When something like this happens you should start looking at the possibility that the process may be exhausting either memory or temp space (/tmp).

If you are running this on the command line (outside of a job scheduling system) then see if you can capture the standard error to a file to see if that has any clues. Depending on the shell you are using google for the ways of capturing this info from command line.

[rndouglas]

Thank you very much for the suggestion! I'll start looking into that right away.

[mastal]

apart from the problem that you may be exhausting the compute resources, as GenoMax mentioned already, there is an error in your code.

What version of velvet are you using?

With velveth you used to have to interleave paired reads. With more recent versions of velvet, you have the option of keeping the R1 and R2 reads in separate files, but you need to specify '-separate',

$ velveth  output_dir kmer_len  -fastq  -shortPaired  -separate R1.fastq R2.fastq

[bioBob]

Hi.

based on the number of reads you have, you need to be on a pretty large memory system. As GenoMax mentioned, resources are a likely culprit. I would look at memory first. Also, if it is dying on the velveth part, velvetg is going to be an issue. If you don't have a larger machine to to run on, you might look at the digital normalization methods to reduce your data set size. Ditto to Mastal's comment or run the supplied shuffle sequences perl script.

Good luck.

[rndouglas]

Thanks for all the help! So far, I've had luck by splitting my input files in half, correcting my input code, and finding a machine with much more memory for my runs.