[drdna]

We have a MiSeq which gives us scads of high quality data but almost never gives us perfectly paired reads. I'll have to check with our tech to see if she sets any kind of paired data flag. Where would that be?

[westerman]

Maybe I'm stupid but I would have thought that a program that is designed to take raw reads, quality trim them and sort them in to paired and unpaired datasets would realize that raw data coming off a sequencing machine often has large numbers of unpaired reads to start off with. As it stands, it appears I have to run one script to cull unpaired mates and then run Trimmomatic. How inefficient is that?

Inefficient indeed. But I want to know what type of machine you have that has large numbers of unpaired reads? My miSeqs and hiSeqs always pair reads -- assuming that I tell them that the project is paired.

[drdna]

Do your 2 input files have the same number of reads, in the same order?

This was the problem. Interesting, there was, to my knowledge, no stipulation in the original publication, or in the online manual, that the program requires input files with perfectly paired reads. Maybe I'm stupid but I would have thought that a program that is designed to take raw reads, quality trim them and sort them in to paired and unpaired datasets would realize that raw data coming off a sequencing machine often has large numbers of unpaired reads to start off with. As it stands, it appears I have to run one script to cull unpaired mates and then run Trimmomatic. How inefficient is that?

[mastal]

The command seems OK, except that you haven't specified the number of threads.

Does trimmomatic give any error messages?

Is there something wrong with the format of read 27 in one of your files
that causes it to be read incorrectly?

Do your 2 input files have the same number of reads, in the same order?

What does the trimmomatic log file indicate is happening when the output files get out of phase?

[drdna]

Trimmomatic is not working correctly in paired end mode:

Read names in output files are not in the correct order. Correct phase was lost at read #27 and there are additional phase changes thereafter. Command line was as follows:

java -jar /opt/Trimmomatic-0.32/trimmomatic-0.32.jar PE -threads -phred33 -trimlog Trim_Lesion6.txt 6_S1_L001_R1_001_clip2.fastq 6_S1_L001_R2_001_clip2.fastq 6_S1_L001_R1_001_paired.fastq 6_S1_L001_R1_001_unpaired.fastq 6_S1_L001_R2_001_paired.fastq 6_S1_L001_R2_001_unpaired.fastq MINLEN:40

[travelk]

Ok, I re-ran the files using the exact same script on the exact same files as suggested to check if it happened consistently on the same line... but now there is no problem with the new output files and tophat runs them just fine. So, I'm not sure what I did the first time around to have the files corrupt like that.

Thank you nevertheless for taking the time to help me!

[tonybolger]

Obviously trimmomatic didn't put + on the next line and now tophat can't read the line properly. This has happened in multiple files. Does anyone know if a) this is normal for trimmomatic and I need to fix this manually or if b) I did something wrong to cause it?

It is certainly not normal - it's one of the strangest trimmomatic issue i have heard of. None the less, it should be possible to track down. Some questions:

1) Does this happen consistently on the same lines of the same files if they are run more than once?
1.1) If so, can you isolate and send me a short example where it happens?

2) What OS are you using?

Thanks,

Tony.

[westerman]

Obviously trimmomatic didn't put + on the next line and now tophat can't read the line properly. This has happened in multiple files. Does anyone know if a) this is normal for trimmomatic and I need to fix this manually or if b) I did something wrong to cause it?

a) No. Not normal.

b) Probably. But your command line looks ok and nothing obvious is popping up.

[travelk]

Hey everyone,

I've used trimmomatic to clean up my reads but when I use the cleaned files in tophat, I get an error. When I examined a bit closer I discovered that trimmomatic converted this:

@D3VDZHS1:119:H036PADXX:1:1202:12533:34018 2:N:0:GGACTCCTTAGATCGC
ATAGACAAATGCCTGCAACAACGCAGGGATCTCTTTCCCGGTAAACCAACCGTCGTCATTGAAGATATGCATGCTGGCTCGGGTATCCCATTGCTGATAC
+
@@CADDFFBBFHHJIGIJJIIJIBDDGE@ABGEHGGGIJIGF:CFFHGIJG<?8ADBDB@AC;.;>A>>@>:@ACC@@?C<BB(0>(:@(4::@(+4>A>
@D3VDZHS1:119:H036PADXX:1:1202:12611:34155 2:N:0:GGACTCCTTAGATCGC
GGTATCAACGCAGAGTACTTTTTTTTTTTCTTTTTTTTTTTTTTTTTTAAAGGAAAACCAGACAAATCATGAAGCCACATACGCTAGAGAAGCTCAATAC
+
B@@DFFDFHHGHHGIEHHIJIIJJJJJJI)BFGG):BCDDDDDDDDB#####################################################
@D3VDZHS1:119:H036PADXX:1:1202:12731:34205 2:N:0:GGACTCCTTAGATCGC
TAATAAATCCGCTACCGACGCTGACTAACATTTCGCGATCGTTCATCGCATCACCAAAGGCCGTGCAATCGCGCAACGATAAACCTAAATGTTGGGTCAG
+
@@CFFFFFHHHHHIJJIIJIIIJJJGIIJJJJJIIJJBAHG@GHEHFGF=BCEEEC?AC?AAB8888@CC??>BBDD>BBBBCDDAACDCDDC@8<?CBC


TO:

@D3VDZHS1:119:H036PADXX:1:1202:12533:34018 2:N:0:GGACTCCTTAGATCGC
ATAGACAAATGCCTGCAACAACGCAGGGATCTCTTTCCCGGTAAACCAACCGTCGTCATTGAAGATATGCATGCTGGCTCGGGTAT+
@@CADDFFBBFHHJIGIJJIIJIBDDGE@ABGEHGGGIJIGF:CFFHGIJG<?8ADBDB@AC;.;>A>>@>:@ACC@@?C<BB(0>
@D3VDZHS1:119:H036PADXX:1:1202:12731:34205 2:N:0:GGACTCCTTAGATCGC
TAATAAATCCGCTACCGACGCTGACTAACATTTCGCGATCGTTCATCGCATCACCAAAGGCCGTGCAATCGCGCAACGATAAACCTAAATGTTGGGTCAG
+
@@CFFFFFHHHHHIJJIIJIIIJJJGIIJJJJJIIJJBAHG@GHEHFGF=BCEEEC?AC?AAB8888@CC??>BBDD>BBBBCDDAACDCDDC@8<?CBC

Obviously trimmomatic didn't put + on the next line and now tophat can't read the line properly. This has happened in multiple files. Does anyone know if a) this is normal for trimmomatic and I need to fix this manually or if b) I did something wrong to cause it?

My input code:

java -jar /path/to/Trimmomatic-0.32/trimmomatic-0.32.jar PE -threads 8 -phred33 -trimlog Sample1trimlog sample1_R1.fastq sample1_R2.fastq sample1_R1_TP.fastq sample1_R1_TU.fastq sample1_R2_TP.fastq sample1_R2_TU.fastq ILLUMINACLIP:/path/to/Trimmomatic-0.32/adapters/adapters.fa:2:30:10 LEADING:3 TRAILING:3 SLIDINGWINDOW:4:15 MINLEN:36

Thanks for your help!

[GenoMax]

I may be answering in part my question- or at least clarifying my question.
I am no longer certain that the sleeping process list actually really effectively did anything to the cache- its almost always used to capacity by linux so I guess my question is-

what the heck are those 40 + sleeping processes doing- and are they effectively actually consuming any resource whatsoever. Do they really hold ram or is it just cached...

Would be curious.

Most modern linux distros manage memory internally so one can't depend on output of htop/top alone. With a TB of RAM you should have no worries about memory consumption

See: http://www.linuxatemyram.com/

[rufessor]

Glad it helped- and happy to hear your crusin along.

I may be answering in part my question- or at least clarifying my question.
I am no longer certain that the sleeping process list actually really effectively did anything to the cache- its almost always used to capacity by linux so I guess my question is-

what the heck are those 40 + sleeping processes doing- and are they effectively actually consuming any resource whatsoever. Do they really hold ram or is it just cached...

Would be curious.

[atarrant_notspam]

Thank you so much for your help rufessor. You definitely pointed me in the right direction. Using vi I was able to see (and remove) a hidden windows "^M". Rookie mistake, but hopefully someone else can learn from it...

[rufessor]

new to this program as well... but when I see this error- at least the text

Exception in thread "main" java.lang.RuntimeException: Unknown trimmer:

it has been because I misspelled the actual command (e.g. ILUMINACLIP... not ILLUMINACLIP). You did not, but it makes me wonder if you have an extra space or extra file in there prior to the ILLUMINACLIP statement- that might cause trimmomatic to parse a file name as a command, which would yield this error- I surmise.

Just a guess... but may stimulate you to look again and see something your missing.

[atarrant_notspam]

I've used Trimmomatic successfully in the past, but I'm now getting an error I don't understand (and haven't been able to find through searches of this forum and google searches).

My command:
java -jar ~/hostshare/trimmomatic-0.32.jar PE -phred33 -trimlog ~/hostshare/trimtest.log ~/hostshare/C2_l.fastq.gz ~/hostshare/C2_r.fastq.gz ~/hostshare/C2_l_pairtrim.fastq.gz ~/hostshare/C2_l_uptrim.fastq.gz ~/hostshare/C2_r_pairtrim.fastq.gz ~/hostshare/C2_r_uptrim.fastq.gz ILLUMINACLIP:NRC_trim_C2.fa:2:40:15 LEADING:20 TRAILING:20 MINLEN:50

I've also tried the "classpath" route and various other permutations.

The output:

[First it reports the adaptors, e.g., Using Long Clipping Sequence: 'GAT....GCTTG'
There are 7 sequences reported, the ones that are present in my fasta file. The remaining output follows]

ILLUMINACLIP: Using 0 prefix pairs, 6 forward/reverse sequences, 1 forward only sequences, 0 reverse only sequences
Exception in thread "main" java.lang.RuntimeException: Unknown trimmer:
at org.usadellab.trimmomatic.trim.TrimmerFactory.makeTrimmer(TrimmerFactory.java:60)
at org.usadellab.trimmomatic.TrimmomaticPE.run(TrimmomaticPE.java:495)
at org.usadellab.trimmomatic.Trimmomatic.main(Trimmomatic.java:35)

Any help really appreciated. I've reached the end of my abilities here.
Also I really tried to figure out this answer, so if someone could point out a resource I should have checked, I will certainly learn from the mistake.

[rufessor]

I am just starting to use Trimomatic on a SE read set from a HiSeq lane-

Here is the command I ran

nohup nice -n 10 java -jar ~/Trimmomatic-0.32/trimmomatic-0.32.jar SE -threads 3 -trimlog trimmomaticX1TRIMLOG 11145X1_141017_D00294_0137_AC5N9DANXX_3.txt 11145X1AdapterTRIMout.txt ILLUMINACLIP:/home/user/Trimmomatic-0.32/adapters/TruSeq3-SE.fa:2:30:7 LEADING:3 TRAILING:3 > X1Trim_logfile &

My question is related to this. I was watching the system status after I started this program and noted that I had a couple processes running at maybe 90 and 150% cpu... three cores.

Then I looked further down the process log and see about 40 like this... (see below)
And note that the cache was basically down to zero on the system (there was free memory but the cache was filled).... This is a single lane 50 bp data set from Illumina.... so its not something odd. The job finished in about 4 hours, and seems to have worked perfectly. Just curious what the story behind that monster list of sleeping processes is... if you could provide a bit of detail so that I can understand the consequences of this on other running jobs or ability to start other high load jobs I would feel better.
This server is 64 core >TB memory- thus my need to figure out for myself (I am no sysadmin) whats going on here.

I assume its reflecting in some way the way its multithreaded and these are some kinda feeder process train.... but maybe thats wrong too.... just curious if this is supposed to happen and what it is.

51839 user 30 10 35.2G 10208M 10908 S 3.0 2.0 0:42.10 java -jar /home/user/Trimmomatic-0.32/trimmomatic-0.32.jar SE -threads 3 -trimlog trimmomaticX1TRIMLO
51838 user 30 10 35.2G 10208M 10908 S 3.0 2.0 0:42.14 java -jar /home/user/Trimmomatic-0.32/trimmomatic-0.32.jar SE -threads 3 -trimlog trimmomaticX1TRIMLO
51837 user 30 10 35.2G 10208M 10908 S 3.0 2.0 0:44.30 java -jar /home/user/Trimmomatic-0.32/trimmomatic-0.32.jar SE -threads 3 -trimlog trimmomaticX1TRIMLO
51834 user 30 10 35.2G 10208M 10908 S 2.0 2.0 0:33.67 java -jar /home/user/Trimmomatic-0.32/trimmomatic-0.32.jar SE -threads 3 -trimlog trimmomaticX1TRIMLO
51835 user 30 10 35.2G 10208M 10908 S 3.0 2.0 0:46.21 java -jar /home/user/Trimmomatic-0.32/trimmomatic-0.32.jar SE -threads 3 -trimlog trimmomaticX1TRIMLO
51787 user 30 10 35.2G 10208M 10908 S 0.0 2.0 0:03.72 java -jar /home/user/Trimmomatic-0.32/trimmomatic-0.32.jar SE -threads 3 -trimlog trimmomaticX1TRIMLO
51788 user 30 10 35.2G 10208M 10908 S 0.0 2.0 0:04.03 java -jar /home/user/Trimmomatic-0.32/trimmomatic-0.32.jar SE -threads 3 -trimlog trimmomaticX1TRIMLO
51796user 30 10 35.2G 10208M 10908 S 0.0 2.0 0:03.93 java -jar /home/user/Trimmomatic-0.32/trimmomatic-0.32.jar SE -threads 3 -trimlog trimmomaticX1TRIMLO
51807 user 30 10 35.2G 10208M 10908 S 0.0 2.0 0:04.00 java -jar /home/user/Trimmomatic-0.32/trimmomatic-0.32.jar SE -threads 3 -trimlog trimmomaticX1TRIMLO
51822 user 30 10 35.2G 10208M 10908 S 0.0 2.0 0:03.47 java -jar /home/user/Trimmomatic-0.32/trimmomatic-0.32.jar SE -threads 3 -trimlog trimmomaticX1TRIMLO
51799 user 30 10 35.2G 10208M 10908 S 0.0 2.0 0:03.25 java -jar /home/user/Trimmomatic-0.32/trimmomatic-0.32.jar SE -threads 3 -trimlog trimmomaticX1TRIMLO
51782 user 30 10 35.2G 10208M 10908 S 0.0 2.0 0:04.20 java -jar /home/user/Trimmomatic-0.32/trimmomatic-0.32.jar SE -threads 3 -trimlog trimmomaticX1TRIMLO
51810 user 30 10 35.2G 10208M 10908 S 0.0 2.0 0:04.00 java -jar /home/user/Trimmomatic-0.32/trimmomatic-0.32.jar SE -threads 3 -trimlog trimmomaticX1TRIMLO
51820 user 30 10 35.2G 10208M 10908 S 0.0 2.0 0:03.22 java -jar /home/user/Trimmomatic-0.32/trimmomatic-0.32.jar SE -threads 3 -trimlog trimmomaticX1TRIMLO
51812 user 30 10 35.2G 10208M 10908 S 0.0 2.0 0:03.20 java -jar /home/user/Trimmomatic-0.32/trimmomatic-0.32.jar SE -threads 3 -trimlog trimmomaticX1TRIMLO
51808 user 30 10 35.2G 10208M 10908 S 0.0 2.0 0:04.28 java -jar /home/user/Trimmomatic-0.32/trimmomatic-0.32.jar SE -threads 3 -trimlog trimmomaticX1TRIMLO
51790 user 30 10 35.2G 10208M 10908 S 0.0 2.0 0:04.63 java -jar /home/user/Trimmomatic-0.32/trimmomatic-0.32.jar SE -threads 3 -trimlog trimmomaticX1TRIMLO
51817 user 30 10 35.2G 10208M 10908 S 0.0 2.0 0:04.04 java -jar /home/user/Trimmomatic-0.32/trimmomatic-0.32.jar SE -threads 3 -trimlog trimmomaticX1TRIMLO
51818 user 30 10 35.2G 10208M 10908 S 0.0 2.0 0:04.77 java -jar /home/user/Trimmomatic-0.32/trimmomatic-0.32.jar SE -threads 3 -trimlog trimmomaticX1TRIMLO
51793 user 30 10 35.2G 10208M 10908 S 0.0 2.0 0:03.55 java -jar /home/user/Trimmomatic-0.32/trimmomatic-0.32.jar SE -threads 3 -trimlog trimmomaticX1TRIMLO
51791 user 30 10 35.2G 10208M 10908 S 0.0 2.0 0:03.51 java -jar /home/user/Trimmomatic-0.32/trimmomatic-0.32.jar SE -threads 3 -trimlog trimmomaticX1TRIM