[kevinrue]

Issue with shared memory (solved)

Hi sdriscoll,

Many thanks for the very helpful answer, it seems to work on our system exactly the way you described it.

For those interested, I attached a file containing the memory usage as outputed by top when submitting multiple jobs the way sdriscoll described.

In short:

• Loading the genome was accompanied with 52GB increase of RAM usage (probably some independent process account for part of this, as bovine genome is expected ~27GB)
• Each job was accompanied by a marginal increase of RAM usage (jobs confirmed by a 3% increase in CPU usage)
• Killing the jobs left the RAM usage constant (but reduced the CPU usage)
• Removing the genome reduced the RAM usage by ~27GB (expected size of bovine genome)

Our confusion was that we understood from the STAR manual, that the LoadAndRemove option would share a genome the same way as LoadAndKeep, except for removing the genome from memory after the last job finishes.
Apparently, this is not the case.

If I got it right:

• each job using LoadAndRemove will ignore any pre-loaded instance of the genome, load a new instance and remove it when done
• each job using LoadAndKeep will share the existing instance of the genome, or load one if absent

Please correct me if I am wrong! (peer-review always appreciated )
Kevin

[sdriscoll]

First load the genome into shared memory without aligning anything at all:

STAR --genomeDir /.../STAR2.3.0e_no_annotation/ --genomeLoad LoadAndExit

Then for each instance you need to specify the option '--genomeLoad LoadAndKeep'. This instructs STAR to check for the shared genome in shared memory and load it for the run then leave it loaded in shared memory.

When you're finally done aligning you need to run the following to unload the genome from shared memory:

STAR --genomeDir /.../STAR2.3.0e_no_annotation/ --genomeLoad Remove

They also suggest that if a genome has been loaded into shared memory for some time it may need to be unloaded and reloaded because it may get "paged out" by the system if it wasn't being used. This can have a serious impact to STAR's performance.

[kevinrue]

Issue with shared memory (2nd post)

Hi NitaC,

Thank you for your answer. Submitting one job at a time with more resources is an option, but the sharing of genome in memory is definitely a powerful and attractive feature which I would like to use.

I am not sure what you mean by:

loads it all into memory

From my point of vie, "all" would be a single instance of the bovine genome. I suspect you meant "loads all the genomes for each separate job" ?
From what the manual states, if one instance is already present in memory, any subsequent job should use this instance, instead of loading another instance of the same genome. (LoadAndRemove option)

Meanwhile, what I observed is that for each concurrent job submitted (all pointing at the same genome folder), the amount of cache RAM used increases by approx. 25GB.
In htop, for each job, I read the three columns:
VIRT RES SHR
25.9G 25.7G 25.1G

These numbers suggest that the genome is loaded in the shared memory, although we recently submitted 10 jobs simultaneously, and the machine almost froze (extremely slow, even to log in), and even the SWAP memory was fully used.

I hope this explains more clearly my dilemna, between my understanding of the manual (one instance loaded) and my observations (increasing RAM usage, frozen machine)

Anyone has a further insight into the sharing of genome?

Many thanks
Kevin

[NitaC]

I may not be entirely correct as I suppose I am still a novice, but yes, I do believe Star loads it all into memory. However, after the first run the remaining alignments are faster. (At least this is what I've observed.) I also attempted to submit multiple runs at one time and I ended up just killing the process. If you have the computing power, just run Star with a few more threads one alignment at a time. It really is a fast tool.

[kevinrue]

Issue with shared memory

Dear STAR community and developers,

STAR is great and efficient tool for RNA reads mapping, thank you.

Meanwhile, I think I have an issue being that STAR does not seem to share the genome between parallel runs:
When running htop while submitting parallel STAR jobs, I observe a propotional increase in the amount of RAM used, about 25GB per genome for the bovine genome. (memory bar was essentially empty before submitting the jobs, and cache filled 25*5GB after submitting 5 jobs)

For the record, in htop the increase of RAM memory used affects the cache memory(yellow bars), while the "used" RAM (green) remains very low (about 5GB).

The system is Ubuntu 12.04.2 LTS (GNU/Linux 3.5.0-34-generic x86_64)

Here are two commands among the several I submitted in parallel (replace some paths with ...):

STAR --runMode alignReads --genomeDir /.../STAR2.3.0e_no_annotation/ --genomeLoad LoadAndRemove --readFilesIn /.../N1855_CN_0H_pe1.fastq /.../N1855_CN_0H_pe2.fastq --runThreadN 3 --outFilterMultimapNmax 10 --outSAMmode Full --outSAMattributes Standard --outFileNamePrefix ./N1855_CN_0H_ --outReadsUnmapped Fastx

STAR --runMode alignReads --genomeDir /.../STAR2.3.0e_no_annotation/ --genomeLoad LoadAndRemove --readFilesIn /.../N1178_CN_0H_pe1.fastq /.../N1178_CN_0H_pe2.fastq --runThreadN 3 --outFilterMultimapNmax 10 --outSAMmode Full --outSAMattributes Standard --outFileNamePrefix ./N1178_CN_0H_ --outReadsUnmapped Fastx

Here is the memory status (as output from top) after 5 jobs submitted.
Mem: 264131104k total, 193475272k used, 70655832k free, 89844k buffers


Note:
I obtained the numbers above by submitting the jobs right after restarting the machine. About 15min into the jobs, the 256 GB of RAM were fully in use:
Mem: 264131104k total, 263666924k used, 464180k free, 62412k buffers


Am I right in stating the genome is loaded separately in memory for the different jobs, as they seem to add up in the htop memory bar? If so, is it something in the command lines submitted which is wrong?

Many thanks,
Kevin Rue

[sdriscoll]

I believe it's best to feed Cufflinks only with the highest confidence alignments, and non-canonical junctions in my experience contain more false positives.
Also, many non-canonical splices occur just a few bases away from the highly expressed canonical, which could be caused by sequencing/mapping errors, and possibly by spliceosome errors. These splices will likely throw Cufflinks assembly off.

to add to that recommendation I've also found that cufflinks seems to perform worse if your alignments contain primary and secondary positions for reads. It's best to only have one alignment per read or pair.

[Nicolas Nalpas]

Dear Nino,
That's great, thanks a lot for your answer.
It makes a lot of sense since I set up the alignment to output only uniquely mapping reads.
Thanks for the help.
Regards,
Nicolas

[Nino]

Hey Nicolas,

Star doesnt actually have a mapping quality score for the reads theirs is setup like this:

255 = uniquely mapped reads
3 = read maps to 2 locations
2 = read maps to 3 locations
1 = reads maps to 4-9 locations
0 = reads maps to 10 or more locations

So in essence they only have 5 values for their mapping quality score 255, 3, 2, 1, and 0 with respect to what I just mentioned above earlier. I hope this helps in answering your question

Thanks,
Nino

[Nicolas Nalpas]

Dear Alex,

I am currently using STAR on a new strand-specific paired-end RNA-seq dataset, and I am very happy with the output.
I just have one question though (as a matter of interest really), does STAR calculate the MAPQ value for the reads? I am asking because at the moment with my new dataset (and also with a simulated dataset) all my mapped reads have a MAPQ value of 255, see below:

FCC1GLHACXX:3:1101:16858:25563#GTGAAA   163     5       19275940        255     1S65M24S        =       19275940        65      GCCTTTTGAGACAATACAAATCAAAATATTTACAGAGATAAGGCAGAATCAAACTACATTAAGGAGGGTCTGAGATCGGAAGAGCGTCGT      bbbeeeeegggggiiiiiiiiiiiiiiiiiiiiihiiiiiiiiiiiiihihiiiiiiiiihiiiiiiidggggggeeeeccccccccccc      NH:i:1  HI:i:1	AS:i:124        nM:i:2
FCC1GLHACXX:3:1101:16858:25563#GTGAAA   83      5       19275940        255     19S65M6S        =       19275940        -65     CGTGTGCTCTTCCGATCTGCCTTTTGAGACAATACAAATCAAAATATTTACAGAGATAAGGCAGAATCAAACTACATTAAGGAGGGTCTG      cdddeeeegggihhhiiiiiihhfhiihghiihgiiihghiiiiiiiiiiihiiiiiiiiiiiiiiiiiiihhgiiigggggeeeeebbb      NH:i:1  HI:i:1	AS:i:124        nM:i:2
FCC1GLHACXX:3:1101:16918:25584#GTGAAA   99      14      81019100        255     90M     =       81019191        181     TTGTACCAGTTATCAAACTGTGTTTTGATGGGATAGAGATTGATATTTTGTTTGCAAGATTAGCACTGCAGACTATTCCAGAAGACTTGG      bbbeeeeeggggghiiiiiiehdghiifhhiifghiegfhhhihiiiiiigiihifhihhhiiiiiiiiiiiiiiiihgggfggeee_cd      NH:i:1  HI:i:1  AS:i:176	nM:i:1
FCC1GLHACXX:3:1101:16918:25584#GTGAAA   147     14      81019191        255     90M     =       81019100        -181    CTTAAGAGATGACAGTCTGCTTAAAAATTTAGATATAAGATGTATAAGAAGTCTTAACGGTTGCAGGGTAACCGATGAAATTTTACATCT      ddddddeeedeeeggggihdeiiiiiiiiiiihiihiiihhiihhhhiiihhihiihihhhiiiiiiiiiiiiiiiigggggeeeeeab_      NH:i:1  HI:i:1  AS:i:176	nM:i:1

Thanks a lot for you help, regards,
Nicolas

[alexdobin]

I believe it's best to feed Cufflinks only with the highest confidence alignments, and non-canonical junctions in my experience contain more false positives.
Also, many non-canonical splices occur just a few bases away from the highly expressed canonical, which could be caused by sequencing/mapping errors, and possibly by spliceosome errors. These splices will likely throw Cufflinks assembly off.

[NGSfan]

hi priya, you may want to post this and carry on the conversation at the google groups for rna-star:

https://groups.google.com/forum/#!forum/rna-star

[priya]

If you have stranded RNA-seq data, you do not need to use any specific STAR options. Instead, you need to run Cufflinks with the library option --library-type options. For example,
cufflinks ... ... --library-type fr-firststrand
should be used for the “standard” dUTP protocol. This option has to be used only for Cufflinks runs and not for STAR runs.
It is recommended to remove the non-canonical junctions for Cufflinks runs using STAR's options:
--outFilterIntronMotifs RemoveNoncanonical OR RemoveNoncanonicalUnannotated

Hi Alex,
I am trying STAR to align the reads and then use the Cufflinks to look for expression values.I have stranded RNA-seq data. MAy I know why it is recommended to remove the non-canonical junctions for cufflinks run. How is it gonna affect in Cufflinks if I use the default parameter "no filtering" ??

[Auction]

You can also try following commands, it works for me.
fq1=`ls -m *_R1_*.fastq.gz | tr -d '\n' | tr -d ' '`
fq2=${fq1//"_R1_"/"_R2_"}
STAR --readFilesIn $fq1 $fq2

[bruce01]

Ok, asked over on Stackoverflow, this works:

group1=( $files/Sample1*r1* );
group2=( $files/Sample1*r2* );
( IFS=,; STAR --readFilesIn "${group1[*]}" "${group2[*]}" [OPTIONS]);

Thanks for the help and ideas Dpryan.

##Edit: DPryan, sorry, getting wires crossed between here and Stackoverflow. I was asking how to give STAR the input that I had created, above works. I am reticent to concatenate gzip files, I dont want to create doubles and don't want to change the gzips in any way before aligning: paranoia!

[dpryan]

My example didn't use wildcards, so I'm not sure where that idea came from.

You can just concatenate the gzipped files together without uncompressing them first.

The other normal process would be to simply write your script to generate the comma separated list that's then fed to STAR. You should be able to do that easily enough in bash, which whatever you're using for job scheduling probably already can handle.