Bioscope 1.3 Error, suggestions anyone?

[rdeborja]

I've installed and am currently running the latest version of Bioscope (v1.3) on some test data set. All the demos that accompanied Bioscope 1.3 worked flawlessly but running one of our standard whole transcriptome runs has proven to be problematic. I'm including a snippet of the log file. Hopefully someone has seen the issue and potentially resolved it.

==== Start of LOG ====
2010 Nov 28 23:04:07, 138 GMT INFO [main] MAFileMerger:202 - Merging mapreads results.
.
at com.apldbio.aga.analysis.exec.JobWatcher.watchJobs(JobWatcher.java:28)
at com.apldbio.aga.analysis.exec.PluginSubJobLauncherImp.waitForJobs(PluginSubJobLauncherImp.java:249)
at com.apldbio.aga.analysis.exec.AbstractDistributionHelper.waitForJobs(AbstractDistributionHelper.java:236)
at com.apldbio.aga.analysis.tertiary.wt.plugin.MergeDistributionHelper2.run(MergeDistributionHelper2.java:58)
at com.apldbio.aga.analysis.exec.PluginRunner.distributePlugin(PluginRunner.java:130)
at com.apldbio.aga.analysis.exec.PluginRunner.doMain(PluginRunner.java:204)
at com.apldbio.aga.analysis.exec.PluginRunner.main(PluginRunner.java:516)
29 Nov 2010 00:39:29,297 INFO [main] PluginJobManager:128 - >>>> END of PluginJobManager >>>> date=2010-11-29 00:39:29.297 GMT
29 Nov 2010 00:39:29,297 INFO [main] PluginJobManager:130 - >>>> END of PluginJobManager >>>> date DURATION=7 hours 40 minutes 58 secs
29 Nov 2010 00:39:29,300 INFO [ShutdownHook] BioscopeShutdownHook:39 - Stopping all submitted jobs.
29 Nov 2010 00:39:29,300 INFO [ShutdownHook] BioscopeShutdownHook:41 - Stopping all running processes.
29 Nov 2010 00:39:29,304 INFO [ShutdownHook] EventTransportFactory:140 - Closing JMS connection and session
on call from:
com.apldbio.aga.analysis.workflow.BioscopeShutdownHook.run(BioscopeShutdownHook.java:55)

==== End of LOG ====

[KevinLam]

I think there is a new version 1.31 not sure if it will help you solve this. are there any errors in other log files?

[azneto]

I stumble into this problem and it seems that there's not enough memory for merging the mapping results.

I solved this by altering the following file:

$BIOSCOPEROOT/etc/plugins/properties/ma.to.bam.properties

You should lower this value:
ma.to.bam.distribute.number.of.workers = 40

Try 8...

[rdeborja]

Thanks for the advice. I've implemented the change as per your recommendation. Unfortunately it didn't resolve my issue. I'm digging into the logs and can hopefully provide an update here to help anyone else going through the same issue.

I stumble into this problem and it seems that there's not enough memory for merging the mapping results.

I solved this by altering the following file:

$BIOSCOPEROOT/etc/plugins/properties/ma.to.bam.properties

You should lower this value:
ma.to.bam.distribute.number.of.workers = 40

Try 8...

[KevinLam]

Spotted these issues in release notes. how many reads are you trying to align? to convert from matoBam?

When a reference has many entries (>40,000 entries), there is a
possibility that MaToBam will run out of memory. The default
setting java.heap.space=2000 in ma.to.bam.properties has been
tested for a reference of 200,000 entries. If a reference has
more entries than 200,000 entries, it is advised to increase the
value of the key java.heap.space 1000 per 100,000 entries
roughly.



There is significant disk space required for converting ma to BAM
when the option output.filter=none is used, which roughly needs
2TB peak disk space for converting a 500 million reads ma file.
Other options do not need such large peak disk space. The disk
space required per node is smaller if more jobs are dispatched to
more nodes.

[rdeborja]

Thanks for the tip Kevin. I ended up making a change to our grid engine (implemented consumables for memory and expanded a queue the number of nodes being used) and all is fine. I'm quite impressed in the output that it provides and the speed in which the analysis was performed. The PE whole transcriptome plugin has definitely peaked my interest. Thanks again everyone!