Hi, I am interested in how others are accelerating their reassemblies with GEALRD when processing HiSeq 2000 data.
Specifically, parallel make will work perfectly fine on a multicore server. For instance we typically run 8 lanes of PE GAIIx data in 24 hours using a 8 core Xeon server.
The HiSeq data presents new challenges with processing taking nearly a week to process on 8 lanes on an 8 core server.
We are looking at purchasing a 32 or 64 core SMP like server, but are also interested in whether folks are taking advantage of beowulf clusters and dividing the reassembly across nodes. We have considered running one lane/node as an alternative, but this presents some overhead for collating the data once the runs are complete.
Some folks have mentioned using Sun Grid Engine and qmake to divide up the problem. We use PBS Pro so face some issues with porting this.
Can anyone comment on how they accelerated their implementation of the reassembly pipeline?
Thanks
Specifically, parallel make will work perfectly fine on a multicore server. For instance we typically run 8 lanes of PE GAIIx data in 24 hours using a 8 core Xeon server.
The HiSeq data presents new challenges with processing taking nearly a week to process on 8 lanes on an 8 core server.
We are looking at purchasing a 32 or 64 core SMP like server, but are also interested in whether folks are taking advantage of beowulf clusters and dividing the reassembly across nodes. We have considered running one lane/node as an alternative, but this presents some overhead for collating the data once the runs are complete.
Some folks have mentioned using Sun Grid Engine and qmake to divide up the problem. We use PBS Pro so face some issues with porting this.
Can anyone comment on how they accelerated their implementation of the reassembly pipeline?
Thanks
Comment