We have 8500USD. Need a Blast-capable system

[Blaze9]

Hey guys,

So we're a startup bioinformatics lab. We have a "workstation" running 2x E5-2650 v2s and 128GB ram. So essentially it's a down-stream analysis machine. We're working with transcriptomes roughly around 30-70k genes in size.

Right now we have been doing as much on the cloud as we can. Locally we have only cleaned our raw reads and assembled on iPlant or Galaxy public servers. We do want to add a capability to do the assembly locally.

Right now our limiting step is the Blastx process. Splitting the assembled fasta file into 10 smaller fragments, we can run all 10 at once and have the completed sequence finished in roughly 15 days.

Now:

We have obtained around 8500 dollars for an increase to our current computational power. What I'm thinking is obtaining some off-lease, refurbished blade systems from eBay or another retailer like Xbyte. If we do go this route, a quick eBay search brought me to setups similar to these few items:

http://www.ebay.com/itm/Dell-PowerEdge-M1000e-16x-M610-Blade-Quad-Core-X5560-2-8GHZ-48GB-2x-300GB-SATA-/281424074825?pt=COMP_EN_Servers&hash=item41862e9849

http://www.ebay.com/itm/Dell-PowerEdge-M1000e-16x-M610-Blade-Quad-Core-L5520-2-26GHz-24GB-2x-300GB-10K-/131174295383?pt=COMP_EN_Servers&hash=item1e8a98e757

In your professional opinion, how much of an upgrade would these systems be if we build a cluster using this setup? Would we be able to significantly cut down on our Blast time, if we can dedicate more cores/break-up the fasta file into smaller segments?

Thank you for your feedback!

[Blaze9]

Bump. Can anyone provide their input on this? If anyone uses a cluster for blast, can you provide us with the physical specs of the cluster? Number of processors, processor model, ram, etc.

Thank you!

[jpummil]

Clusters can quickly get into a non-zero amount of administrative time compared to a large memory workstation (also non-zero, but much more manageable...).

Have you considered applying for free cycles on the XSEDE systems? These are NSF funded resources provided to researchers (non-profit) requiring computational cycles that don't have them available at their institution.

[GenoMax]

Thing to keep in mind is these blade servers share a NIC across the set of blades in a box. So if you are planning to build a cluster then you would need to take network bandwidth into consideration. Unless you have high performance storage doing nfs disk mounts across a cluster will lead to degraded performance (you could read BLAST db into memory if you have enough of it).

10G ethernet is not inexpensive and you would need 10G capable switch on the backend (unless you are only going to get one of these blade enclosures) to tie the blade enclosures together. Heat/noise would all be considerations if you do not have access to a dedicated server room to host these in. Managing off warranty hardware is tricky. Parts would not be cheap and may not be available in some cases. So consider that before jumping in.

It sounds like you are looking to do brute force parallelization (i.e. no real mpiBLAST) so that part is straightforward.

[Blaze9]

Thing to keep in mind is these blade servers share a NIC across the set of blades in a box. So if you are planning to build a cluster then you would need to take network bandwidth into consideration. Unless you have high performance storage doing nfs disk mounts across a cluster will lead to degraded performance (you could read BLAST db into memory if you have enough of it).

10G ethernet is not inexpensive and you would need 10G capable switch on the backend (unless you are only going to get one of these blade enclosures) to tie the blade enclosures together. Heat/noise would all be considerations if you do not have access to a dedicated server room to host these in. Managing off warranty hardware is tricky. Parts would not be cheap and may not be available in some cases. So consider that before jumping in.

It sounds like you are looking to do brute force parallelization (i.e. no real mpiBLAST) so that part is straightforward.

Yeah, we are looking into purchasing 10G cable and a switch. We do have a lab room right now which houses an MRI and confocal equipment. The room is cooled with dedicated 220v power + backup generators. There are a few computers/laptops in that room right now so the MRI shouldn't cause any interference with the equipment.

For the blast DB setup, we are thinking of creating a DB on each of the blade's HDDs, so we don't run into any issues. And yes, we're trying to accomplish brute-force parallelization.

jpummil, Thanks for that. I'm applying right now for some computational time.

[GenoMax]

You would want to check if the 10G is fiber or copper on these enclosures.

[jpummil]

jpummil, Thanks for that. I'm applying right now for some computational time.

Just request a start-up on the initial account...the request period for a substantially larger allocation is coming between Sept 15-Oct15 for a block of time that would begin on Jan 1 2015.

If you need assistance, you might well have an XSEDE Campus Champion on your campus who's task is to assist new XSEDE users thru the process of selecting the right machine(s), writing a proposal for larger block of time, etc...

List of CC's here:

https://www.xsede.org/current-champions