VM specs for NGS bioinformatics

[cmccabe]

I am in the process of building a VM for NGS data analysis of human DNA-seq data. The specs are below:

Linux RedHat OS
128GB RAM
2 processors with 6 cores each
500GB C drive RAID 10
3 TB D drive

Any suggestions?

I am concerned about the cores and RAM, but its a start. Thank you .

[GenoMax]

Hmm a VM with 128G of RAM?

I assume you are referring to a real server config and a not a VM.

[cmccabe]

It is stated as a stand-alone server or VM but I believe the VM is a better option if it is feasible. What would you recommend? Thank you .

[GenoMax]

Are we referring to the same thing? To me VM = Virtual Machine.

[cmccabe]

Yes you are correct, VM is Virtual Machine, thank you .

[blancha]

It doesn't make much sense to me to run a virtual machine on your own server, among other reasons because of the overhead.

Commercial service providers often offer virtual machines on their own servers at a lower cost than a dedicated server.
I haven't seen a commercial company offering a virtual machine with those specifications, though.
Even a dedicated server from a third party with those specifications might be difficult to find at a reasonable cost.

As far as I understand from your post, you would want to buy your own server, and run programs directly on the host operating system, without using a virtual machine.

[cmccabe]

The main reasons for a Virtual Machine would be more scalable, less upgrade cost, and I think easier for our hospital to implement and maintain. I have read different posts about hardware requirements and just trying to get what is best suited. Currently, I use a Ubuntu 14.04, 64GB, xeon E5-2630 8 core CPU with 1TB HDD and that doesn't seem like it's enough, but maybe I'm wrong. Thank you .

What do you mean by overhead?

[blancha]

Basically, your programs will run slower on the virtual machine than if you run them directly on your host operating system.

As far as the specifications are concerned, my only recommendation would be to go for a rack server instead of a tower server, if at all possible. Of course, that implies you have the room for a rack server, as well as the staff to manage the rack server, which is not possible in many settings. If you don't have the staff or the room for a rack server, another possibility is using a third-party computing cluster. In Canada, we're lucky to have free computing time made available to researchers through Compute Canada. Again, this may not be possible in all countries.

I had a long discussion on this subject with a colleague who also wanted to build his own set-up. Here were my recommendations.

1. Establish your needs first, before trying to determine the appropriate specifications for the server. For example, how many flowcells per month? For how long should the data be stored?

2. Can you host the server with a third party? Eliminates the cost of keeping staff, and reduces the operating complexity.

3. If you will build your own server, go for a rack server, if at all possible. This is the cheapest and most scalable option. However, it requires a room for the server, and staff. The cost of keeping competent staff may exceed the cost of operating the server.

4. If you must go for a tower server, be aware that this option is appropriate mainly for an individual laboratory, and may not suffice to serve the needs of an entire institute.

[blancha]

It's a bit off-subject, but Dell produced this interesting document on building a Linux cluster for next-generation sequencing analysis.

It's very technical, and it's about a cluster, not a tower server, but it's still a good read for anyone wanting to build their own platform.

http://www.dell.com/learn/us/en/05/business~solutions~whitepapers~en/documents~active-infrastructure-for-hpc-life-sciences-wp.pdf

[blancha]

I would add that 3.5 TB doesn't seem like very much.
Again, it depends on your needs.
How many flowcells per month? For how long will the data be stored on the machine?

It also depends on the configuration of your server.
Can storage be added at a later date, if needed?

[GenoMax]

@cmccabe: You can probably run a VM with the specs you originally listed but that would mean you would want server hardware underneath that would be several times more powerful (unless you plan to run only one VM, which would not make sense). Beyond a certain ceiling (in terms of RAM/sockets) the cost of such hardware escalates rapidly.

If your IT is serious about building this right, send them this way

[cmccabe]

We currently run Ion Torrent sequencing on a proton. The estimate of 3.6 TB was based on that data (180 samples per year, each sample is ~20GB). Our IT department, myself included, is new to this type of data... with only a couple of years of experience. There are plans to move to a NextSeq so we have already inquired as too increasing the TB to roughly 9-12. I work in Chicago at a small 300 bed childrens hospital, but NGS is ordered a lot so I am trying to get a better idea. A VM seems like a good option but it seems like its all in the configuration. Thank you .

[colindaven]

I would recomment 10-20TB storage as a minimum, you'll fill it up in no time at all. You'll need to consider backup too (even external hard disks if you're on a tight budget!).

[cmccabe]

I am looking more into linux clusters as they seem to be a better overall fit. If designed correctly they will be a good fit for the lab today and allow for growth. Thank you .

[GenoMax]

Cluster administration is a non-trivial task so make sure you have someone willing to take that on.