I want a petabyte. (without getting bitten)

[Richard Finney]

Anybody tried a "roll your own" solution to having multi-terabytes of short reads and bam files, just waiting to be analyzed ?

Specifically, has anybody just jammed 3TB internal hard drives into a bunch of rackmounts or towers ???

Anybody tried a "thumper" (sun x4540 : http://en.wikipedia.org/wiki/Sun_Fire_X4500 ) or backblaze storage pod (http://www.protocase.com/products/index.php?e=Backblaze ).

Any thoughts?


Tangent links:

http://news.ycombinator.com/item?id=797995

http://www.enterprisestorageforum.com/technology/news/article.php/3838521

http://openstoragepod.org/

http://slashdot.org/index2.pl?fhfilter=backblaze

[GenoMax]

If the "without getting bitten" part is important then .....

[pmiguel]

Hi Richard,
Yes, we replaced all the 500 GB drives in our X4500 "Thumper" with Hitatchi Deskstar 3 TB drives. I was sweating the result, but so far, so good. We have had one drive go bad over the several months we had them. After replacing it, the drive tested good in another computer.
Obviously this is not a high performance solution but for a total cost of around $5000 for 120 TB (prior to RAIDing), I don't see a down side. That said, we do the majority of our read mapping from our BlueArc storage, then move the results to the Thumper for "medium term storage".

--
Phillip

[Richard Finney]

Hey pmiguel, thanks for the response. I have some follow-up questions ...

Does the x4500 Thumper have a CPU?
Is the connection to the outside only via NFS and a NIC card? Do you have I/O bottleneck issues? Do you have USB ports? How do you back this up?

[pmiguel]

Hey pmiguel, thanks for the response. I have some follow-up questions ...

Does the x4500 Thumper have a CPU?

Yes, it is a Sun box running solaris. I think it has 2 cores and 16GB of RAM.

Is the connection to the outside only via NFS and a NIC card?

Yes, I think so. There may be a monitor and keyboard that can be hooked up to it in the machine room. Not sure about that.

Do you have I/O bottleneck issues?

Sure. Like I said, we don't use it for mapping much. We use the BlueArc. But it works fine for users pulling their end results off web pages or doing the odd mapping run.

Do you have USB ports?

No idea. The Thumper is off in a machine room. I do not see it often.

How do you back this up?

Over 1 gigabit ethernet to a long term storage unit here at Purdue. For the stuff we back up. We saved the images from our last Illumina run, but we intend to delete them without ever backing them up once we are sure everything is okay.
--
Phillip