GALAXY: Huge amount of data?

[sklages]

Hi all,

I have a (probably) very naive question ..

We are running a few machines here as a core facility (Hiseq2000, GAIIx, 454tit,SOLiD) ... there are a couple of users interested in the capabilities of GALAXY, so I am thinking about a local install of the whole package.

My concern is the data amount. E.g. one Hiseq2000 lane with a PE whole exome lib is roughly 2x20G (unzipped) fastq. I can definitively reduce the amount by zipping the datasets. But this is still a huge amount of data to be uploaded via Browser ...
How is this working in practice if you have more than one lane?
How are the jobs scheduled?

I probably need to read more en detail before starting to set up my own installation ...

thanks for any comment ..

cheers,
Sven

[maubp]

Regarding the upload problem: Galaxy can be setup to let users upload their files by FTP, however, in your situation as a core facility you can import the files directly into Galaxy from disk. People do this as part of an automatic sequencing service pipeline - have a search on the Galaxy mailing list.

Regarding the general data volume problem: Galaxy by default keeps all the files on disk, and you can have cron jobs to clean up "deleted" datasets. Some users on the Galaxy mailing list have reported needing to be more aggressive on their server to avoid running out of space.