Organizing projects, data inputs and output

[ramirob]

Hello!

Do you have any suggestions, pointers, references, models on how to organize projects and their correspondingraw data, rmarkdowns, results, etc?

We do a lot of bioinformatics projects, most of them have as raw data, fastq files, alignments files, differential expression counts matrices, etc; we develop some kind of pipeline and data processing and as output have some kind of report that we generate, as well as output data. We need to organize this information so that we can answer for example:

- What have we done for investigator X?
- When was the last time that we did such and such analysis?
- What was the output for the analysis we did for investigator X or project Y?
- I need the output for project Y again.
- For whom have we done type of analysis Z (e.g. Differential expression, etc.)
- Project X that we did in 2010, what did it consist of? what was the input raw data, and output?

And other things like this. Right now, for answering some of these things I basically do a very crude search on the directory tree, which is getting bigger and bigger, and rely on memory, past emails, etc. Dangerous.

Any suggestions? Including any commercial software for helping with these kinds of things?

Thanks,
Ramiro

[hoytpr]

Ramiro, you're gonna need a query-able database. If MySql is not on your favorites list, Microsoft Access has a business template that can be hacked to run a core facility. It's a pain to transfer old data, but it'll work for most of the things you've asked about. My suggestion is to do as much as possible using dates in the yyyymmdd format for starting projects.