[dandrews]

Very interesting thread. From my perspective, as a bioinformatician, is that commercial bioinformatics software lags well behind the leading edge. If you are doing research and your questions contain bioinformatics challenges, then you can't use commercial software - you need bioinformaticians to talk with and/or to build you an appropriate solution. However, as certain analysis techniques get reduced to common practice commercial tools do a good job in packaging this in an easy to use manner.

For example, ten years ago even research bioinformaticians were only just feeling their way through the complexities of microarray analysis and commercial analysis tools produced garbage. Now that the complexities of microarray analysis are better understood, the commercial tools do a very good job and it would take quite a skilled bioinformatician to replicate their results.

I think with most applications of NGS we are still in the learning phase. Some of the commercial tools do an okay job for certain analyses. However, for many NGS applications, even the cutting edge command-line tools still produce dubious results and in this case you are probably better served with experienced bioinformatician who knows the pitfalls.

[gringer]

I don't think you are getting it. I am not going to learn command line programming, instead I am going to focus on learning more biology. The question was whether it is more cost effective to hire a bioinformaticist (~$75k/year) or buy commercial software (~5k/year).

I've mentioned this before, but you will have a lot of trouble getting reliable, repeatable results out of any software without a bioinformaticist. It requires a bit of computer / maths / biology knowledge in order to interpret the results and understand what results are real, and what results are due to other factors (e.g. input data garbage, covariates, normalisation methods, etc.).

If you're going to focus completely on biology, your choice is basically between a) hiring a bioinformaticist and giving them a minimal budget for spending on software and computing capability and b) hiring a bioinformaticist and giving them a large budget for spending on software and computing capability -- maybe more like $75k/year + $2k/year vs $65k/year + $10k/year.

[MenzZana]

On the same line of inguiries, there is a commercial product which seems promising
namely DNANexus.

https://platform.dnanexus.com/?utm_source=MailChimp&utm_medium=email&utm_campaign=BetaLaunch_Classic_LeadsEA

That perhaps could answer you questions.
Perhaps someone around here has some prior experience with it

[MenzZana]

Galaxy is a fine piece of software but it is lacking in certain aspects when it comes to analyzing NGS data.

How about an alternative approach?
Nowadays there are a lot of pipeline workflow programs that covers NGS analyzing.
The interesting concept is that...
1 Biologist produce the data
2 Bioinformaticians create a pipeline which they share (or some other knowledgeable scientist)
3 Biologist just utilizes the shared pipeline for their analysis

At the moment there are also people who share their pipelines in a variety of software.
Software is in this area come from both the commercial side and freeware side
Suggestions....

Taverna http://www.taverna.org.uk Freeware
KNIME http://www.knime.org/ mostly freeware (HPC version costs)
pipeline pilot http://accelrys.com/products/pipeline-pilot/ commercial

I would suggest a closer look at KNIME. A software which is free for your desktop to play around with and has some NGS tools
http://tech.knime.org/community/next...tionsequencing

Taverna seems a bit immature as a product compared to KNIME, and Pipeline Pilot costs a lot

Or does anyone have other suggestions?

[Nomijill]

As a taxpayer

I work for a commercial company, and our software is compared to Galaxy all the time, and we are glad that Galaxy and other publicly available tools are creating software. It drives us and other commercial companies to be better. Therefore, even the users of commercial software benefit from the government spending. Commercial products have to be a lot better than the free products in order to compete. Otherwise, why would someone pay for something they can get for free.

[priesgo]

It was not my intention to make of this a discussion about Integromics or Spotfire software, it was just an example to justify the use of commercial software for a biomedical researcher with no bioinformatics expertise. And obviously this is the case that I best know.

I will just say about your critics that we use Spotfire for results visualization, which is OK. I don't know if Spotfire deals or not with NGS data but we do. We are not loading a 20GB BAM file into the Spotfire...

Coming back to the main topic I am still of the opinion that for a researcher with no bioinformatics expertise at hand, it will be cheaper, considering that time is money, to go for the commercial software.

[rfilbert]

It was inevitable that the commercial software vendors would recommend their own tools on this thread. If I want to go commercial, why do I want to pay Integromics and also pay Spotfire for this? Spotfire has never been able to deal with data of NGS size and the fact that Integromics is adding an additional layer of expense is extremely unattractive.

[priesgo]

In my honest opinion if you are doing biomedical research and you don't have bioinformaticians around and you are not planning to create a whole department it would be better to go for commercial software.
I am a bioinformatician and processing data from let's say RNAseq, DNAseq and ChIPseq would need three different analysis pipelines that even taking into account that you will be using open source existing tools it takes a loooooot of work and time to get everything ready and efficiently working.

About learning the command line, everybody can learn the command line, but you will also need after learning the time to do the work. It is not a thing of a couple of minutes even for a command line ninja. I agree anyway that it will be no bad practice to get a bit into the command line.

There it goes an example of commercial software doing the work for RNAseq and ChIPseq: https://www.integromics.com/products/genomics/ngs/.
No command line and publication ready figures. But this suits for a particular case you will have to find the one for your precise case.

(Don't blame for the ad, yes I work for this enterprise, but I think it is really a constructive example for this discussion)

[dpryan]

I'm curious as to what people think about outsourcing their NGS analysis needs. Philosophically, this would be similar to using commercial software, but one step further. The advantage would be that you don't need to invest the time to become proficient in using the software (or the infrastructure for maintaining it), but the disadvantage is that you'd be one more step removed from your own data.

Or is outsourcing essentially just hiring your own bioinformatician (but only on an 'as needed' basis)?

We did that early on, as no one in our group had any experience with bioinformatics. I have to say that outsourcing (at least to a company) has some serious issues. In my experience, most companies provide shoddy statistics using highly questionable procedures. Basically, unless you're using long-established protocols (e.g., differential expression using microarray), I wouldn't trust any company to do the work.

Collaborating with another group is probably the best way to go.

Frankly, familiarising yourself with the tools of bioinformatics (including R and Linux, I'm obviously not talking about any programming here) is not terribly difficult. If someone can't do that, they should probably pick a different field.

[Zigster]

Not quite getting the tupperware analogy, but I take it from your logic that Ford charges 300% of a fair price for a car, and the solution is for the government to create a new tax to make cars and give these cars to US taxpayers and non-US taxpayers alike. To be clear, you are 100% anti-business and believe the government should produce all products?

You are either a troll or unfamiliar with how scientific software is developed. I will assume the latter.

Scientific software is written largely by the underpaid masses in academia. Many of these developers are actually international graduate students at US institutions who pay their own way. So in addition to the $21 billion they bring into our economy, US companies can build directly on their research. This is a good thing.

Most commercial NGS software consists largely of GUI wrappers of existing algorithms. (I will exclude CLC and Novocraft from this, perhaps a few others, as they have put considerable effort into their own algorithms. Perhaps it is not a coincidence these are foreign companies.) Other companies specialize in compiling and cleaning data from public resources.

Commercial software companies are largely ill equipped to do the research necessary to develop programming APIs and platforms such as Bioconductor that are the cornerstone of how bioinformatics is actually conducted.

Developing front ends for biologists is a valuable service at which commercial providers excel, but this should not occur until the kinks and best practices have been worked out in the open source community over several years.

Most NGS analyses are simply not yet commoditized enough to be shoehorned into some point and click desktop app. Those commercial packages that are available often fall short in terms of being extensible, reproducible, or scalable.

[scbaker]

I'm curious as to what people think about outsourcing their NGS analysis needs. Philosophically, this would be similar to using commercial software, but one step further. The advantage would be that you don't need to invest the time to become proficient in using the software (or the infrastructure for maintaining it), but the disadvantage is that you'd be one more step removed from your own data.

Or is outsourcing essentially just hiring your own bioinformatician (but only on an 'as needed' basis)?

[Kennels]

May be a little late, but my two cents.

I come from lab like yours with people with basically no experience with NGS analysis and using linux environment. I was hired specifically for this, but my background is microbiology with very limited unix/NGS analysis experience prior to this position. I got this position with the understanding that I would do most of the 'learning' and educate the lab while performing the analysis, so to speak. In a way I guess I was bridging the biology to the analysis side of things, as I understood the strife that biologists often feel when trying to get into this area (no offense to anyone).

And I can say that after two years on the job, it is a lot of time to invest not only to learn how to use the software (both commercial, and open source, and GUIs and command line based. In fact this is the easy part), but to understand the parameters/algorithm behind them, how to properly interpret the output, and also very importantly, how to format datasets. Quite frankly commercial packages and even galaxy are not flexible enough to handle alot of the text based files and give you what you want, and even a half-decent knowledge of the command-line environment helps tremendously. Sequence data format vary from sequencing platform to platform, and manufacturers in fact change their output format time and time again, so it is still a very fluid space with few standards. Infrastructure to store and access the data is also important, and paid-for support are often limited and slow.

So even if you do decide on some commercial/freeware hybrid approach, there is a significant amount of time you still need to invest to understand what goes on behind the wheel, both of the software, and of the results. I often get asked things like 'why don't i just map this data to that reference, and presto, results! It's easy...', and commercial packages do offer this. But it often is not that simple. For example, if you read this thread: http://seqanswers.com/forums/showthread.php?t=24270
It says bowtie2 (an aligner) may not handle reporting multi-aligning reads well, and expression results or small RNA mapping may be underestimated. However if you are not familiar with the command line parameters, or even know what goes on 'behind the wheel', it can be hard to identify whether your results are optimal.

So a dedicated person (don't have to be a bioinformatician per se), or at least someone who is willing to invest the time to understand the software and data in the linux environment is best in my opinion. At least this way you know what questions to ask.

[BAMseek]

If people are interested, I'll post a review of our comparison of the two solutions.

I'd be interested to hear what you think about those tools. You might also want to check out

BaseSpace (Illumina)

I've never used it and it's fairly new but might also be of interest.

Justin

[dnart]

We've decided to evaluate a commercial product, Partek Flow, and a freeware solution, Galaxy. The reason for narrowing it down to those two was that we have already had a good experience with Partek's desktop solution to microarray, and we don't want to buy and maintain the hardware, Linux OS, softwares, etc. and these two seemed like the most viable options. If people are interested, I'll post a review of our comparison of the two solutions.

[xied75]

Also please note that setting up a local computational solution will either way cost quite a lot of money, as NGS analysis requires quite a lot of computational resources. For instance, aligning a batch of 32 RNAseq samples took me 4 weeks on a decent workstation (8 cores, 24GB RAM), so we upgraded our capacities. However, the next batch will be ~60 samples with more to come in the future, thus we went for grid computing instead to circumvent this bottleneck. This also required help from our local IT support team but it reduced the waiting time to 2 weeks only.

Agree with you to go the cloud, otherwise you'll end up with many islands that need look after by IT.

Just curious, how many nodes you got from the Grid, did you manage to get a linear speedup?

Best,

dong