So I am currently considering using a cloud computing service as I have 180,000 blast jobs that need doing and to do that on our Uni servers would take a few months. Each job will take ~30 hours and so I was wondering what services others have used and how expensive they are etc. The main one I am looking at is Amazon as I am unaware of any others however their sales team seem to be taking ages to get back to me with pricing. Any advice would be great.
Seqanswers Leaderboard Ad
Collapse
Announcement
Collapse
No announcement yet.
X
-
-
Google: https://cloud.google.com/
Microsoft: http://azure.microsoft.com/en-us/
Amazon's EC2 pricing is on the web unless you were looking for some specific discounts for your institution: http://aws.amazon.com/ec2/pricing/
What DB are you going to blast against? You probably want to use the AMI that NCBI has for Amazon to make things simple: http://blast.ncbi.nlm.nih.gov/Blast....YPE=CloudBlastLast edited by GenoMax; 02-12-2015, 06:11 AM.
-
Given what you say -- 180,000 jobs at 30 hours each -- I suspect that Amazon will give you big thumbs up. Your Amazon instances run about $0.20/hour so a job is $6.00 and 180,000 jobs will be ... well ... more than I'd like to consider. :-)
Comment
-
Originally posted by westerman View PostGiven what you say -- 180,000 jobs at 30 hours each -- I suspect that Amazon will give you big thumbs up. Your Amazon instances run about $0.20/hour so a job is $6.00 and 180,000 jobs will be ... well ... more than I'd like to consider. :-)
Comment
-
Originally posted by GenoMax View PostPerhaps you are not running your blast jobs efficiently? Just a thought.
Swissprot/Trembl restricted to bacteria has got to be smaller than nr. 30 hours seems fairly long unless your input file has hundreds/thousands of sequences.
Comment
-
One problem of running multiple Blast jobs on a cluster is reading in the Blast database into each cluster node. I find that if I run Blast on too many nodes, even with a screaming fast file server, my I/O wait time goes sky high.
Another possible solution is to use the program called 'Diamond' which is a blastx replacement.
Comment
-
Originally posted by GenoMax View PostCan you elaborate what exactly you are trying to do with the blasting? You are using multiple threads for the blast?
Comment
-
Originally posted by thh32 View PostWe are trying to provide functional annotation to an assembly we have recently created. We are using 1 core per blast job as to allow as many as possible to get onto the server at once.
Try using all cores on a physical server for one job with multiple threads (depending on the scheduler you should be able to ask it to run those threads on one physical machine) and see if that speeds things up. Logically it should, though I can't predict the drop in number from 30h per job (since you would still need to chunk through an equivalent number of jobs sequentially).
If you have access to a server with enough RAM you could try making a RAMdisk, cache the database there and do without disk access for index access part. Worth a try.
Comment
-
Depending on what you are looking for and your stringency requirements, could you switch to BLAT instead? Maybe adopt a tiered approach of a first pass with BLAT to reduce the search space, then BLAST or HMMER (in parallel runs) for the higher stringency search on selected hits.Michael Black, Ph.D.
ScitoVation LLC. RTP, N.C.
Comment
Latest Articles
Collapse
-
by seqadmin
The field of epigenetics has traditionally concentrated more on DNA and how changes like methylation and phosphorylation of histones impact gene expression and regulation. However, our increased understanding of RNA modifications and their importance in cellular processes has led to a rise in epitranscriptomics research. “Epitranscriptomics brings together the concepts of epigenetics and gene expression,” explained Adrien Leger, PhD, Principal Research Scientist...-
Channel: Articles
04-22-2024, 07:01 AM -
-
by seqadmin
Proteins are often described as the workhorses of the cell, and identifying their sequences is key to understanding their role in biological processes and disease. Currently, the most common technique used to determine protein sequences is mass spectrometry. While still a valuable tool, mass spectrometry faces several limitations and requires a highly experienced scientist familiar with the equipment to operate it. Additionally, other proteomic methods, like affinity assays, are constrained...-
Channel: Articles
04-04-2024, 04:25 PM -
ad_right_rmr
Collapse
News
Collapse
Topics | Statistics | Last Post | ||
---|---|---|---|---|
Started by seqadmin, 04-25-2024, 11:49 AM
|
0 responses
19 views
0 likes
|
Last Post
by seqadmin
04-25-2024, 11:49 AM
|
||
Started by seqadmin, 04-24-2024, 08:47 AM
|
0 responses
18 views
0 likes
|
Last Post
by seqadmin
04-24-2024, 08:47 AM
|
||
Started by seqadmin, 04-11-2024, 12:08 PM
|
0 responses
62 views
0 likes
|
Last Post
by seqadmin
04-11-2024, 12:08 PM
|
||
Started by seqadmin, 04-10-2024, 10:19 PM
|
0 responses
60 views
0 likes
|
Last Post
by seqadmin
04-10-2024, 10:19 PM
|
Comment