Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Erwan
    replied
    Originally posted by lindenb View Post
    **may be** the biomart server doesn't want to handle too many requests coming from the same IP at the same time ?
    Yep that's what I think too, but I didn't find any informations about that in the documentation or on the web.

    As a test I would flag those specific targets with NOTPARALLEL ( http://www.gnu.org/software/make/man...pecial-Targets ) and add a 'sleep' to wait a few seconds before running another target.
    I will try that.

    Merci Pierre

    Leave a comment:


  • lindenb
    replied
    OHH! BTW, "BONJOUR Erwan" :-)

    Leave a comment:


  • lindenb
    replied
    **may be** the biomart server doesn't want to handle too many requests coming from the same IP at the same time ? As a test I would flag those specific targets with NOTPARALLEL ( http://www.gnu.org/software/make/man...pecial-Targets ) and add a 'sleep' to wait a few seconds before running another target.

    Leave a comment:


  • Erwan
    started a topic Biomart getBM and qmake parallelization

    Biomart getBM and qmake parallelization

    Hi there,

    I'm using Biomart in my R script to retrieve gene symbols from ensembl IDs.
    Here is the part of my script making a call to Biomart.

    Code:
    resultTable <- DEXSeqResults(dxd)
    
    ensembl67=useMart(host="may2012.archive.ensembl.org",biomart="ENSEMBL_MART_ENSEMBL")
    mart <- useDataset("mmusculus_gene_ensembl",ensembl67)
    
    ensemblGeneId <- resultTable$groupID
    
    geneNames <-getBM(filters="ensembl_gene_id",
    attributes=c("ensembl_gene_id","external_gene_id"),
    values=ensemblGeneId,
    mart=mart)
    When I execute my script on a single computer, everything goes fine. But when I try to execute it in parallel on a cluster (running on SGE) with a qmake, some jobs work fine and some others produce the following error :

    Code:
    Error in scan(file, what, nmax, sep, dec, quote, skip, nlines, na.strings,  :
      line 1 did not have 3 elements
    Calls: getBM -> read.table -> scan
    I then set the getBM verbose attribute to TRUE and I got the resulting response from the server for failed jobs :

    Code:
    Results from server:
    [1] "\n<html><!-- InstanceBegin template=\"/Templates/biomart_standalone.dwt\" codeOutsideHTMLIsLocked=\"false\" -->\t\n\t<head> \n\t\t<!-- InstanceBeginEditable name=\"doctitle\" -->\n<meta name=\"Description\" content=\"BioMart is a simple and robust data integration system for large scale data querying. It has been designed to provide researchers with an easy and interactive access to both the wealth of data available on the Internet and for in house data integration.\">\n<meta name=\"keywords\" content=\"BioMart, DATABASE, SEARCHES, protein\">\n\n<title>BioMart Webservice</title>\n<script language = \"javascript\">\nvar currentTab=\"Martservice\";\n</script>\n\n<!-- InstanceEndEditable -->\n<link rel=\"stylesheet\" href=\"/biomart/mview/martview.css\" type=\"text/css\" />\n <script language=\"javascript\" type=\"text/javascript\">\n\n\tfunction colourTab(){\n\t\tif((document.getElementById(currentTab))){\n\t\t\t(document.getElementById(currentTab)).className='menuh';\n\t\t}\n\t}\n\n\tfunction rollOver(overTab){\n\t\tif(currentTab!=overTab){\n\t\t\t(document.getElementById(overTab)).className='menuh';\n\t\t}\n\t}\n\tfunction rollOut(overTab){\n\t\tif(currentTab!=overTab){\n\t\t\t(document.getElementById(overTab)).className='menu';\n\t\t}\n\t}\n\n</script>\n\n</head>\n<body style=\"margin: 0px;\" onLoad=\"colourTab()\" >\n\n<div style=\" width:100%; height:15%;\">\n<table height=\"65%\" width=\"100%\" border=\"0\" cellspacing=\"0\" cellpadding=\"0\">\n\t<tr>\n\t\t\n    <td width=\"270\" align=\"right\">\n    <a href=\"index.html\">\n    \t\t<img style = \"margin: 15px 0px 5px 8px;\" height=\"33\" width=\"105\" src=\"/martview/images/biomart-logo.gif\" alt=\"EBI Home Page\" border=\"0\" />\n    </a>\n    </td>\n\t\t <td valign=\"top\" align=\"right\" width=\"100%\">\n\n  \t</td>\n\t</tr>\n</table>\n\n\n<table border=\"0\"  valign=\"bottom\" cellpadding=\"\" cellspacing=\"2\" height=\"35%\" width=\"1000\">\n        <tr>\n        \n\t\t\t<td align=\"center\" valign=\"bottom\" width=\"14%\">\n\t          <div class=\"menu\" id=\"BioMart Home\" onClick=\"location='/martview/../index.html'\" onmouseover=\"rollOver(this.id)\" onMouseout=\"rollOut(this.id)\">HOME</div> \n\t\t\t</td>\n\t\t\t\n\t\t\t<td align=\"center\" valign=\"bottom\" width=\"14%\">\n          <div   class=\"menu\"  id=\"Documentation\" onClick=\"location='/martview/../install.html'\" onmouseover=\"rollOver(this.id)\" onMouseout=\"rollOut(this.id)\">DOCS</div> \n\t\t\t</td>\n\t\t\t\n\t\t\t<td align=\"center\" valign=\"bottom\" width=\"14%\">\n\t\t  <div   class=\"menu\"  id=\"News\" onClick=\"location='/martview/../news.html'\" onmouseover=\"rollOver(this.id)\" onMouseout=\"rollOut(this.id)\">NEWS</div> \n\t\t\t</td>\n\t\t\t\n\t\t\t<td align=\"center\" valign=\"bottom\" width=\"14%\">\n          <div   class=\"menu\"  id=\"Contact\" onClick=\"location='/martview/../contact.html'\" onmouseover=\"rollOver(this.id)\" onMouseout=\"rollOut(this.id)\">CONTACT</div> \n\t\t\t</td>\n\t\t\t\n\t\t\t\n\t\t\t<td align=\"center\" valign=\"bottom\" width=\"14%\">\n\t\t\t<div    class=\"menu\"  id=\"Credits\" onClick=\"location='/martview/../credits.html'\" onmouseover=\"rollOver(this.id)\" onMouseout=\"rollOut(this.id)\">CREDITS</div> \n\t\t\t</td>\n\n\t\t\t<td align=\"center\" valign=\"bottom\" width=\"14%\">\n\t\t\t<div     class=\"menuh\" id=\"Service\" onClick=\"location='/biomart/martservice'\" onmouseover=\"rollOver(this.id)\" onMouseout=\"rollOut(this.id)\">MARTSERVICE</div>\n\t\t\t</td>\n\t\t\t\n\t\t\t<td align=\"center\" valign=\"bottom\" width=\"14%\">\n\t\t\t<div    class=\"menu\"  id=\"Query\" onClick=\"location='/biomart/martview'\" onmouseover=\"rollOver(this.id)\" onMouseout=\"rollOut(this.id)\">MARTVIEW</div>    \n\t\t\t</td>\n\t\t\t\n\t\t</tr>\n</table>\n\n</div>\n\n<div class=\"mart_header\" style=\" width:100%; height:85%;\">\n<table bgcolor=\"black\" width=\"100%\" height=\"5\" border=\"0\" cellspacing=\"0\" cellpadding=\"0\">\n             <tr> \n                <td bgcolor=\"black\" width=\"100%\" height=\"20\" align=\"center\"><!-- InstanceBeginEditable name=\"topnav\" --><nobr></nobr><!-- InstanceEndEditable --></td>\n              </tr> \n</table>\n\n\n<h3>Martservices usage:</h3><br>\n(a) Querying BioMart<br><br>\nTo submit a query using our webservices generate an XML document conforming to our Query XML syntax. This can be achieved simply by building up your query using MartView and hitting the XML button. This XML should be posted to http://may2012.archive.ensembl.org/martservice attached to a single parameter of <b>query</b>. For example you could either:<br><br> \n- save your query as Query.xml and then POST this using the <a href=\"http://cvs.sanger.ac.uk/cgi-bin/viewcvs.cgi/*checkout*/biomart-perl/scripts/webExample.pl?root=BioMart&content-type=text%2Fplain\">webExample.pl</a> script in our biomart-perl/scripts installation.<br>\n- submit using wget: wget -O results.txt 'http://may2012.archive.ensembl.org/biomart/martservice?query=MY_XML' replacing MY_XML with the XML obtained above, first removing any new lines.<br>\n<br>\n(b) Retrieving Meta Data<br>\n<br>\n- to retrieve registry information: <a href=\"/biomart/martservice?type=registry\">http://may2012.archive.ensembl.org/biomart/martservice?type=registry</a><br>\n- to retrieve datasets available for a mart: <a href=\"/biomart/martservice?type=datasets&mart=ENSEMBL_MART_ENSEMBL\">http://may2012.archive.ensembl.org/biomart/martservice?type=datasets&mart=ENSEMBL_MART_ENSEMBL</a><br>\n- to retrieve attributes available for a dataset: <a href=\"/biomart/martservice?type=attributes&dataset=oanatinus_gene_ensembl\">http://may2012.archive.ensembl.org/biomart/martservice?type=attributes&datatset=oanatinus_gene_ensembl</a><br>\n- to retrieve filters available for a dataset: <a href=\"/biomart/martservice?type=filters&dataset=oanatinus_gene_ensembl\">http://may2012.archive.ensembl.org/biomart/martservice?type=filters&datatset=oanatinus_gene_ensembl</a><br>\n- to retrieve configuration for a dataset: <a href=\"/biomart/martservice?type=configuration&dataset=oanatinus_gene_ensembl\">http://may2012.archive.ensembl.org/biomart/martservice?type=configuration&datatset=oanatinus_gene_ensembl</a><br>\n<br>\nFor more information see the webservices section of our <a href=\"http://www.biomart.org/user-docs.pdf\">documentation</a>.\n</div>\n</body>\n</html>\n"
    attr(,"Content-Type")
                      charset 
    "text/plain"      "utf-8"
    whereas in successfull jobs the response looks like that :

    Code:
    Results from server:
    [1] "ENSMUSG00000085299\tGm16627\nENSMUSG00000067978\tVmn2r-ps113\nENSMUSG00000020530\tGgnbp2\nENSMUSG00000074155\tKlk5\nENSMUSG00000087276\tGm11378\nENSMUSG00000048534\tAmica1 ......
    I can also mention that queries in both successfull and failed jobs look the same, and if I re-run a qmake the errors are not necessarly on the same data.

    I guess it's a connection problem but I would like to know if somebody already experienced it and solved it.

    Thanks

    PS : I'm using R-3.1.0 and biomaRt_2.20.0

Latest Articles

Collapse

  • seqadmin
    Exploring the Dynamics of the Tumor Microenvironment
    by seqadmin




    The complexity of cancer is clearly demonstrated in the diverse ecosystem of the tumor microenvironment (TME). The TME is made up of numerous cell types and its development begins with the changes that happen during oncogenesis. “Genomic mutations, copy number changes, epigenetic alterations, and alternative gene expression occur to varying degrees within the affected tumor cells,” explained Andrea O’Hara, Ph.D., Strategic Technical Specialist at Azenta. “As...
    07-08-2024, 03:19 PM
  • seqadmin
    Exploring Human Diversity Through Large-Scale Omics
    by seqadmin


    In 2003, researchers from the Human Genome Project (HGP) announced the most comprehensive genome to date1. Although the genome wasn’t fully completed until nearly 20 years later2, numerous large-scale projects, such as the International HapMap Project and 1000 Genomes Project, continued the HGP's work, capturing extensive variation and genomic diversity within humans. Recently, newer initiatives have significantly increased in scale and expanded beyond genomics, offering a more detailed...
    06-25-2024, 06:43 AM

ad_right_rmr

Collapse

News

Collapse

Topics Statistics Last Post
Started by seqadmin, 07-16-2024, 05:49 AM
0 responses
24 views
0 likes
Last Post seqadmin  
Started by seqadmin, 07-15-2024, 06:53 AM
0 responses
31 views
0 likes
Last Post seqadmin  
Started by seqadmin, 07-10-2024, 07:30 AM
0 responses
40 views
0 likes
Last Post seqadmin  
Started by seqadmin, 07-03-2024, 09:45 AM
0 responses
205 views
0 likes
Last Post seqadmin  
Working...
X