Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • guo
    replied
    Originally posted by Milan Radovich View Post
    You may have already done this, but is your JDK version at least JDK 1.6.0_12?
    Thanks, Milan. The problem now is fixed. It happened at the installation, we did not use the correct user name, like 'guo', in my case... seems funny, but suddenly things turned to be fine, when we changed the name.

    Leave a comment:


  • Milan Radovich
    replied
    You may have already done this, but is your JDK version at least JDK 1.6.0_12?

    Leave a comment:


  • guo
    replied
    Dear all,
    Thank you for your comments ---it can be a learning process for me! But my results are still the same, which stay like this:
    $ /home/guo/bioscope_root/bioscope/bin/bioscope.sh –l /file13/guo/bioscope/log/mapping/workflow_mapping.log /file13/guo/examples/demos/mapping/mapping.ini
    +/home/guo/bioscope_root/bioscope/bin+
    +/home/guo/bioscope_root/bioscope+
    2011 Sep 22 20:51:21, 894 HKT INFO [main] PluginJobManager:107 - >>>> START of PluginJobManager >>>> date=2011-09-22 20:51:21.879 HKT
    2011 Sep 22 20:51:21, 905 HKT INFO [main] PluginJobManager:110 - Bioscope version: bioscope-v1.3-rBS131_55029_20101119113500…
    I checked and double checked the .ini/.plan files (global.ini+mapping.ini for the above run),everything is in the full path of directory…… bioscope is re-installed by the administer…..the configuration is with 10 nodes and 8 cores……JMS is running….. the bioscope queue should be OK (assured by my system administer)…
    One thing is the directory structure, I have to install the example folder outside of my HOME because of space limit, does this matter? Which folder should I submit the job? Again does it matter? ….
    According to your comments, things I am not sure (or do not understand) are ------- If the “bioscope.conf” in /bioscope/etc/conf presents the default settings, should I replace it with the “bioscope.conf” generated by installation, and run update_bioscope.sh? how about the configure_bioscope.sh?
    Many thanks for your time and patience!
    Guo

    Leave a comment:


  • westerman
    replied
    I should have clarified that I use the "command line only" which can be installed by a non-root user. Ditto with command line + GUI. A full install with Command Line + GUI + AutoExport does require root access.

    A good sysadmin will carefully control access to the system and will not allow an install program to be run as root without assurances that the program will not stomp all over the file system. My sysadmin is very careful and questioned why I needed the "full install" at which point I realized that a lower level install would be good enough. YMMV.

    My problem with the ini files -- aside from them changing between bioscope versions -- was how any given run had to have parameters changed in multiple ini files. I finally got the ini files arranged so that I could change one file and the setup would ripple down through the other files. With the GUI the ini files are more defined.

    Leave a comment:


  • mbblack
    replied
    Originally posted by westerman View Post
    I've run as non-root forever. I do not recall any special problems in not being root. That being said, bioscope is hard to get set up properly. Especially with the .ini files. But none of it needs to be installed in system directories.
    I never said it needs to be installed as root or in any particular place. I just think that for anyone completely new to the software, having it installed as root and following ABI's suggested installation guide will make things simpler and more straightforward to get up and running with BioScope.

    I have BioScope and LifeScope on our cluster installed as mentioned above, and it was relatively simple and quick to actually get working with them that way, rather than spend a lot of time editing config files and *.ini files and trouble shooting failed runs.

    Just my $0.02 worth for someone wholly unfamiliar with the software to get going with as few headaches as possible.

    Leave a comment:


  • westerman
    replied
    I've run as non-root forever. I do not recall any special problems in not being root. That being said, bioscope is hard to get set up properly. Especially with the .ini files. But none of it needs to be installed in system directories.

    Leave a comment:


  • guo
    replied
    Thank you, ALL. I will start again from the re-installation. Thanks!

    Leave a comment:


  • mbblack
    replied
    Originally posted by guo View Post
    Hi, ALL,

    I’m a green hand in this kind of data handling…but now need to try on BioScope.

    Our system consists of 128 nodes. Each node contains two 64-bit Intel quad-core Nehalem processors of 2.53GHz and 32GB of RAM.

    I installed bioscope_1.3 at /home/guo/bioscope_cm1, and the example folder at /file2/guo/examples, while output folder at /file2/guo/bioscope… is this where the problem happened?

    As I’m testing the ReseqFrag workflow, and I entered the example folder, simply run something like this
    nohup /home/guo/bioscope_cml/bioscope/bin/bioscope.sh -l workflow1.log analysis.plan &

    -------it turns out that nothing moves at all.

    Then I submitted via qsub script like this (I use PBS scheduler):

    #!/bin/sh
    #PBS -N workflow_ReseqFreg
    # request the queue (enter the possible names, if omitted, serial is the default)
    #PBS -q parallel
    #PBS -l nodes=3pn=8
    #PBS -l walltime=10:00:00
    # By default, PBS scripts execute in your home directory, not the
    # directory from which they were submitted. The following line
    # places you in the directory from which the job was submitted.
    cd /file13/chengguo/examples/workflows/ReseqFrag
    # run the program
    /home/guo/bioscope_cml/bioscope/bin/bioscope.sh -l workflow1.log analysis.plan
    exit 0

    This job is terminated after 10 hours with same result like that of the command line above!
    I know it might be really too long, just sincerely want to make it clear and, anyone, please help!!
    Thanks!!

    Did you install this as a user, or as root? It would make your life a lot easier if you install as root, following the install docs guidelines (you will run into all sorts of path and permission issues otherwise). Check in /bioscope/etc/conf and look in the files there to be sure your configuration is correct (I think the default is to limit the useable cluster to 10 nodes and 8 cores per node - as mentioned, there is rapidly diminishing returns from using more nodes than that). Is the Bioscope queue set up correctly for the queue you are submitting to? Have the paths in the *.ini files for the demo been edited to reflect your current environment settings? Is JMS running on the cluster? Did the examples install hg18 in /bioscope/etc/files ?

    Did you try running any of the verification scripts or stress test scripts before running an example analysis? Those scripts are included when you buy a cluster with BioScope pre-installed, but they may be a separate download from somewhere on the ABI web site.

    My initial suggestion would be first, to reinstall as root, or have your sys. admin. install it as root, it makes life simpler as there is far less fussing needed with the configuration files, example and shared files.

    Leave a comment:


  • epigen
    replied
    Hi guo,

    making Bioscope run gives even system administrators a hard time. (Ours complained a lot ...) When you installed it, did you tell it you have 128 nodes? It's better to reduce the number drastically, otherwise Bioscope will think it's allowed to use the whole cluster, split up your data into too many jobs trying to use all nodes and will most probably fail.

    There's also quite some hardcoding in the scripts the Bioscope wrapper runs. First check if the .ini files that your analysis.plan calls contain all paths. Normally Bioscope complains if something it needs does not exist, but you write that nothing happened at all. Did you use the example analysis.plan and .ini files? And is analysis.plan in the folder you call bioscope.sh from? You may want to try again specifying the full path.

    Leave a comment:


  • guo
    replied
    Hi, ALL,

    I’m a green hand in this kind of data handling…but now need to try on BioScope.

    Our system consists of 128 nodes. Each node contains two 64-bit Intel quad-core Nehalem processors of 2.53GHz and 32GB of RAM.

    I installed bioscope_1.3 at /home/guo/bioscope_cm1, and the example folder at /file2/guo/examples, while output folder at /file2/guo/bioscope… is this where the problem happened?

    As I’m testing the ReseqFrag workflow, and I entered the example folder, simply run something like this
    nohup /home/guo/bioscope_cml/bioscope/bin/bioscope.sh -l workflow1.log analysis.plan &

    -------it turns out that nothing moves at all.

    Then I submitted via qsub script like this (I use PBS scheduler):

    #!/bin/sh
    #PBS -N workflow_ReseqFreg
    # request the queue (enter the possible names, if omitted, serial is the default)
    #PBS -q parallel
    #PBS -l nodes=3pn=8
    #PBS -l walltime=10:00:00
    # By default, PBS scripts execute in your home directory, not the
    # directory from which they were submitted. The following line
    # places you in the directory from which the job was submitted.
    cd /file13/chengguo/examples/workflows/ReseqFrag
    # run the program
    /home/guo/bioscope_cml/bioscope/bin/bioscope.sh -l workflow1.log analysis.plan
    exit 0

    This job is terminated after 10 hours with same result like that of the command line above!
    I know it might be really too long, just sincerely want to make it clear and, anyone, please help!!
    Thanks!!

    Leave a comment:


  • SoftGenetics
    replied
    Originally posted by rdeborja View Post
    Is anyone using/testing Bioscope as a replacement for corona lite and the whole transcriptome pipeline? I've recently installed it on our cluster and was curious to find other opinions/experiences with it.
    You may wish to try NextGENe's tool for this, it is quite robust.

    Leave a comment:


  • zee
    replied
    win804, if you are mapping to the whole genome using SOLiD reads perhaps you can try novoalignCS (available from www.novocraft.com). The whole genome index for colorspace will take about 6-8 minutes to build and you can map csfasta/csqual or csfastq straight away.

    PM me for more info if you would like some help.

    Leave a comment:


  • win804
    replied
    Originally posted by epigen View Post
    Maybe your reads are low quality, especially towards the end? Mapping to a transcriptome library might also not be such a good idea. Usually you align to the whole genome and afterwards assign the genomic regions to genes.
    Considering that there are two mismatches in color space for a SNP in nucleotide space, the default mismatch rate is too strict. Using BWA aln defaults, I got 34% mapping rate to the genome. Allowing for a higher mismatch rate with options l=25 n=8, as suggested somewhere in the forum, improved to 51% mapped but runtime was more than 5 times increased. BWA is great for nucleotide space but not optimized for color space.
    From my recent experience, I'd recommend using BFAST, there mapping rate was 69% with defaults and by smartly piping the commands, runtime was even lower than for BWA with l=25 n=8.
    Thank you very much epigen. I will try as what you recommended and see again. I am considering BFAST also, but building of the index is way too slow. I am still building it now. Probably after finished building the index, I will try the alignment using BFAST.

    Thanks again for your input.

    Leave a comment:


  • epigen
    replied
    Low mapping rate for SOLiD data with BWA

    Originally posted by win804 View Post
    Recently I obtained Solid RNA-Seq data. I have been creating the transcriptome library by using the annotation from UCSC hg19 refFlat file. However, when I aligned using BWA, the percentage of mapping rate is around 7%. I have built the color space index of the transcriptome library using the command:
    bwa index -a bwtsw -c hg19_transcript.fa
    then,
    bwa aln -c hg19_transcript.fa reads.fastq > align-reads.sai

    I was wondering, what could be the reason for such a low mapping rate? Well, I know that some can be mapped to exon junction, but I have created junction library as well, and the increment in mapping rate is very low (less than 0.1%).

    Anybody has similar experience? Do I need to tweak certain parameters in the bwa aln?

    Any input will be highly appreciated. Thanks!
    Maybe your reads are low quality, especially towards the end? Mapping to a transcriptome library might also not be such a good idea. Usually you align to the whole genome and afterwards assign the genomic regions to genes.
    Considering that there are two mismatches in color space for a SNP in nucleotide space, the default mismatch rate is too strict. Using BWA aln defaults, I got 34% mapping rate to the genome. Allowing for a higher mismatch rate with options l=25 n=8, as suggested somewhere in the forum, improved to 51% mapped but runtime was more than 5 times increased. BWA is great for nucleotide space but not optimized for color space.
    From my recent experience, I'd recommend using BFAST, there mapping rate was 69% with defaults and by smartly piping the commands, runtime was even lower than for BWA with l=25 n=8.

    Leave a comment:


  • win804
    replied
    Recently I obtained Solid RNA-Seq data. I have been creating the transcriptome library by using the annotation from UCSC hg19 refFlat file. However, when I aligned using BWA, the percentage of mapping rate is around 7%. I have built the color space index of the transcriptome library using the command:
    bwa index -a bwtsw -c hg19_transcript.fa
    then,
    bwa aln -c hg19_transcript.fa reads.fastq > align-reads.sai

    I was wondering, what could be the reason for such a low mapping rate? Well, I know that some can be mapped to exon junction, but I have created junction library as well, and the increment in mapping rate is very low (less than 0.1%).

    Anybody has similar experience? Do I need to tweak certain parameters in the bwa aln?

    Any input will be highly appreciated. Thanks!

    Leave a comment:

Latest Articles

Collapse

  • seqadmin
    Exploring the Dynamics of the Tumor Microenvironment
    by seqadmin




    The complexity of cancer is clearly demonstrated in the diverse ecosystem of the tumor microenvironment (TME). The TME is made up of numerous cell types and its development begins with the changes that happen during oncogenesis. “Genomic mutations, copy number changes, epigenetic alterations, and alternative gene expression occur to varying degrees within the affected tumor cells,” explained Andrea O’Hara, Ph.D., Strategic Technical Specialist at Azenta. “As...
    07-08-2024, 03:19 PM
  • seqadmin
    Exploring Human Diversity Through Large-Scale Omics
    by seqadmin


    In 2003, researchers from the Human Genome Project (HGP) announced the most comprehensive genome to date1. Although the genome wasn’t fully completed until nearly 20 years later2, numerous large-scale projects, such as the International HapMap Project and 1000 Genomes Project, continued the HGP's work, capturing extensive variation and genomic diversity within humans. Recently, newer initiatives have significantly increased in scale and expanded beyond genomics, offering a more detailed...
    06-25-2024, 06:43 AM

ad_right_rmr

Collapse

News

Collapse

Topics Statistics Last Post
Started by seqadmin, Today, 07:20 AM
0 responses
11 views
0 likes
Last Post seqadmin  
Started by seqadmin, 07-16-2024, 05:49 AM
0 responses
31 views
0 likes
Last Post seqadmin  
Started by seqadmin, 07-15-2024, 06:53 AM
0 responses
36 views
0 likes
Last Post seqadmin  
Started by seqadmin, 07-10-2024, 07:30 AM
0 responses
41 views
0 likes
Last Post seqadmin  
Working...
X