Header Leaderboard Ad

Collapse

SMRTpipe Error after runCaToUnitig

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • SMRTpipe Error after runCaToUnitig

    I am getting strange memory issues after runCaToUnitig stage is compeleted:
    Code:
    [INFO] 2017-01-30 17:39:23,826 Workflow Completion Status 35/56 in (    ...... 62%) tasks completed.
    [INFO] 2017-01-30 17:39:28,836 task P_AssembleUnitig.writeRunCASpec successfully completed (4.00 sec (0.07 min)). freed 1 slots, using 1 slots.
    [INFO] 2017-01-30 17:39:28,837 Workflow Completion Status 36/56 in (    ...... 64%) tasks completed.
    [INFO] 2017-01-30 17:39:33,845 Starting task://Anonymous/P_AssembleUnitig/runCaToUnitig
    [INFO] 2017-01-30 17:39:50,860 task P_PreAssemblerDagcon.preAssemblerJsonReport successfully completed (25.00 sec (0.42 min)). freed 1 slots, using 1 slots.
    [INFO] 2017-01-30 17:39:50,861 Workflow Completion Status 37/56 in (    ...... 66%) tasks completed.
    slurmstepd: Job 4910074 exceeded memory limit (52468956 > 52428800), being killed
    slurmstepd: *** JOB 4910074 ON c5a-s17 CANCELLED AT 2017-01-30T17:47:40 ***
    slurmstepd: Exceeded job memory limit
    slurmstepd: Exceeded step memory limit at some point.
    I am using 50GB memory which should be more than enough for a 10Mb organism. I get stuck with the same error while using data from 1 cell or data from 3 cells.
    All logs seem to exit with code 0:
    Code:
     tail */*.log -n 1
    ==> P_AssembleUnitig/genFrgFile.log <==
    Task genFrgFile with nproc 1 with exit code 0.
    
    ==> P_AssembleUnitig/runCaToUnitig.log <==
    exit ${rcode}
    ==> P_AssembleUnitig/writeRunCASpec.log <==
    Task writeRunCASpec with nproc 1 with exit code 0.
    
    ==> P_Fetch/adapterRpt.log <==
    Task adapterRpt with nproc 1 with exit code 0.
    
    ==> P_Fetch/overviewRpt.log <==
    Task overviewRpt with nproc 1 with exit code 0.
    
    ==> P_Fetch/toFofn.log <==
    Task toFofn with nproc 1 with exit code 0.
    
    ==> P_Filter/filter_001of009.log <==
    Task filter_001of009 with nproc 2 with exit code 0.
    
    ==> P_Filter/filter_002of009.log <==
    Task filter_002of009 with nproc 2 with exit code 0.
    
    ==> P_Filter/filter_003of009.log <==
    Task filter_003of009 with nproc 2 with exit code 0.
    
    ==> P_Filter/filter_004of009.log <==
    Task filter_004of009 with nproc 2 with exit code 0.
    
    ==> P_Filter/filter_005of009.log <==
    Task filter_005of009 with nproc 2 with exit code 0.
    
    ==> P_Filter/filter_006of009.log <==
    Task filter_006of009 with nproc 2 with exit code 0.
    
    ==> P_Filter/filter_007of009.log <==
    Task filter_007of009 with nproc 2 with exit code 0.
    
    ==> P_Filter/filter_008of009.log <==
    Task filter_008of009 with nproc 2 with exit code 0.
    
    ==> P_Filter/filter_009of009.log <==
    Task filter_009of009 with nproc 2 with exit code 0.
    
    ==> P_Filter/filter.plsFofn.Scatter.log <==
    Task filter.plsFofn.Scatter with nproc 1 with exit code 0.
    
    ==> P_Filter/filter.rgnFofn.Gather.log <==
    Task filter.rgnFofn.Gather with nproc 1 with exit code 0.
    
    ==> P_Filter/filter.summary.Gather.log <==
    Task filter.summary.Gather with nproc 1 with exit code 0.
    
    ==> P_FilterReports/loadingRpt.log <==
    Task loadingRpt with nproc 1 with exit code 0.
    
    ==> P_FilterReports/statsRpt.log <==
    Task statsRpt with nproc 1 with exit code 0.
    
    ==> P_FilterReports/subreadRpt.log <==
    Task subreadRpt with nproc 1 with exit code 0.
    
    ==> P_Filter/subreads_001of009.log <==
    Task subreads_001of009 with nproc 1 with exit code 0.
    
    ==> P_Filter/subreads_002of009.log <==
    Task subreads_002of009 with nproc 1 with exit code 0.
    
    ==> P_Filter/subreads_003of009.log <==
    Task subreads_003of009 with nproc 1 with exit code 0.
    
    ==> P_Filter/subreads_004of009.log <==
    Task subreads_004of009 with nproc 1 with exit code 0.
    
    ==> P_Filter/subreads_005of009.log <==
    Task subreads_005of009 with nproc 1 with exit code 0.
    
    ==> P_Filter/subreads_006of009.log <==
    Task subreads_006of009 with nproc 1 with exit code 0.
    
    ==> P_Filter/subreads_007of009.log <==
    Task subreads_007of009 with nproc 1 with exit code 0.
    
    ==> P_Filter/subreads_008of009.log <==
    Task subreads_008of009 with nproc 1 with exit code 0.
    
    ==> P_Filter/subreads_009of009.log <==
    Task subreads_009of009 with nproc 1 with exit code 0.
    
    ==> P_Filter/subreads.subreadFastq.Gather.log <==
    Task subreads.subreadFastq.Gather with nproc 1 with exit code 0.
    
    ==> P_Filter/subreads.subreads.Gather.log <==
    Task subreads.subreads.Gather with nproc 1 with exit code 0.
    
    ==> P_Filter/subreadSummary.log <==
    Task subreadSummary with nproc 1 with exit code 0.
    
    ==> P_PreAssemblerDagcon/filterLongReadsByLength.log <==
    Task filterLongReadsByLength with nproc 1 with exit code 0.
    
    ==> P_PreAssemblerDagcon/hgapAlignForCorrection.log <==
    Task hgapAlignForCorrection with nproc 12 with exit code 0.
    
    ==> P_PreAssemblerDagcon/hgapCorrection.log <==
    Task hgapCorrection with nproc 12 with exit code 0.
    
    ==> P_PreAssemblerDagcon/hgapFilterM4.log <==
    Task hgapFilterM4 with nproc 1 with exit code 0.
    
    ==> P_PreAssemblerDagcon/preAssemblerJsonReport.log <==
    Task preAssemblerJsonReport with nproc 1 with exit code 0.
    The log from runCaToUnitig.log is as follows:

    Code:
    #!/bin/bash
    # Setting up SMRTpipe environment
    echo "Setting up ENV on $(uname -n)" for task runCaToUnitig
    
    SEYMOUR_HOME=/apps/pacbio/2.3.0.140936.p5/install/smrtanalysis_2.3.0.140936
    source $SEYMOUR_HOME/etc/setup.sh
    
    # Create the local TMP dir if it doesn't exist
    tmp_dir=$(readlink -m "/apps/pacbio/2.3.0.140936.p5/tmpdir")
    if [ ! -e "$tmp_dir" ]; then
       stat=0
       mkdir -p $tmp_dir || stat=$?
       if [[ $stat -ne 0 ]]; then
           echo "SMRTpipe Unable to create TMP dir '/apps/pacbio/2.3.0.140936.p5/tmpdir' on $(uname -n)" 1>&2
           exit 1
       else
           echo "successfully created or found TMP dir '/apps/pacbio/2.3.0.140936.p5/tmpdir'"
       fi
    elif [[ ! -d "$tmp_dir" ]]; then
       echo "SMRTpipe TMP /apps/pacbio/2.3.0.140936.p5/tmpdir must be a directory on $(uname -n)" 1>&2
       exit 1
    fi
    
    ########### TASK metadata #############
    # Task            : runCaToUnitig
    # Module          : P_AssembleUnitig
    # Module Version  : 2.2.UnknownVersion
    # TaskType        : None
    # URL             : task://Anonymous/P_AssembleUnitig/runCaToUnitig
    # createdAt       : 2017-01-30 11:18:51.164802
    # createdAt (UTC) : 2017-01-30 16:18:51.164828
    # ncmds           : 2
    # LogPath         : /smrtpipe2/matt_review/new/log/P_AssembleUnitig/runCaToUnitig.log
    # Script Path     : /smrtpipe2/matt_review/new/workflow/P_AssembleUnitig/runCaToUnitig.sh
    
    # Input       : /smrtpipe2/matt_review/new/data/corrected.frg
    # Input       : /smrtpipe2/matt_review/new/data/runCA.spec
    # Output      :/smrtpipe2/matt_review/new/data/ca_finished
    #
    ########### END TASK metadata #############
    
    cd /smrtpipe2/matt_review/new/log/P_AssembleUnitig
    # Writing to log file
    cat /smrtpipe2/matt_review/new/workflow/P_AssembleUnitig/runCaToUnitig.sh >> /smrtpip
    e2/matt_review/new/log/P_AssembleUnitig/runCaToUnitig.log;
    
    
    
    echo "Running task://Anonymous/P_AssembleUnitig/runCaToUnitig on $(uname -a)"
    
    echo "Started on $(date -u)"
    echo 'Validating existence of Input Files'
    if [ -e /smrtpipe2/matt_review/new/data/corrected.frg ]
    then
    echo 'Successfully found /smrtpipe2/matt_review/new/data/corrected.frg'
    else
    echo 'WARNING: Unable to find necessary input file, or dir /smrtpipe2/matt_review/new/data/corrected.frg.'
    fi
    if [ -e /smrtpipe2/matt_review/new/data/runCA.spec ]
    then
    echo 'Successfully found /smrtpipe2/matt_review/new/data/runCA.spec'
    else
    echo 'WARNING: Unable to find necessary input file, or dir /smrtpipe2/matt_review/new/data/runCA.spec.'
    fi
    echo 'Successfully validated input files'
    
    # Task runCaToUnitig commands:
    
    
    # Completed writing Task runCaToUnitig commands
    
    
    # Task 1
    runCA -d /smrtpipe2/matt_review/new/data -p celera-assembler -s /smrtpipe2/matt_revie
    w/new/data/runCA.spec /smrtpipe2/matt_review/new/data/corrected.frg || exit $?
    echo "Task 1 completed at $(date)"
    # Task 2
    touch /smrtpipe2/matt_review/new/data/ca_finished || exit $?
    echo "Task 2 completed at $(date)"
    
    
    
    rcode=$?
    echo "Finished on $(date -u)"
    echo "Task runCaToUnitig with nproc 1 with exit code ${rcode}."
    exit ${rcode}
    Can anyone please help?
    Thanks

Latest Articles

Collapse

  • seqadmin
    A Brief Overview and Common Challenges in Single-cell Sequencing Analysis
    by seqadmin


    ​​​​​​The introduction of single-cell sequencing has advanced the ability to study cell-to-cell heterogeneity. Its use has improved our understanding of somatic mutations1, cell lineages2, cellular diversity and regulation3, and development in multicellular organisms4. Single-cell sequencing encompasses hundreds of techniques with different approaches to studying the genomes, transcriptomes, epigenomes, and other omics of individual cells. The analysis of single-cell sequencing data i...

    01-24-2023, 01:19 PM
  • seqadmin
    Introduction to Single-Cell Sequencing
    by seqadmin
    Single-cell sequencing is a technique used to investigate the genome, transcriptome, epigenome, and other omics of individual cells using high-throughput sequencing. This technology has provided many scientific breakthroughs and continues to be applied across many fields, including microbiology, oncology, immunology, neurobiology, precision medicine, and stem cell research.

    The advancement of single-cell sequencing began in 2009 when Tang et al. investigated the single-cell transcriptomes
    ...
    01-09-2023, 03:10 PM

ad_right_rmr

Collapse
Working...
X