Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Rsamtools memory leak

    Hi there,

    I've just updated Rsamtools to the latest version ("1.22.0") and I've noticed a weird memory leak that is crashing my system.

    I have a folder of ~150 bam files that I'm trying to process, feeding in a data.frame called filter, that splits the genome into 7744 genomic coordinates

    Code:
    > head(filter)
            V1       V2       V3                         V4
    1 GL896898        0 30180132        GL896898:0-30180132
    2 GL896898 30180133 52375790 GL896898:30180133-52375790
    3 GL896899        0 24984016        GL896899:0-24984016
    4 GL896899 24984017 40953715 GL896899:24984017-40953715
    5 GL896905        0 18438707        GL896905:0-18438707
    6 GL896905 18438708 27910907 GL896905:18438708-27910907
    For some reason, Rsamtools is not releasing RAM after each iteration. I assumed it was my code, but I just ran the following simple test, and unless I'm being completely stupid, Rsamtools is doing something funny. To test this I looped one bam file with the following script and monitored RAM usage:

    Code:
    #R implementation of top
    library(NCmisc)
    
    #R value of all objects
    library(pryr)
    
    library(Rsamtools)
    
    filter=read.table('ferret_merged_sce_events.bed')
    fileName='AAAGCA.merge.bam'
    objectRam=vector()
    totalRam=vector()
    
    for(seq in 1:10)
    {
         message(paste("iteration", seq))
         bam.dataframe <- scanBam(fileName, param=ScanBamParam(which=GRanges(seqnames = c(as.character(filter[,1])), ranges = IRanges(c(filter[,2]), c(filter[,3]) )), mapqFilter=10, what=c("rname","pos","strand")))
         #measure RAM across all R objects
         objectRam <- c(objectRam, mem_used()[1]/10^6)
         #measure RAM across system
         totalRam <- c(totalRam,  suppressWarnings(top(CPU=F)$RAM$used*10^6))
         #remove scanBam object to reset for next iteration
         rm(bam.dataframe)
         #empty garbage collection
         gc()
    
    }
    plot(objectRam, type='l', xlab='iteration', ylab='Ram usage (Mb)')
    plot(totalRam, type='l', xlab='iteration', ylab='Ram usage (Mb)')
    This generates the two attached graphs. You'll see that while the RAM used for the R object remains stable, the overall RAM usage of my computer increases, and only decreases when I end my R session. Is there any way of purging the RAM? Is this a bug in Rsamtools or have I done something wrong? The additive overall RAM usage also occurs when the scanBam(which()) is not used, but occurs at a much reduced rate.

    Any help would be greatly appreciated.
    Attached Files

  • #2
    What happens if you run gc() to have the garbage collector run?

    Comment


    • #3
      Hi dpryan,

      So after running the loop described, my system memory states this session is using 2.579Gb RAM.

      If I run gc() a few times, the system memory usage doesn't change and I get the following output:

      Code:
      > gc()
                used  (Mb) gc trigger  (Mb) max used  (Mb)
      Ncells 1912702 102.2    3205452 171.2  3205452 171.2
      Vcells 1579324  12.1    7866470  60.1 21256102 162.2
      > gc()
                used  (Mb) gc trigger  (Mb) max used  (Mb)
      Ncells 1912693 102.2    3205452 171.2  3205452 171.2
      Vcells 1579327  12.1    6293176  48.1 21256102 162.2
      > gc()
                used  (Mb) gc trigger  (Mb) max used  (Mb)
      Ncells 1912693 102.2    3205452 171.2  3205452 171.2
      Vcells 1579327  12.1    5034540  38.5 21256102 162.2
      > gc()
                used  (Mb) gc trigger  (Mb) max used  (Mb)
      Ncells 1912693 102.2    3205452 171.2  3205452 171.2
      Vcells 1579327  12.1    5034540  38.5 21256102 162.2
      > gc()
                used  (Mb) gc trigger  (Mb) max used  (Mb)
      Ncells 1912693 102.2    3205452 171.2  3205452 171.2
      Vcells 1579327  12.1    5034540  38.5 21256102 162.2

      Comment


      • #4
        Apparently RSamtools isn't using the garbage collector. You might post this to the bioconductor support site and see if the maintainers reply.

        Comment


        • #5
          Great,

          Thanks dpryan. I'll cross post at bioconductor support and wait for a site maintainer to get back to me.

          Cheers!

          Comment

          Latest Articles

          Collapse

          • seqadmin
            Genetic Variation in Immunogenetics and Antibody Diversity
            by seqadmin



            The field of immunogenetics explores how genetic variations influence immune responses and susceptibility to disease. In a recent SEQanswers webinar, Oscar Rodriguez, Ph.D., Postdoctoral Researcher at the University of Louisville, and Ruben Martínez Barricarte, Ph.D., Assistant Professor of Medicine at Vanderbilt University, shared recent advancements in immunogenetics. This article discusses their research on genetic variation in antibody loci, antibody production processes,...
            11-06-2024, 07:24 PM
          • seqadmin
            Choosing Between NGS and qPCR
            by seqadmin



            Next-generation sequencing (NGS) and quantitative polymerase chain reaction (qPCR) are essential techniques for investigating the genome, transcriptome, and epigenome. In many cases, choosing the appropriate technique is straightforward, but in others, it can be more challenging to determine the most effective option. A simple distinction is that smaller, more focused projects are typically better suited for qPCR, while larger, more complex datasets benefit from NGS. However,...
            10-18-2024, 07:11 AM

          ad_right_rmr

          Collapse

          News

          Collapse

          Topics Statistics Last Post
          Started by seqadmin, 11-08-2024, 11:09 AM
          0 responses
          39 views
          0 likes
          Last Post seqadmin  
          Started by seqadmin, 11-08-2024, 06:13 AM
          0 responses
          30 views
          0 likes
          Last Post seqadmin  
          Started by seqadmin, 11-01-2024, 06:09 AM
          0 responses
          34 views
          0 likes
          Last Post seqadmin  
          Started by seqadmin, 10-30-2024, 05:31 AM
          0 responses
          23 views
          0 likes
          Last Post seqadmin  
          Working...
          X