Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Rsamtools memory leak

    Hi there,

    I've just updated Rsamtools to the latest version ("1.22.0") and I've noticed a weird memory leak that is crashing my system.

    I have a folder of ~150 bam files that I'm trying to process, feeding in a data.frame called filter, that splits the genome into 7744 genomic coordinates

    Code:
    > head(filter)
            V1       V2       V3                         V4
    1 GL896898        0 30180132        GL896898:0-30180132
    2 GL896898 30180133 52375790 GL896898:30180133-52375790
    3 GL896899        0 24984016        GL896899:0-24984016
    4 GL896899 24984017 40953715 GL896899:24984017-40953715
    5 GL896905        0 18438707        GL896905:0-18438707
    6 GL896905 18438708 27910907 GL896905:18438708-27910907
    For some reason, Rsamtools is not releasing RAM after each iteration. I assumed it was my code, but I just ran the following simple test, and unless I'm being completely stupid, Rsamtools is doing something funny. To test this I looped one bam file with the following script and monitored RAM usage:

    Code:
    #R implementation of top
    library(NCmisc)
    
    #R value of all objects
    library(pryr)
    
    library(Rsamtools)
    
    filter=read.table('ferret_merged_sce_events.bed')
    fileName='AAAGCA.merge.bam'
    objectRam=vector()
    totalRam=vector()
    
    for(seq in 1:10)
    {
         message(paste("iteration", seq))
         bam.dataframe <- scanBam(fileName, param=ScanBamParam(which=GRanges(seqnames = c(as.character(filter[,1])), ranges = IRanges(c(filter[,2]), c(filter[,3]) )), mapqFilter=10, what=c("rname","pos","strand")))
         #measure RAM across all R objects
         objectRam <- c(objectRam, mem_used()[1]/10^6)
         #measure RAM across system
         totalRam <- c(totalRam,  suppressWarnings(top(CPU=F)$RAM$used*10^6))
         #remove scanBam object to reset for next iteration
         rm(bam.dataframe)
         #empty garbage collection
         gc()
    
    }
    plot(objectRam, type='l', xlab='iteration', ylab='Ram usage (Mb)')
    plot(totalRam, type='l', xlab='iteration', ylab='Ram usage (Mb)')
    This generates the two attached graphs. You'll see that while the RAM used for the R object remains stable, the overall RAM usage of my computer increases, and only decreases when I end my R session. Is there any way of purging the RAM? Is this a bug in Rsamtools or have I done something wrong? The additive overall RAM usage also occurs when the scanBam(which()) is not used, but occurs at a much reduced rate.

    Any help would be greatly appreciated.
    Attached Files

  • #2
    What happens if you run gc() to have the garbage collector run?

    Comment


    • #3
      Hi dpryan,

      So after running the loop described, my system memory states this session is using 2.579Gb RAM.

      If I run gc() a few times, the system memory usage doesn't change and I get the following output:

      Code:
      > gc()
                used  (Mb) gc trigger  (Mb) max used  (Mb)
      Ncells 1912702 102.2    3205452 171.2  3205452 171.2
      Vcells 1579324  12.1    7866470  60.1 21256102 162.2
      > gc()
                used  (Mb) gc trigger  (Mb) max used  (Mb)
      Ncells 1912693 102.2    3205452 171.2  3205452 171.2
      Vcells 1579327  12.1    6293176  48.1 21256102 162.2
      > gc()
                used  (Mb) gc trigger  (Mb) max used  (Mb)
      Ncells 1912693 102.2    3205452 171.2  3205452 171.2
      Vcells 1579327  12.1    5034540  38.5 21256102 162.2
      > gc()
                used  (Mb) gc trigger  (Mb) max used  (Mb)
      Ncells 1912693 102.2    3205452 171.2  3205452 171.2
      Vcells 1579327  12.1    5034540  38.5 21256102 162.2
      > gc()
                used  (Mb) gc trigger  (Mb) max used  (Mb)
      Ncells 1912693 102.2    3205452 171.2  3205452 171.2
      Vcells 1579327  12.1    5034540  38.5 21256102 162.2

      Comment


      • #4
        Apparently RSamtools isn't using the garbage collector. You might post this to the bioconductor support site and see if the maintainers reply.

        Comment


        • #5
          Great,

          Thanks dpryan. I'll cross post at bioconductor support and wait for a site maintainer to get back to me.

          Cheers!

          Comment

          Latest Articles

          Collapse

          • seqadmin
            The Impact of AI in Genomic Medicine
            by seqadmin



            Artificial intelligence (AI) has evolved from a futuristic vision to a mainstream technology, highlighted by the introduction of tools like OpenAI's ChatGPT and Google's Gemini. In recent years, AI has become increasingly integrated into the field of genomics. This integration has enabled new scientific discoveries while simultaneously raising important ethical questions1. Interviews with two researchers at the center of this intersection provide insightful perspectives into...
            02-26-2024, 02:07 PM
          • seqadmin
            Multiomics Techniques Advancing Disease Research
            by seqadmin


            New and advanced multiomics tools and technologies have opened new avenues of research and markedly enhanced various disciplines such as disease research and precision medicine1. The practice of merging diverse data from various ‘omes increasingly provides a more holistic understanding of biological systems. As Maddison Masaeli, Co-Founder and CEO at Deepcell, aptly noted, “You can't explain biology in its complex form with one modality.”

            A major leap in the field has
            ...
            02-08-2024, 06:33 AM

          ad_right_rmr

          Collapse

          News

          Collapse

          Topics Statistics Last Post
          Started by seqadmin, 02-28-2024, 06:12 AM
          0 responses
          23 views
          0 likes
          Last Post seqadmin  
          Started by seqadmin, 02-23-2024, 04:11 PM
          0 responses
          70 views
          0 likes
          Last Post seqadmin  
          Started by seqadmin, 02-21-2024, 08:52 AM
          0 responses
          79 views
          0 likes
          Last Post seqadmin  
          Started by seqadmin, 02-20-2024, 08:57 AM
          0 responses
          68 views
          0 likes
          Last Post seqadmin  
          Working...
          X