Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • RepeatMasker for 7.5 GB of FASTA data

    Hi,

    I am trying to use RepeatMasker with full fasta file. The size of my fasta file is 7.5 GB and i would like to identify all repeats.

    I use a cluster and requested 50 GB of memory from the cluster. And it still complaints that it is out of memory.

    I am using the following options:

    RepeatMasker -e crossmatch -q -species mouse -no_is -dir . -html -gff *.fasta

    How can I run this and also I want it to be fast as well for a whole fasta file?

  • #2
    Repeatmasker isn't exactly known for its speed. Since you have a cluster, your best option is to split the fasta file by chromosome/contig and run those on different nodes. You can then merge the results back together. In fact, I believe this is how the repeat masked files that are available from UCSC et al. were done.

    Comment


    • #3
      Thanks dpryan, but i do not know how to identify the chromosomes or contigs from my data.

      The header of the FASTA file looks something like
      >HS15_6922:7:2307:21180:13152#6/1

      I am not sure how can I partition it using the above information. Can you please advise?

      Comment


      • #4
        Originally posted by roll View Post
        Thanks dpryan, but i do not know how to identify the chromosomes or contigs from my data.

        The header of the FASTA file looks something like
        >HS15_6922:7:2307:21180:13152#6/1

        I am not sure how can I partition it using the above information. Can you please advise?
        That looks like the read name for a FASTQ read, not a contig name. Are you sure this file is fasta?

        Comment


        • #5
          I converted fastq to fasta myself. The original fastq have the headers like

          @HS15_6922:7:2307:21180:13152#6/1
          mySequenceHere
          +
          CBFFJ=BJIIJKFKHFLIJJLIIAGLCKKKIHKEKJKJ9JEKQJ;MJIJHNKLHKLHJI=KJ5DFCEIB+H?4?A?I31<FE=>ACG?F?A576;>./
          Last edited by roll; 09-30-2013, 01:56 AM.

          Comment


          • #6
            Originally posted by dpryan View Post
            That looks like the read name for a FASTQ read, not a contig name. Are you sure this file is fasta?
            What is the best way to convert fastq 2 fasta then so that i keep the chromosome information?

            Comment


            • #7
              Originally posted by roll View Post
              What is the best way to convert fastq 2 fasta then so that i keep the chromosome information?
              You don't want to repeat mask that file (you could, but the results would be completely useless). What is the actual biological question you're trying to answer. From context, I'm guess that this is an organism that hasn't been sequenced before and you'd like to determine its repeat structure or something like that. If that's the case, you need to de novo assemble the genome first. That will produce a proper fasta file that can be meaningfully repeatmasked.

              Comment


              • #8
                Originally posted by dpryan View Post
                You don't want to repeat mask that file (you could, but the results would be completely useless). What is the actual biological question you're trying to answer. From context, I'm guess that this is an organism that hasn't been sequenced before and you'd like to determine its repeat structure or something like that. If that's the case, you need to de novo assemble the genome first. That will produce a proper fasta file that can be meaningfully repeatmasked.
                himmmmm, that is an interesting point. it is mouse data that i am dealing with so it has definitely been sequenced before.
                I am not an expert in the field, still learning and my boss would like to know if and how many retrotransposons ( L1, SINE etc. ) are found in the data that we generated. May be there is a better to analyse this rather than RepeatMask?

                Comment


                • #9
                  Originally posted by roll View Post
                  himmmmm, that is an interesting point. it is mouse data that i am dealing with so it has definitely been sequenced before.
                  I am not an expert in the field, still learning and my boss would like to know if and how many retrotransposons ( L1, SINE etc. ) are found in the data that we generated. May be there is a better to analyse this rather than RepeatMask?
                  Ah, actually repeat masking the reads won't do what you want then. What you want to do is align your reads to the mouse genome and then download the repeat masker output from UCSC. There are then a number of ways to compare your alignments to where the repeats are (e.g., just visually inspecting things with IGV or using bedtools or something similar to intersect the alignments).

                  Comment


                  • #10
                    Originally posted by dpryan View Post
                    Ah, actually repeat masking the reads won't do what you want then. What you want to do is align your reads to the mouse genome and then download the repeat masker output from UCSC. There are then a number of ways to compare your alignments to where the repeats are (e.g., just visually inspecting things with IGV or using bedtools or something similar to intersect the alignments).
                    great, so we are getting there

                    I would like to have something with numbers rather than examining it as visual.

                    how do I get the repeatmasker output from uscs? do i upload my sam file and then tick repeatmask option?

                    What about bedtools? which option i should look for?

                    Comment


                    • #11
                      Originally posted by roll View Post
                      great, so we are getting there

                      I would like to have something with numbers rather than examining it as visual.

                      how do I get the repeatmasker output from uscs? do i upload my sam file and then tick repeatmask option?

                      What about bedtools? which option i should look for?
                      I have used tophat2 for mapping. Shall i use the bam file? Other outputs are

                      accepted_hits.bam
                      align_summary.txt
                      deletions.bed
                      insertions.bed
                      junctions.bed
                      logs
                      prep_reads.info
                      unmapped.bam

                      Comment


                      • #12
                        Originally posted by roll View Post
                        great, so we are getting there

                        I would like to have something with numbers rather than examining it as visual.

                        how do I get the repeatmasker output from uscs? do i upload my sam file and then tick repeatmask option?

                        What about bedtools? which option i should look for?
                        Originally posted by dpryan View Post
                        Ah, actually repeat masking the reads won't do what you want then. What you want to do is align your reads to the mouse genome and then download the repeat masker output from UCSC. There are then a number of ways to compare your alignments to where the repeats are (e.g., just visually inspecting things with IGV or using bedtools or something similar to intersect the alignments).
                        I have used tophat2 for mapping. Shall i use the bam file? Other outputs are

                        accepted_hits.bam
                        align_summary.txt
                        deletions.bed
                        insertions.bed
                        junctions.bed
                        logs
                        prep_reads.info
                        unmapped.bam

                        Comment


                        • #13
                          Assuming that you're using the mm10 reference, you can download the repeatmasker output here (mm9 is here). The general idea is to extract the type of feature(s) you want from the repeatmasker .out file and convert that to bed format and use "bedtools intersect ..." to get a count of how many reads align there. There are many other ways to do this, but that should work.

                          In fact, a more straight-forward way might be simply to run cufflinks on your alignments and then intersect the novel transcripts it finds with the repeatmasker output file. That might end up being easier.

                          Comment


                          • #14
                            BTW, you can find more detailed repeatMask results here:

                            http://hgdownload.soe.ucsc.edu/goldenPath/mm10/database/rmsk.{sql,txt.gz}

                            It is easy to convert this file to BED, I believe.

                            Comment


                            • #15
                              Originally posted by dpryan View Post
                              Assuming that you're using the mm10 reference, you can download the repeatmasker output here (mm9 is here). The general idea is to extract the type of feature(s) you want from the repeatmasker .out file and convert that to bed format and use "bedtools intersect ..." to get a count of how many reads align there. There are many other ways to do this, but that should work.

                              In fact, a more straight-forward way might be simply to run cufflinks on your alignments and then intersect the novel transcripts it finds with the repeatmasker output file. That might end up being easier.
                              Thanks a lot. This has been very helpful so far. i am trying what you have suggested and will let you know how it goes.

                              Do you know where i can download a genes and coordinates in a bed format? (Alternatively how can i assign gene names to my bed file)?

                              Comment

                              Latest Articles

                              Collapse

                              • seqadmin
                                Current Approaches to Protein Sequencing
                                by seqadmin


                                Proteins are often described as the workhorses of the cell, and identifying their sequences is key to understanding their role in biological processes and disease. Currently, the most common technique used to determine protein sequences is mass spectrometry. While still a valuable tool, mass spectrometry faces several limitations and requires a highly experienced scientist familiar with the equipment to operate it. Additionally, other proteomic methods, like affinity assays, are constrained...
                                04-04-2024, 04:25 PM
                              • seqadmin
                                Strategies for Sequencing Challenging Samples
                                by seqadmin


                                Despite advancements in sequencing platforms and related sample preparation technologies, certain sample types continue to present significant challenges that can compromise sequencing results. Pedro Echave, Senior Manager of the Global Business Segment at Revvity, explained that the success of a sequencing experiment ultimately depends on the amount and integrity of the nucleic acid template (RNA or DNA) obtained from a sample. “The better the quality of the nucleic acid isolated...
                                03-22-2024, 06:39 AM

                              ad_right_rmr

                              Collapse

                              News

                              Collapse

                              Topics Statistics Last Post
                              Started by seqadmin, 04-11-2024, 12:08 PM
                              0 responses
                              17 views
                              0 likes
                              Last Post seqadmin  
                              Started by seqadmin, 04-10-2024, 10:19 PM
                              0 responses
                              21 views
                              0 likes
                              Last Post seqadmin  
                              Started by seqadmin, 04-10-2024, 09:21 AM
                              0 responses
                              16 views
                              0 likes
                              Last Post seqadmin  
                              Started by seqadmin, 04-04-2024, 09:00 AM
                              0 responses
                              46 views
                              0 likes
                              Last Post seqadmin  
                              Working...
                              X