RepeatMasker for 7.5 GB of FASTA data

[roll]

Ah, actually repeat masking the reads won't do what you want then. What you want to do is align your reads to the mouse genome and then download the repeat masker output from UCSC. There are then a number of ways to compare your alignments to where the repeats are (e.g., just visually inspecting things with IGV or using bedtools or something similar to intersect the alignments).

great, so we are getting there

I would like to have something with numbers rather than examining it as visual.

how do I get the repeatmasker output from uscs? do i upload my sam file and then tick repeatmask option?

What about bedtools? which option i should look for?

[dpryan]

himmmmm, that is an interesting point. it is mouse data that i am dealing with so it has definitely been sequenced before.
I am not an expert in the field, still learning and my boss would like to know if and how many retrotransposons ( L1, SINE etc. ) are found in the data that we generated. May be there is a better to analyse this rather than RepeatMask?

Ah, actually repeat masking the reads won't do what you want then. What you want to do is align your reads to the mouse genome and then download the repeat masker output from UCSC. There are then a number of ways to compare your alignments to where the repeats are (e.g., just visually inspecting things with IGV or using bedtools or something similar to intersect the alignments).

[roll]

You don't want to repeat mask that file (you could, but the results would be completely useless). What is the actual biological question you're trying to answer. From context, I'm guess that this is an organism that hasn't been sequenced before and you'd like to determine its repeat structure or something like that. If that's the case, you need to de novo assemble the genome first. That will produce a proper fasta file that can be meaningfully repeatmasked.

himmmmm, that is an interesting point. it is mouse data that i am dealing with so it has definitely been sequenced before.
I am not an expert in the field, still learning and my boss would like to know if and how many retrotransposons ( L1, SINE etc. ) are found in the data that we generated. May be there is a better to analyse this rather than RepeatMask?

[dpryan]

What is the best way to convert fastq 2 fasta then so that i keep the chromosome information?

You don't want to repeat mask that file (you could, but the results would be completely useless). What is the actual biological question you're trying to answer. From context, I'm guess that this is an organism that hasn't been sequenced before and you'd like to determine its repeat structure or something like that. If that's the case, you need to de novo assemble the genome first. That will produce a proper fasta file that can be meaningfully repeatmasked.

[roll]

That looks like the read name for a FASTQ read, not a contig name. Are you sure this file is fasta?

What is the best way to convert fastq 2 fasta then so that i keep the chromosome information?

[roll]

I converted fastq to fasta myself. The original fastq have the headers like

@HS15_6922:7:2307:21180:13152#6/1
mySequenceHere
+
CBFFJ=BJIIJKFKHFLIJJLIIAGLCKKKIHKEKJKJ9JEKQJ;MJIJHNKLHKLHJI=KJ5DFCEIB+H?4?A?I31<FE=>ACG?F?A576;>./

[dpryan]

Thanks dpryan, but i do not know how to identify the chromosomes or contigs from my data.

The header of the FASTA file looks something like
>HS15_6922:7:2307:21180:13152#6/1

I am not sure how can I partition it using the above information. Can you please advise?

That looks like the read name for a FASTQ read, not a contig name. Are you sure this file is fasta?

[roll]

Thanks dpryan, but i do not know how to identify the chromosomes or contigs from my data.

The header of the FASTA file looks something like
>HS15_6922:7:2307:21180:13152#6/1

I am not sure how can I partition it using the above information. Can you please advise?

[dpryan]

Repeatmasker isn't exactly known for its speed. Since you have a cluster, your best option is to split the fasta file by chromosome/contig and run those on different nodes. You can then merge the results back together. In fact, I believe this is how the repeat masked files that are available from UCSC et al. were done.

[roll]

Hi,

I am trying to use RepeatMasker with full fasta file. The size of my fasta file is 7.5 GB and i would like to identify all repeats.

I use a cluster and requested 50 GB of memory from the cluster. And it still complaints that it is out of memory.

I am using the following options:

RepeatMasker -e crossmatch -q -species mouse -no_is -dir . -html -gff *.fasta

How can I run this and also I want it to be fast as well for a whole fasta file?