Download BOLD sequences with R and remove duplicates

[BioMidge]

Hello,

I am using the package bold in R to download COI sequences from BOLD and specifying the taxon of interest and the marker like this:

bold_seq(taxon='Chironomidae', marker='COI-5P', response=TRUE) as well as
bold_seq(taxon = 'Chironomidae', marker = "COX1", response = TRUE)

for which I get a fasta file. When I tried to format a database in BLAST (makeblastdb) using this file I got an error that duplicate IDs were present. I looked inside the fasta and found that the duplicates
were sequences of the same organism one from COI-5P and another from CAD, for example:

>CHRSV056-08|Cricotopus glacialis|CAD
CTGGCGTCAAAAGTATAAGCTCGCTGAGTGGATGAAGAAGCACAACGTCGTTGGAATCAGTGGAATTGACACC...
>CHRSV056-08|Cricotopus glacialis|COI-5P|KC130785
AACATTATATTTTATTTTCGGGGCTTGATCAGGGATAGTAGGAACTTCCTTAAGAATCTTAATTCGAGCTGAA...

and there's plenty of them. I can't removed them all one by one. There's like 160k sequences in this file.
Any idea how I can remove duplicate ID+seq from the fasta file or any way to fix the R comand to avoid getting sequences from other genes?
thanks

[GenoMax]

You could replace the spaces in the names with an underscore and see if that gets you past the "duplicate" name problem.

$ sed 's/ /\_/g' your_file > new_file

The example you posted has "duplicate" names since makeblastdb stops at the first space it encounters in the name. The sequences themselves are not duplicates.

[BioMidge]

Hello,

if I understand correctly, if I replaced the spaces with an _ using sed, wouldnt this happen on all the IDs and basically end up with duplicate IDs again only with underscores?
Yes, the sequences are not the same (different genes) so a "duplicate sequence" removal wouldnt work because they are not really duplicates.
Maybe a command that can look up the IDs and remove the ones with CAD? Is there something like that? But it should take the sequence out too..

[GenoMax]

So you are saying that for each entry there is a CAD and a COI-5P. If you just want to take out all entries with "CAD" then something simple like

$ cat your_file | grep -A 1 "COI-5P" > new_file

should work, as long as your file has sequences that don't wrap around on multiple lines.

BTW: With the two line example you posted "makeblastdb" did not generate an error for me.