[zhengz]

Hi ali,

I guess the sfffile looks for the barcode set which you define in your barcode file and the name you specify in the command line after -s. If no match, it continues to look for gsMID and rlMID. Maybe you can remove the lines after gsMID or rlMID (yes, save a copy first), and see what the error message suggests you. Hope it works out for you eventually.

A side note: I think it is better to avoid using "*.sff", which will combine the samples having the same barcode but from different physically separated Regions (Lanes) in one sff file. Better to spell out the sff file name, instead of using wildcard. But if you have no sent the results to others, no worries, you will notice the problem, and it is quick to run the sfffile command again. It finishes in a minute.

[aligenie]

With -s you tell sfffile to split default barcodes from default barcode file.
Put your barcodes as supposed into a file, and then, in your case use e.g.

sfffile -s barcode -o myNewSFF -mcf /path/to/barcodefile *.sff

cheers,
Sven

For some reason, that still doesn't work. Oh well, thanks for your help. Maybe I'll figure something out eventually. This is a bit of a mystery to me!

[sklages]

So when I try these commands, I get ERROR: Default MID sets gsMIDs and flMIDs do not exist in configuration file. What is odd is that they do. I don't even want to use the MIDConfig.Parse file which is why I use the -mcf command so I think it is very odd. Should I put my barcode.txt file in the config folder?

With -s you tell sfffile to split default barcodes from default barcode file.
Put your barcodes as supposed into a file, and then, in your case use e.g.

sfffile -s barcode -o myNewSFF -mcf /path/to/barcodefile *.sff

cheers,
Sven

[aligenie]

Hi aligenie,

Since you define the name for the set of barcodes as 'barcode', which is the line above {, would the following command work?

sfffile -mcf barcode.txt -s barcode read.sff


In my case, for my own customized adapters, I use a barcode file with example in the comment lines (change the x with your barcodes):

/* User-defined MID sets for the 8 Y adapters...

An example:

sfffile -s Y -mcf Yscheme.txt -o region1 NameOfYourSFFfile1.sff > MIDyieldR1.txt
sfffile -s Y -mcf Yscheme.txt -o region2 NameOfYourSFFfile2.sff > MIDyieldR2.txt

*/
Y
{
mid = "Y3", "xxxxxxxxxx", 1, "xxxxxxxxxx";
mid = "Y5", "xxxxxxxxxx", 1, "xxxxxxxxxx";
mid = "Y8", "xxxxxxxxxx", 1, "xxxxxxxxxx";
mid = "Y9", "xxxxxxxxxx", 1, "xxxxxxxxxx";
mid = "Y10", "xxxxxxxxxx", 1, "xxxxxxxxxx";
mid = "Y11", "xxxxxxxxxx", 1, "xxxxxxxxxx";
mid = "Ya1", "xxxxxxxxxx", 1, "xxxxxxxxxx";
mid = "Ya2", "xxxxxxxxxx", 1, "xxxxxxxxxx";
}

So when I try these commands, I get ERROR: Default MID sets gsMIDs and flMIDs do not exist in configuration file. What is odd is that they do. I don't even want to use the MIDConfig.Parse file which is why I use the -mcf command so I think it is very odd. Should I put my barcode.txt file in the config folder?

[JackieBadger]

What error do you get? The syntax looks ok.
How did you call 'sfffile' (command line)?

And Geneious, that's my impression too, very nice looking but too slow for NGS.

cheers,
Sven

Their latest release 5.4.1 is supposed to be designed for NGS, but yes I agree that loading and moving files around is SLOW, and can cause the program to hang!
jMHC operates on a much better level for barcodes.

Cheers
j

[zhengz]

Hi aligenie,

Since you define the name for the set of barcodes as 'barcode', which is the line above {, would the following command work?

sfffile -mcf barcode.txt -s barcode read.sff


In my case, for my own customized adapters, I use a barcode file with example in the comment lines (change the x with your barcodes):

/* User-defined MID sets for the 8 Y adapters...

An example:

sfffile -s Y -mcf Yscheme.txt -o region1 NameOfYourSFFfile1.sff > MIDyieldR1.txt
sfffile -s Y -mcf Yscheme.txt -o region2 NameOfYourSFFfile2.sff > MIDyieldR2.txt

*/
Y
{
mid = "Y3", "xxxxxxxxxx", 1, "xxxxxxxxxx";
mid = "Y5", "xxxxxxxxxx", 1, "xxxxxxxxxx";
mid = "Y8", "xxxxxxxxxx", 1, "xxxxxxxxxx";
mid = "Y9", "xxxxxxxxxx", 1, "xxxxxxxxxx";
mid = "Y10", "xxxxxxxxxx", 1, "xxxxxxxxxx";
mid = "Y11", "xxxxxxxxxx", 1, "xxxxxxxxxx";
mid = "Ya1", "xxxxxxxxxx", 1, "xxxxxxxxxx";
mid = "Ya2", "xxxxxxxxxx", 1, "xxxxxxxxxx";
}

[sklages]

Hi Sven, thanks for your help. Unfortunately I still cannot get sfffile to parse by MID. I used sfffile -mcf barcode.txt -s read.sff and I get errors. My barcode file looks like this
barcode
{
mid = "MID1", "ACGAGTGCGT", 2;
[...] mid = "MID67", "TCGATAGTGA", 2;
}

Any idea with the -mcf function isn't working? Is my syntax wrong? sorry for all the questions but this is frustrating!!

I find geneious to be really slow....

Cheers

What error do you get? The syntax looks ok.
How did you call 'sfffile' (command line)?

And Geneious, that's my impression too, very nice looking but too slow for NGS.

cheers,
Sven

[aligenie]

Yes, sfffile is very fast and reliable. I don't know the other tools mentioned, but the advantage of sfffile is, that it works on the SFF file and generates new SFF files. If you are done with MID clipping, then you extract the fasta sequences from the newly created SFF files (without MID, except when you use '-n' with sffinfo).

cheers,
Sven

Hi Sven, thanks for your help. Unfortunately I still cannot get sfffile to parse by MID. I used sfffile -mcf barcode.txt -s read.sff and I get errors. My barcode file looks like this
barcode
{
mid = "MID1", "ACGAGTGCGT", 2;
mid = "MID2", "ACGCTCGACA", 2;
mid = "MID3", "AGACGCACTC", 2;
mid = "MID5", "ATCAGACACG", 2;
mid = "MID6", "ATATCGCGAG", 2;
mid = "MID7", "CGTGTCTCTA", 2;
mid = "MID8", "CTCGCGTGTC", 2;
mid = "MID10", "TCTCTATGCG", 2;
mid = "MID11", "TGATACGTCT", 2;
mid = "MID13", "CATAGTAGTG", 2;
mid = "MID14", "CGAGAGATAC", 2;
mid = "MID15", "ATACGACGTA", 2;
mid = "MID16", "TCACGTACTA", 2;
mid = "MID17", "CGTCTAGTAC", 2;
mid = "MID18", "TCTACGTAGC", 2;
mid = "MID19", "TGTACTACTC", 2;
mid = "MID20", "ACGACTACAG", 2;
mid = "MID21", "CGTAGACTAG", 2;
mid = "MID22", "TACGAGTATG", 2;
mid = "MID23", "TACTCTCGTG", 2;
mid = "MID24", "TAGAGACGAG", 2;
mid = "MID25", "TCGTCGCTCG", 2;
mid = "MID26", "ACATACGCGT", 2;
mid = "MID27", "ACGCGAGTAT", 2;
mid = "MID28", "ACTACTATGT", 2;
mid = "MID68", "TCGCTGCGTA", 2;
mid = "MID30", "AGACTATACT", 2;
mid = "MID31", "AGCGTCGTCT", 2;
mid = "MID32", "AGTACGCTAT", 2;
mid = "MID33", "ATAGAGTACT", 2;
mid = "MID34", "CACGCTACGT", 2;
mid = "MID35", "CAGTAGACGT", 2;
mid = "MID36", "CGACGTGACT", 2;
mid = "MID37", "TACACACACT", 2;
mid = "MID38", "TACACGTGAT", 2;
mid = "MID39", "TACAGATCGT", 2;
mid = "MID40", "TACGCTGTCT", 2;
mid = "MID69", "TCTGACGTCA", 2;
mid = "MID42", "TCGATCACGT", 2;
mid = "MID43", "TCGCACTAGT", 2;
mid = "MID44", "TCTAGCGACT", 2;
mid = "MID45", "TCTATACTAT", 2;
mid = "MID46", "TGACGTATGT", 2;
mid = "MID47", "TGTGAGTAGT", 2;
mid = "MID48", "ACAGTATATA", 2;
mid = "MID49", "ACGCGATCGA", 2;
mid = "MID50", "ACTAGCAGTA", 2;
mid = "MID67", "TCGATAGTGA", 2;
}

Any idea with the -mcf function isn't working? Is my syntax wrong? sorry for all the questions but this is frustrating!!

I find geneious to be really slow....

Cheers

[sklages]

I wish our core did this!! LOL they definitely don't. I just received several .sff files but they are not parsed by MID or for anything else for that matter. I guess I will look into Roche tools for separating by MID and trimming. Sfffile can do something like this I think although I find the syntax very confusing. Is this what you use?

Yes, sfffile is very fast and reliable. I don't know the other tools mentioned, but the advantage of sfffile is, that it works on the SFF file and generates new SFF files. If you are done with MID clipping, then you extract the fasta sequences from the newly created SFF files (without MID, except when you use '-n' with sffinfo).

cheers,
Sven

[JackieBadger]

I wish our core did this!! LOL they definitely don't. I just received several .sff files but they are not parsed by MID or for anything else for that matter. I guess I will look into Roche tools for separating by MID and trimming. Sfffile can do something like this I think although I find the syntax very confusing. Is this what you use?

jMHC and Geneious (links above) are super easy to use, with graphical interfaces.
You designate your primer, adapter length, and Bob's your uncle!

[aligenie]

Yes, that's what you usually do: split a run SFF file into individual SFF files according to their barcodes/MIDs. If you use the Roche tools for this task, then the offsets are getting shiftet. We usually send these SFF files to our customers, MID removed, files sorted.
The OP didn't mention what kind of SFF he received .. individual ones? Whole region SFFs?

cheers,
Sven

I wish our core did this!! LOL they definitely don't. I just received several .sff files but they are not parsed by MID or for anything else for that matter. I guess I will look into Roche tools for separating by MID and trimming. Sfffile can do something like this I think although I find the syntax very confusing. Is this what you use?

[JackieBadger]

Ahhh so you preprocess the MIDs for the customer?
How nice of you! haha I'm sure most I know would charge $ for this.

Anyway, the programs I mentioned are a great way for a non-code based approach.

Cheers

J

[sklages]

Right, but you remove the MIDs once you have sorted by them. I think this was the original question "how can i sort by barcodes?"

Yes, that's what you usually do: split a run SFF file into individual SFF files according to their barcodes/MIDs. If you use the Roche tools for this task, then the offsets are getting shiftet. We usually send these SFF files to our customers, MID removed, files sorted.
The OP didn't mention what kind of SFF he received .. individual ones? Whole region SFFs?

cheers,
Sven

[JackieBadger]

Well, when I "remove MIDs", I do this usually by splitting a SFF file from either region of the 454 into individual SFF files (according to their MID); in this step, the MID is removed (== offset shifted in SFF file). That's a normal process when working with multiplexed data.

No need to refuse paying ;-)

cheers,
Sven

Right, but you remove the MIDs once you have sorted by them. I think this was the original question "how can i sort by barcodes?"

[sklages]

Well, when I "remove MIDs", I do this usually by splitting a SFF file from either region of the 454 into individual SFF files (according to their MID); in this step, the MID is removed (== offset shifted in SFF file). That's a normal process when working with multiplexed data.

No need to refuse paying ;-)

cheers,
Sven