[SDPA_Pet]

You might find this benchmark to be helpful:

http://www.nature.com/articles/srep19233

My understanding is that you are better off using newer k-mer based approaches as opposed to BLAST. I've had reasonable success with kraken, although the memory requirements are somewhat onerous. You also have to be extremely careful about removing contaminant (aka human) sequences as these tend to get misclassified.

Another option is kallisto (https://github.com/pachterlab/metakallisto) but I have yet to be able to even build a database due to memory constraints.

I will try. Can you gimme the Kraken link? Thanks.

[fanli]

You might find this benchmark to be helpful:

http://www.nature.com/articles/srep19233

My understanding is that you are better off using newer k-mer based approaches as opposed to BLAST. I've had reasonable success with kraken, although the memory requirements are somewhat onerous. You also have to be extremely careful about removing contaminant (aka human) sequences as these tend to get misclassified.

Another option is kallisto (https://github.com/pachterlab/metakallisto) but I have yet to be able to even build a database due to memory constraints.

[SDPA_Pet]

Would something like kraken or CLARK not be helpful? Are you trying to assemble and annotate de novo genomes? Or trying to figure out the microbial composition and functional content? I guess my point is you would discard ~40% of your data in the joining process, which may not be necessary depending on your task of interest.

I am not interested in a specific genome in the soil community. Basically, I am just interested in the microbial composition and functional content. There is another way that people usually do. They don't assemble or merge pairs and they just blast using raw data. However, I think blast using ~150bp reads is worse. Some publication shows the intermediate length (merged pair) is better than assembled longer reads or unasembled short reads for the question that I am asking for.

[fanli]

Would something like kraken or CLARK not be helpful? Are you trying to assemble and annotate de novo genomes? Or trying to figure out the microbial composition and functional content? I guess my point is you would discard ~40% of your data in the joining process, which may not be necessary depending on your task of interest.

[SDPA_Pet]

Out of curiosity, why are you joining the read pairs? A lot of the metagenomics software out there now supports paired end reads as input. The metaSPAdes assembler @GenoMax mentioned requires paired end data IIRC.

Hey, I did try metaSPAdes, less than 1% of total reads assembled. A lot of people tried alternative methods, joined paired-ends and get long reads, but don't assembled reads. Then, use the long merged reads to do BLAST or other annotations.

[fanli]

Out of curiosity, why are you joining the read pairs? A lot of the metagenomics software out there now supports paired end reads as input. The metaSPAdes assembler @GenoMax mentioned requires paired end data IIRC.

[SDPA_Pet]

Clustering means bridge amplification for pre ExAmp (non-patterned flowcells) - in situ PCR on the flow cell surface oligos lawn. Has similar rukes/laws to a regular PCR, only the product stays in situ, forming a forest from DNA strands.

For ExAmp Chemistry (patterned flowcells) - Clustering means cluster formation using Isothermal Amplification.
(In theory only on the occupied nanowell, in practice, especially at low loading concentrations a few neighbours may join in too...).

Have a read about ExAmp & Hiseq4000:
http://core-genomics.blogspot.co.uk/...d-to-know.html

Thank you. I did my sequencing on old platform HiSeq 2500.

[Markiyan]

Clustering = Bridge Amplification (for pre ex amp).

Clustering means bridge amplification for pre ExAmp (non-patterned flowcells) - in situ PCR on the flow cell surface oligos lawn. Has similar rukes/laws to a regular PCR, only the product stays in situ, forming a forest from DNA strands.

For ExAmp Chemistry (patterned flowcells) - Clustering means cluster formation using Isothermal Amplification.
(In theory only on the occupied nanowell, in practice, especially at low loading concentrations a few neighbours may join in too...).

Have a read about ExAmp & Hiseq4000:

http://core-genomics.blogspot.co.uk/2016/01/almost-everything-you-wanted-to-know.html

[SDPA_Pet]

Hi, thanks. Can you explain more about the clustering stage. I don't know much details about HiSeq? Clustering stage -- do you mean it is a step of library building or Bridge amplification?

Also do not forget the clustering efficiency dependency on insert size.

Basically despite your library having 600bp fragments, they would clusters less efficiently (~10x?) than 200-300 bp fragments present in the sample. As a result one gets a peak on FLASH histogram in the area that is ~1/3x on the rising side of the bell curve produces by bioanalyzer. (You get enrichment of the smaller fragments during the clustering stage.)

PS: with latest iteration of the Illumina instruments (Hiseq4000/NovaSeq) they seem to continue to support libraries with up to 350 bp insert size - Shorter insets give you smaller and brighter (clusters/wells) + less likely to be long enough to jump to neighbouring wells - so can be sequenced on higher densities. As the result we get max 2x150 bp max. support from (Hiseq4000/NovaSeq). If you need 2x250 stick with HiSeq2500 or MiSeq.

[yzzhang]

for our soil samples, the assembled reads normally account for ~50% of the original reads. BTW, our data is >10 Gb per sample.

[Markiyan]

Do not forget the clustering efficiency change ws insert size.

Also do not forget the clustering efficiency dependency on insert size.

Basically despite your library having 600bp fragments, they would clusters less efficiently (~10x?) than 200-300 bp fragments present in the sample. As a result one gets a peak on FLASH histogram in the area that is ~1/3x on the rising side of the bell curve produces by bioanalyzer. (You get enrichment of the smaller fragments during the clustering stage.)

PS: with latest iteration of the Illumina instruments (Hiseq4000/NovaSeq) they seem to continue to support libraries with up to 350 bp insert size - Shorter insets give you smaller and brighter (clusters/wells) + less likely to be long enough to jump to neighbouring wells - so can be sequenced on higher densities. As the result we get max 2x150 bp max. support from (Hiseq4000/NovaSeq). If you need 2x250 stick with HiSeq2500 or MiSeq.

[GenoMax]

They are soil samples. I did shotgun metagenomics and I am insterested in microbial communities. I will not assemble it, because normally less than 1% of reads can be assemble. My plan is joined whatever reads can be joined and get longer reads. Then, to annotate it using the long reads. Those samples are from environments and you don't really have prior knowledge about what is in it. The workflow is different from model organism.

SPAdes has a meta option for doing assemblies with metagenomes. I am sure there are other options for this type of assemblies. You would need access to a server with ample RAM but it should be possible to assemble the data you have to some extent. Unless you have already tried this and are reporting 1% assembly based on that.

PS, I don't understand in your previous post about "If you have a reference available then making the libraries so the two ends do not overlap makes sense since you can sample a larger region". Just curious.

As long as you can map the two ends of a fragment on a reference genome at the expected distance you could consider that region as "sampled". Since you will have reads that will randomly cover the genome, you should get reads mapping/spanning across entire genome.

[atcghelix]

Are you sure they subtracted the adapter length from the fragment sizes to get the insert sizes (meaning, are you sure they're reporting insert size from the bioanalyzer?)? If the fragments themselves are an average of 600bp with a fairly wide distribution, it wouldn't be surprising if 60% of your reads merged with 150bp PE.

That said, we've (very rarely) had libraries that gave drastically different results between bioanalyzer, fragment analyzer, and tapestation, with the empirical insert size distributions determined after sequencing not agreeing with any of them.

[SDPA_Pet]

What kind of samples are these and what will you be doing with them (assembly?) downstream?

They are soil samples. I did shotgun metagenomics and I am insterested in microbial communities. I will not assemble it, because normally less than 1% of reads can be assemble. My plan is joined whatever reads can be joined and get longer reads. Then, to annotate it using the long reads. Those samples are from environments and you don't really have prior knowledge about what is in it. The workflow is different from model organism.

PS, I don't understand in your previous post about "If you have a reference available then making the libraries so the two ends do not overlap makes sense since you can sample a larger region". Just curious. I don't do model organisms and so normally there is no reference database. However, if they chose 2X150bp and have a reference database, but use 600 bp inserts. You can only sequence 150 bp from either end, but I still can't get information about 300 bp in the middle of the fragment. Why would they build a larger fragment library?

[GenoMax]

What kind of samples are these and what will you be doing with them (assembly?) downstream?