[GW_OK]

Finally, I graphed duplicate read coordinates relative to the initial read coordinates (x1-x2/y1-y2 from the file I made above). I've attached that file below since it's not tremendously large. Most clump fairly close together, as others have shown, but there does seem to be a Y-bias to my eyes. Perhaps this means that there is some merit to the direction of flow duplicate theory.

[GW_OK]

@GW_OK: You may have missed this post from QC Fail. They did something similar.

I don't think you need to include "spantiles=t" for NovaSeq (or 4000 data). We have been keeping that off. That is a specific issue with NextSeq and the large clusters it has.

There is some oddity about the "dupedist=" setting as well. We have not been able to nail that one down for NovaSeq.

So, yeah. I didn't miss that qcfail blog post. I've read through it several times. I wanted to recapitulate their analysis on a data set that:
(A) had not undergone PCR amplification and
(B) was across an entire tile, not just a small region of a tile
(C) was performed by Illumina and/or someone with a vested interest in having their data set showing the theoretical "best" of what the machine can do. It's all well and good to throw a library on two machines but I don't know what that library looked like prior to loading.

I did want to use spantiles to demonstrate the 'mode' of duplication. Are the duplicates moving from well to well, or across the whole tile, or from tile to tile and surface to surface? Based off what I've seen here they're not just moving across interconnected wells.

I picked dupedist 2500 based solely on what people have used for the 4000, as given in the clumpify thread in the bioinformatics subforum.

[GenoMax]

Can you confirm that the three posts from today use data from NovaSeq (is that data PCR free I don't recollect)? spantiles=t actually spans across all tiles. This was essential to capture the edge-duplicate effect that appears to be specific for NextSeq flowcells.

Since the NovaSeq flowcells should have more nanowells (3-4x?), so using 2500 distance is probably not optimal (though as I said before we have not been able to pin a distance down based on the data available).

I think Illumina is sampling this library do deeply that we are starting to see duplicates across the FC/tiles just because there are only so many sequenceable fragments in the library. I have tried to test this by pooling two lanes of NovaSeq data together to see if the number of clumps does not go up appreciably. Unfortunately I have not been able to get clumpify to work with this pooled data (and @Brian has not had a chance to look at why that is happening).

[GW_OK]

It's from the BaseSpace project

NovaSeq: WGS TruSeq PCR-Free 450 (6plex)

So I reckon it must be PCR-free.

NA12878-rep1 only looking at data from lane 1.

[GW_OK]

Regarding sampling depth, I am dubious.

The Truseq PCR-free protocol (which I have to assume they're using) has you start with 1ug for 350bp inserts and 2ug for 550 bp inserts. Since they say they're using 450bp inserts I'll split the difference and say they started with 1.5ug of DNA. The entire human genome weighs 3.6pg (as per IDT) so that 1.5 ug is ~417k human genome equivalents. If they shear it on a Covaris (which is fairly random) you would have to rely on two copies of the genome shearing at the exact same base pair on both ends.

Then, from the fairly random fragment assortment of ~417k genomes you then take ~1.8E10 molecules (assuming they loaded at 200pM) and from that sample 599M molecules.

Someone with more statistical chops than me on a Monday morning can do the actual math but I have a feeling we're not close to oversampling these libraries. I could be wrong, though, so don't hold me to it.

[GenoMax]

I was thinking about "cluster-able/sequence-able" fragments present in the library. In any case this is just a hypothesis and your calculations may be spot-on.

We are approaching data never seen before (e.g. there are 1.6 B reads in largest example data lane and this could potentially go up 4x with S4 cells).

[GW_OK]

Maybe we need to start talking in terms of "library coverage" instead of "genome coverage"...

Also, don't confuse read numbers with cluster numbers. There'll be 2 reads to every cluster. And on this NovaSeq data you'll have to split their cluster counts across both lanes.

[SNPsaurus]

Thanks for the plots, GW_OK!

[GW_OK]

A few more interesting points.

The tile map for the S2 flowcell:
-The first digit is the lane number: 1 or 2.
-The second digit represents the surface: 1 for top or 2 for bottom.
-The third digit represents the swath number:1, 2, 3, or 4.
-The last 2 digits represent the tile number, 01 through 88. Tile numbering starts with 01 at the outlet end of the flow cell through 88 at the inlet end.

The stuff on BaseSpace is from a pre-release flowcell but I think we can assume the tiling map holds true except their tiles range from 05 to 90.

Mapping the number of intra-tile, inter-tile, and inter-surface-tile duplicates shows that libraries will jump tiles in the direction of flow but less so horizontally or diagonally. There's also a large amount of inter-surface jumping from one surface to the direct opposite. My previous analysis was done on tile 1105, which you can see in the table below in the upper left corner. I also mapped the duplicates in a more centrally located tile (1240) where the duplications are more pronounced surrounding the tile in question.

This seems to reinforce the previous observations that most duplicates stay (relatively) close to their origin with some bias in the Y direction. However there also appears to be a Z-axis bias as well.

[Brian Bushnell]

What a giant hairball! You can clearly see that there are libraries duplicating in both the horizontal and vertical direction. Another striking thing is just how long some of the lines are. Since I don't think there's an a priori way of telling which well came first I refrained from assuming directionality. That being said, I'm looking at triplicate (or higher) wells to see if there is a "shotgun" pattern that could indicate a directional "spray".

That's downright strange...

[misterc]

Does anyone have even a lane's worth of these new .cbcl files from a NovaSeq? I'd like to test our bioinformatics pipeline with the new bcl2fastq converter v.2.19 that supports NovaSeq.

[GenoMax]

Does anyone have even a lane's worth of these new .cbcl files from a NovaSeq? I'd like to test our bioinformatics pipeline with the new bcl2fastq converter v.2.19 that supports NovaSeq.

Illumina does not appear to have made the input files for NovaSeq data available on BaseSpace. Just the outputs.

[austinso]

On another note:

150 uL of a 1 nM library (~90 billion molecules) minimum for loading is a lot of library when you consider you can get by with 1.4 billion for the NextSeq and 7 billion for the HiSeq.

FWIW...

[misterc]

Is 150ul of a 1nM library what Illumina recommends for a single S2 flow cell?!?

[pmiguel]

On another note:

150 uL of a 1 nM library (~90 billion molecules) minimum for loading is a lot of library when you consider you can get by with 1.4 billion for the NextSeq and 7 billion for the HiSeq.

FWIW...

Okay I take your point, but an S2 should produce 3 billion clusters per flowcell, whereas a HiSeq 2500 produces about 1.6 billion with v3 chemistry. So the NovaSeq is about 4x less efficient than the HiSeq 2500 in this regard.

A NextSeq produces about 0.4 billion clusters per flowcell. So, the relative efficiencies would be:

(I'm using PF clusters per flowcell / ~number of input amplicon molecules)
HiSeq2500v3 = 1.6/7 = 23%
NextSeq = 0.4/1.4 = 29%
NovaSeqS2 = 3/90 = 3.3%

So, it absolutely looks like a much lower efficiency of clustering on the NovaSeq. (Anyone know if this is also the case for the HiSeq3000/4000?)

That said, how much difference will this make for most runs? If you use the standard HiSeq2500 method, you start with 10ul of a 2nM library pool for denaturation. Since it gets diluted down to 20 pM (at least) you end up with 1 ml for each denaturation you do. One denaturation could be used to cluster all 8 lanes of the flowcell. But how often does that happen?

For us, I can't think of a single case where we have clustered more than 2-3 of lanes per denatured sample pool. Usually it is 8 sample pools for 8 lanes.

There are cases where the amount of library produced is limiting. And the NovaSeq would not be a good choice where this is your critical parameter.

So in most cases I would say it is being forced from 8 lanes to 1 lane along with losing the flexibility to run a much smaller flowcell (with rapid chemistry 2 lane flow cells) that are the major limitation of the NovaSeq.

Illumina expects you to just buy a NextSeq to deal with the 2nd issue above. That would okay (for some definitions of "okay") if they hadn't just decided all the NextSeqs should now have the ability to scan their microarrays. But the option is there.

Then there are the data issues considered in this thread. But I'm pretty sure that is something Illumina can fix (as they had for a period of time with the NextSeq, just after they introduced the v2 version of its chemistry/software) if they focus their attention on it.

--
Phillip