Obtaining .CIF or raw images from HiSeq/NextSeq?

[dorix]

Is there even space on the disk?

What do you mean?

Just curious, why do you want to store CIFs on a NextSeq system?

As Hyeshik Chang, we want to analyze homopolymeric sequences.

[sklages]

Just curious, why do you want to store CIFs on a NextSeq system?

[Codemonkey]

I've also got the answer from tech support which warned me that .cif files is not supported due to memory limitations. I wonder if you saved .cif files on machine or on external drive and if any problem with data transfer speed occured?

Is there even space on the disk?

[dorix]

I've also got the answer from tech support which warned me that .cif files is not supported due to memory limitations. I wonder if you saved .cif files on machine or on external drive and if any problem with data transfer speed occured?

[dorix]

Hi,
We're planning to save .cif files from NextSeq sequencing. We have been warned that new chemistry is not compatible with it. Your comments seems promising, however I would like to be sure that our instrument configuration is the same. We're using software version 2.0.2 (RTA 2.4.11).

Is there anything else we should take into consideration?

[sklages]

Is Bustard still being used on current Illumina systems? I had thought that they are using something else.

No. OLB development has been dropped a few years ago. After 1.9.4 Illumina had plans to develop RTAOLB, which was an "external" RTA (windows) binary. Well, I never saw a working binary :-) That was back in 2012/13. Then Illumina decided to further optimise on-device RTA (e.g. for unbalanced/low complex. libs) that there is no need anymore to use some external application for basecalling (at least from their point of view). They did a quite good job on that (at least IMHO).

[bilyl]

Probably yes, CIFs (among other files) are the basis for illumina's basecaller 'bustard'.

Is Bustard still being used on current Illumina systems? I had thought that they are using something else.

[sklages]

Interesting... I wonder if you could implement your own basecaller with the CIFs, especially for specialized assays? Or are these files already too far along in the processing?

Probably yes, CIFs (among other files) are the basis for illumina's basecaller 'bustard'.

[Hyeshik Chang]

Interesting... I wonder if you could implement your own basecaller with the CIFs, especially for specialized assays? Or are these files already too far along in the processing?

The purpose of my library was a lot simpler than the general use cases of specialized base calling. Mine was just for rough length measurement of homopolymers (poly(A) tails in mRNA). It was not very difficult to modify my previous codes to take up two channel signals than four channels.

[bilyl]

Hi ScottC,

Thank you for the comment. I could get CIF files from NextSeq by editing the line as you suggested. It looks that they did not change the file format of CIF files except the number of channels in a cycle. Here's my final change set to the configuration file:

--- NextSeq.Configuration.xml.orig	2015-09-09 09:40:53.000000000 +0900
+++ NextSeq.Configuration.xml	2015-09-09 09:41:35.000000000 +0900
@@ -24,14 +24,14 @@
     <DataManager>DataManagerAllInMem</DataManager>
     <InterOpManager>InterOpManagerTwoChannel</InterOpManager>
     <AggregateOutput>true</AggregateOutput>
-    <ThumbnailTileMatch>...0[16]|...12</ThumbnailTileMatch>
+    <ThumbnailTileMatch>.....</ThumbnailTileMatch>
     <WorkProviderTypes>
       <WorkProviderOptions>ServiceMultiImageBased</WorkProviderOptions> 
     </WorkProviderTypes>
-    <CifQValTileMatch>1[13].03|12.09|2[13].09|22.03</CifQValTileMatch>
+    <CifQValTileMatch>.....</CifQValTileMatch>
   </Processing>
   <ColorMatrix>
-    <SaveCifs>false</SaveCifs>
+    <SaveCifs>true</SaveCifs>
   </ColorMatrix>
   <Basecalling>
     <Basecaller>BasecallerTwoChannel</Basecaller>

Interesting... I wonder if you could implement your own basecaller with the CIFs, especially for specialized assays? Or are these files already too far along in the processing?

[Hyeshik Chang]

Hi ScottC,

Thank you for the comment. I could get CIF files from NextSeq by editing the line as you suggested. It looks that they did not change the file format of CIF files except the number of channels in a cycle. Here's my final change set to the configuration file:

--- NextSeq.Configuration.xml.orig	2015-09-09 09:40:53.000000000 +0900
+++ NextSeq.Configuration.xml	2015-09-09 09:41:35.000000000 +0900
@@ -24,14 +24,14 @@
     <DataManager>DataManagerAllInMem</DataManager>
     <InterOpManager>InterOpManagerTwoChannel</InterOpManager>
     <AggregateOutput>true</AggregateOutput>
-    <ThumbnailTileMatch>...0[16]|...12</ThumbnailTileMatch>
+    <ThumbnailTileMatch>.....</ThumbnailTileMatch>
     <WorkProviderTypes>
       <WorkProviderOptions>ServiceMultiImageBased</WorkProviderOptions> 
     </WorkProviderTypes>
-    <CifQValTileMatch>1[13].03|12.09|2[13].09|22.03</CifQValTileMatch>
+    <CifQValTileMatch>.....</CifQValTileMatch>
   </Processing>
   <ColorMatrix>
-    <SaveCifs>false</SaveCifs>
+    <SaveCifs>true</SaveCifs>
   </ColorMatrix>
   <Basecalling>
     <Basecaller>BasecallerTwoChannel</Basecaller>

[ScottC]

Hi. This is fairly obvious, but did you try changing the configuration files like you would do with the MiSeq? They're not exactly the same, of course, but there's an field <SaveCifs> in NextSeq.Configuration.xml that looks promising. I've never tried with the NextSeq, so I can't say whether it works or not.

[Hyeshik Chang]

Hi,

Illumina recently changed the data flow inside their HiSeq and NextSeq machines to reduce I/O burden during sequencing dramatically. It now skips saving .TIFF image files and .CIF intensity files by passing them through internal spaces in shared memory. Illumina technical support confirmed that there will be no way to get .CIF file or .TIFF images in HiSeq/NextSeq, soon.

As HiSeq Rapid V1, which is the last kit version that allows .CIF generation, is to be discontinued soon, there remains no commercially available kit in large-scale Illumina sequencing kits. MiSeq and GA IIx are still able to produce those files, but they don't meet the modern needs of throughput in many types of experiments.

If you're using .CIF files or images in your study, how do you deal with this changes? Is there anybody succeed accessing the shared memory or the "web HTTP interface" mentioned in the user guide?