Regular RTA Stalls with HiSeq 2000

[acunning]

We are currently experience fairly regular stalls with RTA 1.13.48 and HiSeq 2000s.

RTA stops copying, and in some cases never restarts. When we attempt to restart RTA, in a large number of cases RTA goes into a not respondind state, won't shut down cleanly and the entire system must be restarted.

Has anyone seen similar issues, and if so, how did you resolve or work around them?

We are of course working with Illumina Support, but 'the more eyes, the shallower the bugs'

Thank you

[genomeseeker]

We had similar issues at the beginning of 2012 with our HiSeq.

Are you doing full formatting of the hard drives after each run? We do this followed by full power cycle of the HiSeq and PC after each run and no longer stagger start the 2 flow cells. This has eliminated our problems.

[acunning]

Wow, complete reformats of the data drives. We haven't tried that, but it appears to be coming to it, though

We're still stagger starting, trying to optimize throughput, but that may be worth looking at as well.

We're power cycling after each run, but not as often as we might.

Were you analyzing your data in process, or did you wait till your runs were complete?

[genomeseeker]

Our data is analyzed in process.

The quick formatting just doesn't cut it; we tried. Apparently it gets more complicated when you stagger start flow cells with all the run pausing/stopping.

I hear you on trying to optimize throughput...a tough battle. Our experience is that it only adds a small chunk of labor to start both flow cells at the same time compared to starting them days apart.

At the end of the day, we're spending a lot less time troubleshooting with tech support and explaining issues to upper mgmt and researchers waiting for their data. Extra time to push through more library preps so we can start full runs whenever possible.

It's worth a go, imo. Good luck!

[GenoMax]

We are currently experience fairly regular stalls with RTA 1.13.48 and HiSeq 2000s.

RTA stops copying, and in some cases never restarts. When we attempt to restart RTA, in a large number of cases RTA goes into a not respondind state, won't shut down cleanly and the entire system must be restarted.

Has anyone seen similar issues, and if so, how did you resolve or work around them?

We are of course working with Illumina Support, but 'the more eyes, the shallower the bugs'

Thank you

You should check into the possibility of a failing hard drive on the instrument workstation since stalling RTA has been indication of that (for us).

We have several HiSeq's and rarely have problems with stalling RTA.

[pmiguel]

Our data is analyzed in process.

The quick formatting just doesn't cut it; we tried. Apparently it gets more complicated when you stagger start flow cells with all the run pausing/stopping.

I hear you on trying to optimize throughput...a tough battle. Our experience is that it only adds a small chunk of labor to start both flow cells at the same time compared to starting them days apart.

At the end of the day, we're spending a lot less time troubleshooting with tech support and explaining issues to upper mgmt and researchers waiting for their data. Extra time to push through more library preps so we can start full runs whenever possible.

It's worth a go, imo. Good luck!

Just catching up on a backlog of SeqAnswers post. Hmmm, we had this exact issue a few time. Advice from Tech support came down to never use pause, always stop a run. Then, for good measure, do a full system shutdown -- not just a computer reboot.
(1) Stop run. (Actually there are 3 options when you stop the run. I think you don't want option 1 unless the lab is burning down or something.)
(2) Exit HCS.
(3) Turn off (power down) the HiSeq! Yes, actually reach around the left side of the instrument and depress the rocker switch such that the instrument powers down.
(4) Power down the computer. I do this from the Vista start menu "shutdown" -- but there is a caveat. For reasons I don't understand, our shutdown button from this menu defaults to "standby". So to speed things up you want to not press that button but go to the dropdown menu just to the right of it and choose "shut down".
(5) Wait 5 minutes. (Probably not necessary, but why not?)
(6) Power up computer, log in to your normal user account. You may have mount/login to network drives, etc. Normal startup procedure for the computer.
(7) Wait until the "do not eject" (or is it "do not remove") drive appears. This can take a bit -- several minutes. If it doesn't happen, reboot the computer. I have never had this drive not appear the second time.
(8) Then launch HCS. You will probably have to re-seat the flowcells and possibly mop up some liquid around the gaskets.
(9) Resume the run.

You definitely want to do this during hours when tech support is available.

I am still hazy on what protocol would be if you have one flowcell running for several days and then want to add another flowcell and start a run. I mean, you need to do a maintenance wash on the 2nd flowcell fluidics, right? So you what, stop the first run, then start a maintenance wash on the second flowcell, then add your reagents and flowcell and start the new run?

That is pretty much what we did. But it seemed to invariable lead to that RTA slow down that is described here. Also, unless we went through the instrument power cycle, we would get some strange communication error between the instrument and the computer that would cause the focus to be off for the remaining swaths of the current cycle of the first flowcell.

By the way, I don't think we ever saw this issue with the old "death wish RAID" (RAID0) data drive configuration we had pre-HiSeq2500 upgrade. Correlation does not imply causation, though. I don't see a reason the new RAID5 configuration would not be superior in most ways.

We just installed a new version of HCS2.0.10 before our most recent run. This is supposed to ameliorate many of these sorts of issues.

--
Phillip