MiSeq MCS memory issues

[mcnelson.phd]

For all other MiSeq owners out there, I'm wondering if you've ever noticed any issues with your system operating sluggishly while sitting on standby for a while? Specifically, that MCS seems slow to respond to touches/clicks?

Our system has been doing this for a while, but this past Friday when I checked on it there was a Windows error message stating that MCS was hogging up all of the available system RAM and that I should shut it down. I checked in the task manager and MCS had accumulated ~11.5GB of RAM, even though the system had been on standby for a week and shouldn't have any need for that much memory.

I ended up power cycling the system because MCS wouldn't shut down, and threw it back in standby when it powered back up, but each day as I've been checking on the system MCS is continually accumulating RAM for no apparent reason. From my checks, MCS is acquiring an additional ~1GB of RAM per day, which means that in another few days the system will run out of free RAM again.

I've contacted my FAS about this, and he said he'd look into it, but I'm wondering if anyone else has experienced this issue? My normal complaint would be to blame Windows, but more realistically if this is affecting other users I would suspect that there's some sort of bug in MCS.

Mike

[GenoMax]

We have not noticed any problems that I have heard about. Can you post the exact version of MCS you are on? This sounds like some kind of a memory leak in MCS. I wonder if it is version specific.

[mcnelson.phd]

Yeah, I should have mentioned that in my first post. We're running MCS version 2.2.0.

I just looked on the MiSeq support site, and it appears that Illumina put out an updated version (2.2.2) that's supposed to resolve issues with the chiller temperature, so I may see what my FAS thinks about me installing that version even though we don't have issues with the chiller temperature.

[GenoMax]

I heard about the new patch yesterday. We may apply it to our MiSeq's even though we have not had any issues with chiller temperature (recommendation from FAS was to apply the patch if the MiSeq chiller temp consistently stays above 8 degree C).

BTW: May be worth opening a ticket with tech support. Just in case.

[DRYTCYV]

Hello,
When Miseq was installed in our lab (3 weeks ago), they advised to restart windows every time you need to start a run, just to release memory.

[mcnelson.phd]

Hello,
When Miseq was installed in our lab (3 weeks ago), they advised to restart windows every time you need to start a run, just to release memory.

Yeah, our FAS told me that a few months ago that it couldn't hurt to power cycle the system every few weeks just to keep things "fresh", but let's be honest, that's a pretty poor solution to what's either bad programming or a hardware issue.

Given that no one else has responded that they've experienced this, I'm beginning to wonder if a hardware issue might be to blame...

[AStretton]

We have been having issues with MCS crashing and causing the MiSeq to crash, Tech support have told us with the new version of MCS we have to shut down the MiSeq completely at least once a week preferably before a new run, turning it off at the switch on the back and leave it off for at least 15 minutes. With the new software, they told me not to leave the MiSeq on if we were not planning on running it. I was also told that this was contrary to previous advice to leave the MiSeq running.

[microgirl123]

Ours slows down after about a week without running. I always power it on and off again before performing a run to speed it up again.

And our chiller is always ~10 degrees. I'll have to look into what version of MCS we're running after the current run is done!

[DRYTCYV]

Our Miseq is off, since we don´t have samples to run now. "He take a bath every monday!", a maintenance wash. Till now, anything to report about be slow.

[mcnelson.phd]

Replying to my own thread with an update.

I've talked with a few Illumina techs, and they've said that the memory issue comes down to how MCS is collecting and recording data logs while the system is sitting idle. Basically, MCS is constantly collecting various pieces of system data and constantly writing them to a log-file, and somewhere in the process absolutely no garbage collection is occurring. That means that after a while, MCS will have chewed up all of the available system memory while trying to collect and record log data.

The guy I was speaking with yesterday said he's talked with a bunch of people in the software group, so they know that it's an issue, but it's basically a really low priority to them so who knows when/if they'll get around to fixing the problem and pushing out an update.

So, for now, the only real solution is to power cycle the entire system before doing a run and if your system is going to sit idle for more than a week then you should should power cycle it every fortnight or so.

[microgirl123]

Thanks for that info! I'll start power-cycling it before it's weekly bath!

[behavin]

Ours has had this issue for a while and it has persisted through several updates of MCS. At first a power cycle would release the memory, and everything would run fine. However, after several months of this the MiSeq Reporter software started to lag very badly after every run, regardless of memory status/reboots - it would take up to a day to even recognize there was new data to process. We replaced the HD with an imaged one from customer support which resolved the Reporter issues, but the memory leak is still present.

I had assumed it was a software issue but never got a straight answer from tech support - thanks for confirming that!

[mcnelson.phd]

Ours has had this issue for a while and it has persisted through several updates of MCS. At first a power cycle would release the memory, and everything would run fine. However, after several months of this the MiSeq Reporter software started to lag very badly after every run, regardless of memory status/reboots - it would take up to a day to even recognize there was new data to process. We replaced the HD with an imaged one from customer support which resolved the Reporter issues, but the memory leak is still present.

I had assumed it was a software issue but never got a straight answer from tech support - thanks for confirming that!

Illumina actually replaced our single board computer and HDD, and the issue still persisted, so that's when one of the techs started talking with the software guys and they figured out that the logging was the likely cause. The guy who did the single board computer replacement actually told me later that the main reason they did it was to increase the amount of system RAM, and not because they really expected the hardware to be at fault.

Guess we're getting good value out of our service contract at least.

[MoritzF]

Has the issue been solved by someone?

We just had a failed run that might or might not be related to the issue at hand.

The MiSeq showed only a black screen and in BaseSpace nothing was happening anymore.

So we attached another spare monitor to the machine and got something I personally have rarely seen since Windows XP came out: a classic windows blue screen. See attachment.
It it some error message about some memory issues so I figured it might be related to what is discussed here.

We had a failed run a while ago that resulted in a similar error message, after which illumina tech support informed us that the problem was we loaded too much DNA into the MiSeq (overclustering).
The recent run was perfectly clustered and we got the run fail again, so now I figure it might be a totally different problem i.e. something with RAM managemant.

So, anyone had some updates on this issue?

[mcnelson.phd]

For starters, there's a new update to MCS that fixes that memory leak issues. I would highly recommend that you install it ASAP.

As for Windows killing itself and thus your run, well, that's what happens with Windows unfortunately. Illumina should replace your reagents, and I'd suggest having an FAS come and look at the system to make sure there are no other issues that might warrant a single board computer replacement.

It sucks that it took so long for Illumina to get this memory leak fixed (or at least send out a patch), but since upgrading MCS we haven't had issues with it gobbling up all the RAM and causing the system to become unstable. It's also still recommended to periodically reboot the system (Illumina actually has a tech bulletin saying you should it that before every run).