NUMA v SMP for Sequence Assembly

[plattsa]

Hi, we're increasing the size of the genomes we're assembling using GAII reads. Our old assembly server had an SMP architecture and worked fairly well but was limited in its RAM (96GB). With assemblers such as Velvet we're looking to move up to a 256GB RAM solution and had considered the Dell R910 (1TB Max RAM), Sunfire x4640 and HP DL785.

However to varying degrees these are all NUMA solutions with banks of RAM associated with a specific CPU/CPUs. I am a little concerned about the possibility of high latency where a single threaded process needs to access close to the entire contents of RAM. Essentially it seems a lot of the data would have to pass through several memory bridges before arriving at the active core with the consequent risk of a stalled CPU pipeline.

Cold anyone comment on whether this should be a concern or whether technologies such as quickpath can essentially get around this?

Thanks for your thoughts.

[Zigster]

Hmmm...even the largest (100M read) Velvet assemblies on our 256GB Dell Poweredge machine only take about three hours.

Does anyone make non-NUMA boxes?

[plattsa]

Thanks - that's reassuring to hear!

In the x86 world I think you're right, there's not a lot of SMP kit in the 256GB+ range. But in the power (IBM P series) and perhaps in the other high end risc processors I think it's still possible.

[plattsa]

I had the below from Dell on this topic:

There are two situations where remote memory accesses are unavoidable. First, processes that require more memory than what is available to a single processor. Second, processes that spawn more threads than available cores within the local socket. CPU and memory affinity are not appropriate in either case. 11G servers have a BIOS feature called Node Interleaving that stripes data across both memory controllers when enabled. Interleaved accesses are slower than local-only accesses because every other operation traverses the QPI link. However, this feature prevents the worst case scenario where a process is forced to access remote memory at every operation.