Should one instance of mpileup use a full thread? Computer running slowly.

[Heisman]

Hey all,

I think my computer is running slowly and I have no idea why. I'm also sadly not wonderfully computer savvy so I'm not sure how to troubleshoot this obviously. Are there some basic things to look at (if so please include commands to run if you know them off the top of your head)? In the past I've run mpileup on it and it usually uses a full thread and is fine and right now it's hovering at around 15-20% CPU by looking at it with htop. Thanks for any help.

[Heisman]

Figured it out, good learning experience: I was using a bash script and piping the output of mpileup into multiple commands. I think it was splitting up the thread amongst all of the commands. Maybe that's not what was going on but making mpileup separate and then running the others has made mpileup take up a whole thread like I wanted it to.

[Heisman]

Actually it's still doing it; driving me crazy. I'm running four instances of mpileup and each is using ~25% of a CPU, this computer has 64 GB RAM and 24 cores. Free -m shows ~50 GB RAM is being used. I have no idea how to make each instance use 100% of a CPU, if possible.

[jimmybee]

Have you tried using each command outside the bash script? or perhaps, use something like parallel (https://en.wikipedia.org/wiki/GNU_parallel)?

I havent used it as such but I know a few people are implementing it in everyday scripts to pretty good effect

[Heisman]

I tried running them stand alone and the same thing happened. One thing holding me back is I don't have any background or general knowledge in how memory works in general (of course I don't know it's a memory problem). I'm rather ignorant in this area. They will spike up to ~100% every now and then but then go back down to 15-40%. Right now if I do free -m I get:

             total       used       free     shared    buffers     cached
Mem:         64399      54764       9635          0         86      51385
-/+ buffers/cache:       3292      61107
Swap:       123977          0     123977

No idea what's going on, haha.

[GenoMax]

Your cores are probably waiting on the storage subsystem to deliver data. Unless you are using server grade storage/SSD's in RAID this is probably what is holding your system back.

Check on "these performance metrics"

[lh3]

In addition to the storage issue mentioned by GenoMax, the commands following mpileup may be the bottleneck if you pipe. Mpileup may be waiting for those commands.