Are sleep/pause commands in bash scripts ever necessary?

[Heisman]

Hey all,

Just a question from a novice computer guy; I am doing a lot of targeted capture analysis and basically have a script where commands are run one after the other (ie, align data, convert to .bam, sort, remove duplicates, etc). Without any type of sleep/pause commands between the steps, do I ever have to worry about the script moving on to the next command before the previous one is complete? I've been including them just to be careful but now I want to speed things up as much as possible (without risking screwing up the analysis). Are there scenarios where I do have to be careful about this? Thanks for any input!

[vyellapa]

Not unless there is a task completion dependency from another script. For example, you want to merge two bam's, then merge step should wait for the two input bams to be ready to be merged.

For serial processing, sleep makes little sense

[Heisman]

Ah, cool; if I'm having multiple scripts interact that makes sense. I wasn't sure if there was any chance for a script to move on to the next command while the previous command was still outputting data, for example. If not that's great.

[GenoMax]

Unless the sleep commands are waiting for very long (~ hour instead of few mins, do not go on the large number since that is in seconds) it may be best to leave them alone.

Sounds like someone wrote these scripts for you and may have put these commands in deliberately to allow the disk writes to complete etc.

[Heisman]

That's what I was particularly wondering about; nobody wrote this for me but I was originally advised to include them (maybe not great advice, though). I've analyzed a couple samples worth of data after taking them out and it ran fine with equivalent results, but I want to be sure it will always be fine, if that makes sense.

[GenoMax]

Out of curiosity what is the wait time that you have currently included? Is that truly impacting what you need to do negatively?

[Heisman]

Relatively small, like 3 seconds at multiple steps.There's a lot of samples to crank through so it adds up. However, to be honest, I just like understanding what's going on, haha. IE, does a bash script continue on to the next step when the previous step's PID disappears? If so, and I'm converting a sam file to a bam file with samtools, does that PID go away only after everything is written?

[GenoMax]

Sounds like you are going to remove the sleep/pause statements to get those 3 seconds back.

Go for it

On a serious note - the script would likely go to the next step as soon as the first one finishes. If you are not using fast/enterprise level storage the writes could in theory be lagging. I do not know for sure. Perhaps someone with more experience can comment.

[Heisman]

I'm going for it! Wish I could figure out a legit reason to include or not include them; that has to be documented somewhere.