Bash is Dangerous: How to not lose 77TB of research data

Recently, Kyoto University lost a large amount of research data to a poorly timed script update. In this video I conjecture as to what happened, and provide some advice on how to prevent a similar thing from happening to you.
The root cause of the issue seems to be the manner in which POSIX shells (and others) interpret an undefined variable, combined with they way in which shells process the script file itself.
On encountering an undefined variable, rather than throwing an error, many shells assume the variable to be empty-valued. This is very convenient, but can lead to subtle bugs as well. Most shells have a way to change this behavior, so that an error is thrown instead, but these are not commonly used as they are inconvienent.
That issue seems to have been combined with the fact that some interpreters (including bash), only read small sections of the script from disk at a time. This means that, if the script on disk is changed while the script is running, variables can spontaniously become undefined within the running script. If that variable is used as part of the path in an rm command, for example, this can result in the unintended deletion of files.
Original Article
www.techspot.c...