Isn't it the exact opposite?
I learned that you can never make a mistake if you aren't using git, or any other way for having access to old versions.
With git it is really easy to get back to an old version, or bisect commits to figure out what exact change was the mistake.
The only way I understand this joke is more about not wanting to be caught making a mistake, because that is pretty easy. In other methods figuring out who did the mistake might be impossible.