A woman is out shopping, and suddenly spots her husband. As she's about to say hello to him, she notices the man is filthy: his clothes have stains from spilt food and drinks, his face and hands are dark with mud and grime.
"What happened to you?!" she asks, skipping the hello.
"Oh, it's nothing, don't worry about it..."
"What do you mean don't worry about it? You're dirty like a pig! At least go home and shower!!"
"No, I can't... There's something I have to do. Sorry, honey, I'll see you later tonight."
"Well at least tell me how you got so muddy!"
"I really can't tell you. It's nothing, I promise."
The woman starts getting angry. "Listen to me. Either you tell me what's going on, or go home with me right now to wash yourself!! If not, I'm packing your things and kicking you out!"
The husband thinks about it for a while, then makes a deep sigh and says:
"Alright... I'll come clean."
There's a line somewhere between "computers that teach you not to do that" and computers that prevent dire consequences when you make a human mistake. The "just don't do that" policy is never enough. If there are no safeguards, at one point the mistake will be made.
Even by highly trained astronauts: https://wehackthemoon.com/people/margaret-hamilton-her-daughters-simulation