Well no, there are many situations where you will not be punished if you do something illegal (even if caught). If you do something illegal in the course of your job, it is quite likely that it is your employer who is taken to court, not you personally. If you don't pay the correct amount of tax, you won't be punished if it was found to have been an honest mistake. If caught speeding but not by much, and you aren't a dick to the police, you may be let off with a warning.

There is also a relevant pedantic distinction between "unlawful" and "illegal". The latter means in breach of the law. The former means otherwise than in accordance than the law. What's the difference? If the law says "don't drive over 30mph when the number 30 appears in the red circle" and you drive at 40, you broke the law and did something illegal. If the law says, "the local authority shall consider the conditions of the road when applying speed limits" and the local authority instead assigns speed limits at random, they didn't do something specifically forbidden, but they didn't do what they're supposed to. That's unlawful, and is treated differently.

Ultimately though the difference comes down to a presumption of good faith and the idea that if politicians or civil servants were prosecuted every time they got something wrong, we'd run out pretty quickly.