I just don't care to put substances in my body that are effectively toxic. If you think, being high at all times is the best way to live life, then more power to you.

Really, it's annoying that people think we're judgemental. I'm guessing, lots of folks judge themselves for it, but as long as everyone else does the drugs, they aren't confronted with that.
As soon as I dare to exist, not putting toxic substances into my body, they'll feel judged, because I'm adhering to their moral standard of not doing drugs, effectively reminding them that it would be doable.

If I come up with some bullshit reasoning, like I'm the designated driver, rather than the truth that I just don't want it, they'll feel more at ease. Although, I guess, that would also be the case, if they truly thought I was actively judging them.