Also, afaik, alot of the damage done by things like meth and heroin are side effects. Like meth gives you psychosis and brain damage from staying up for days and not eating any food. The long term effects of meth, like a rotting mouth and permanent chemical imbalances can take years to manifest. If you want an example see people who take Adderall for years. Heroin similarly stops you from getting proper nutrition because all you want is heroin and it can give you brain damage from a lack of oxygen, which is a side effect of getting really close to overdosing.

As you've said, most people (at the very worst a whopping 70%) try these drugs and continue with their life without getting addicted. It's very dangerous to stigmatize them cause all it does is make people afraid of getting treatment.