I don't disagree with your conclusion, but I don't think they replace anything with sugar. Milk is just high in sugars, because it's supposed to give energy and nutrients to a growing calf. If you remove the fats then the sugar content is higher as a percentage.

Milk isn't all that healthy in general though. It's liquid for a calf to grow. It's much better to drink something made for human consumption and dietary needs.