AC is more than a dex save... And in fact may not involve dex at all if the target is wearing heavy armor.

It's a very consistent system. Direct, targeted attacks with a physical manifestation (that is, some kind of targeted projectile or weapon swing) roll against the target's general-purpose defense stat (AC). Indirect attacks (e.g, fireball) or things that are otherwise simply "happening" to the target has the target rolling a save of some sort to resist the effects somehow (dodge out of the way, resist mental influence, hold themselves upright, etc.). There's nothing arbitrary about it, and a unified defense system would no doubt involve a lot of special-casing/ad-hoc calculations to be at all worthwhile, to the point at which it would be far more cumbersome and confusing.

Let's say we only use AC as a defense and have no saves. How does a spell like Hold Person work? Does the target use their wisdom modifer instead of their dex modifer for calculating AC? Does the armor they're wearing affect their defense against it? What about proficiencies or other bonuses? Since the target is no longer making a roll, how do buffs to protect against the effect (e.g, bardic inspiration) work? I don't think there's a way to do it that is not more convoluted.

Skill checks don't really overlap at all, other than the fact that they use the player's attributes, I guess.