You're not wrong, but social anxiety and social guilt are mother fuckers even when you know you have understanding friends

True friends will still want you to come but understand if you bail out.

True friendship is a two-way give and take. For some things, friendship means giving up some level of autonomy and self interest to provide something that your friend wants. In some contexts, showing up is important to the friend and a few repeated snubs/cancellations ends up communicating to the friend that they're not important to you. At that point they can start revisiting whether this is a "true friendship" or not and protect themselves by pushing away.

And it's not just not coming out. It's also the implied precursor here, that two people have made plans together. There's some level of reliance on the other, and bailing at the last minute is often seen as much ruder than just not agreeing to hang out in the first place.

Or, alternatively, the other person starts to understand that you have a preference against hanging out, like it's a chore or a favor. They're your friend, and they want to do right by you, so they just stop inviting you out and asking that favor of you, and then you drift apart and wonder why.

Friendship is about understanding other people, and empathizing even when their personalities and thought process are different. Friendships are hard enough to maintain past 30, and keeping them requires some level of conscious effort, especially for introverts.