My understanding is folks tend to gravitate towards that because it's indeed very close to might've and whatnot phonetically. My anecdotal experience as a non-native speaker is we tend to be less affected since we usually tackle speaking and listening more seriously after we've already familiarized ourselves enough with writing/reading, grammar and vocab.

It blows my mind as well. My native language is Spanish, but for me it's way easier to follow language rules properly in English. May have something to do with the fact that my native language is my regular language for expression, so I don't pay much mind to how I use it, but English being a second language, I actually try to make sure I'm understood. Anyway, that's what I think could potentially be the reason.