The vast majority of people pushing for this word not to be used are people without intellectual disability. I think an argument could be made that these people are being ableist by deciding what is offensive to people with ID for them.

I have never heard the word used to refer to someone with ID in my life, and I would imagine if someone ever did, they would be immediately rebuked or scolded by others around them.

Retard and retarded also have a legitimate definition and use in math, that has nothing to do with intellectual disabilities.