his home treated him as honorable and other cultures don't

Not the point of the story, when NPCs get to know who the character is theirs opinion changes

it wasn't because they mistook him for being the kind of person you get to do that to

That is actually almost what happened. If he was not a mage, that story point would change little.

The "mages are slaves" thing is more akin to FF6's "there is no magic in this world", like it is a somewhat big deal that Terra is mage, but game doesn't spend much time there since it is not a point.