Stagnation of language and culture is a concern.

I think this is a much bigger problem than a lot of the supporters of AI are willing to consider. There's already some evidence that feeding AI-generated content into AIs makes them go a bit... strange in a way that renders their output utterly worthless. So AIs genuinely cannot create anything new on their own, only derivative works based on human-made content that is fed into them.

So, in order for AIs to progress, there still needs to be human creatives making truly original content. But if all human-made content is immediately vacuumed up into an AI, preventing the human creative from ever making a living off their work (and thus buying those pesky luxuries like food and shelter), then under the social system we have right now, humans won't produce new creative work. They won't be able to afford to.

Thus, the only logical solution is that if the developers of AIs want to train them on human-made works, they're just going to have to compensate the authors, artists, etc. Otherwise the AIs will stagnate, and so will language and culture because of the threat AIs pose to the livelihoods of the people who create new language and culture. Even if humans are still creating new works, if there's a genuine risk of it being taken by AI companies and fed into the bots, the humans will be a lot more cautious about posting their work publicly, which again leads to stagnation.

It's almost like new technologies actually work best when the wealth they generate is distributed to everyone, not just hoarded by a few.