The issue is the marketing. If they only marketed language models for the things they are able to be trusted with, summarization, cleaning text, writing assistance, entertainment, etc. there wouldn't be nearly as much debate.

The creators of the image generation models have done a much better job of this, partially because the limitations can be seen visually, rather than requiring a fact check on every generation. They also aren't claiming that they're going to revolutionize all of scociety, which helps.