I'm no GPT booster, but I think that the real problem with detectability here

It will almost always be detectable if you just read what is written. Especially for academic work.

is that it requires you to know the subject and content already, and to be giving the paper a relatively detailed reading. For a rube reading the paper, trying to learn from it - a lot of GPT content is easily mistaken as legitimate. And it's getting better. We're not safe simply assuming that AI today is as good as it will ever get and the clear errors we can detect cannot ever be addressed.

Penetrating academic writing, for academics, is probably one of the highest barriers of any writing task, AI or not.

But being dismissive of the threat of AI content because it's not able to convincingly fake some of the hardest writing that real people do is maybe sidestepping a lot of much more casual writing - that still carries significance and consequence.