Yeah ChatGPT could fill out an application as well. In fact AI is getting to the point now where it would be hard to tell even by voice. Though it's also a matter of effort on the part of the exploiter. They don't have to make it zero occurrence, just enough to keep it at bay.

It may be an AI, or it can also be a real human that is lying. The point of the application filter is to significantly slow down these approaches to bring their impact to a more manageable level. An automated AI bot will not be able to perform much better than a human troll with some free time because any anomalous registration patterns, including registration spikes and periodicity, are likely to be detected by the much more powerful processor that resides in the admin's head.

On the other hand, a catch-all domain e-mail, a VPN with a variable IP, and a captcha-defeating bot can be used to generate thousands of accounts in a very short amount of time. Without the application filter the instance is vulnerable to these high-throughput attacks, and the damage can be difficult to fix.