this is like if there was a distributed model of cults personalized to each person involved

although a big part of this, reading this is that his real life friends seemed to have encouraged him. The article quotes the guy repeatedly saying to the LLM that he's skeptical, asking if the LLM is just roleplaying, and it would shoot back at him authoritatively that it's not a roleplay or a game, but it's a real mathematical discovery.

I'm starting to think LLMs should by law have a message in every chat bubble that says something like "DO NOT TRUST ANYTHING THIS SAYS. I AM NOT REAL AND I DON'T KNOW WHAT I'M SAYING" in big impact font but I don't even know if that would work