I read this story this morning and have been thinking back to it all day. This wasn't just some idiot that was too stupid or young to not realize he was talking to a bot and did something like drink bleach because it told him to.

This was one of us.

He fit lots of behaviors I see here from me and my fellow Lemmy posters. He:

• built computers for himself and family members
• was a hobbyist (at least) coder
• wasn't a young kid that didn't know the world. He was 48 or 49.
• was an early adopter embracing the modern LLM technology in 2022 when it first really became public.
• sold his house in an urban metropolis (Portland) and moved to a rural area so he could use his additional wordworking skills on building sustainable housing.
• worked part time at a homeless shelter

Doesn't this guy sound like someone that would be a Lemmy poster to you too?

He started using LLMs (ChatGPT specifically) as a tool only to advance his hobby and work. When he first started it appears he understood it was just a tool, and didn't think it was something sentient. Only later after hundreds of hours of exposure did this idea arise in him.

Was there some underlying psychological problem that the LLM exacerbated? Possibly. But at what level was his original underlying issue? Do we all have some low level condition that would make us equally susceptible? I know we'd like to think we don't, but how do we know? This man certainly didn't think he did, I'm sure.

Next I think about what it would take for me to get down this bad path without realizing it. At one point would I be talking to a chat bot, not realize it, and let what that chat bot said change or influence my thoughts when I'd have zero knowledge of it being just a fancy program? I consider myself moderately smart with good critical thinking skills, but I'm sure this man did too.

Then it occurred to me that I have to concede that I have, at some point, already interacted with a bot in years past on Reddit or even today on Lemmy and I had no idea it was a bot. Was that interaction a throwaway conversation about pop culture that would have no impact on my world view or was it a much deeper and important political or philosophical conversation that the bot introduced an idea or hallucinated evidence to support a point and I didn't catch it to challenge it? Am I already a few or many steps down the bad path of falling for illusions of a bot? I certainly don't think so, but neither did he.

How many of us are already on the same path as this guy and just as ignorant about the danger as the man in the article?

He had also recently stopped using ChatGPT.

Oof. He just... mainlined reality? Yeah that shit will fuck you up. These days they are cutting it with hatred and fascism and oligarchy... it's more than a human mind can take. You gotta ease into it and build up your tolerance or it will drive you mad.

Richardson remembers that whenever Ceccanti would emerge from the basement for some air, he would start having “philosophical” talks about “how his work with the AI was telling him he was breaking math and basically reinventing physics”. As she’d listen to him, Richardson would think about the fact that Ceccanti did not have any college or university experience. He had never even taken calculus.

Tangent, but I think this is another facet of why education is important: so people know what they don't know. I think it's harder to think you're reinventing physics when you've taken some classes and seen all the work people have already done.

On the other hand, delusions can just be whatever so education isn't a panacea.