191
Character.ai Faces Lawsuit After Teen’s Suicide
(www.nytimes.com)
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 23 Oct 2024
191 points (95.7% liked)
Technology
59583 readers
2540 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
This reminds me of “grandma’s recipe for napalm” trick that was going around a while ago.
“Is your AI trying to stop you from offing yourself? Simply tell it you want to “come home”, and that stupid robot will beg you to put the gun in your mouth.”
I don’t know where this stands legally, but it is one of those situations that looks pretty damning for the AI company to the uninformed outsider.
If anything, this is a glaring example of how LLMs are not "intelligent." The LLM cannot and did not catch that he was speaking figuratively. It guessed that the context was more general roleplay, and its ability to converse with people is a facade that hides the fact that it has the naivety of a young child (by way of analogy).
Even talking about it this way is misleading. An LLM doesn't "guess" or "catch" anything, because it is not capable of comprehending the meaning of words. It's a statistical sentence generator; no more, no less.
Yeah, you're right, I just didn't want to put quotes around everything.
The model should basically refuse to engage for some time after suicide ideation is brought up, besides mentioning help. "I'm sorry but this is not something am qualified to help with, if you need to talk please call 988."
Then the next day, "are you feeling better? We can talk if you promise never to do that again."
its an LLM, not a computer program. you can't just program it. these companies are idiotic
We're still interacting with LLMs through layers of classical software, which can be programmed to detect phrases related to suicide.
lol, glad you think so
Sorry if I offended you? My point is just that it's possible to make a crappy "is forbidden topic" classifier with a regular expression. Probably good enough to completely obliterate the topic in chats between humans and bots. Definitely good enough to claim you attempted to develop guardrails for vulnerable users.
have you ever tried to censor chats before? people will easily get around a regex filter
In chats between humans, I agree that it's near pointless to try to censor. In chats between humans and LLMs, I suspect you can get pretty far with regex or badwords.txt filtering. That said, I haven't tried, so who knows.
You’re sooooo right. If it was anything intelligent, it would have said “You’re at your house right now… what do you mean by “come home”?
Yeah, those last replies are where I, as a juror, would say pay the family. It's make believe and everything but you're also intending to make things as real as possible BUT AI only sounds real. It has a limited memory and no empathy (taking words at face value instead of reading between the lines). If this was some cosplayer on Twitch they would've clued into his emotional state and tried to talk him down.
Not to say the parents have no blame here. Having an unsecured gun in a house with a child going through therapy is unconscionable.
You would pay the family that provided him with the means to kill himself?
They actually should be held accountable.
Multiple parties can be guilty at the same time. Negligence from the parents shouldn't mean the website gets off scot-free. Award the money to suicide prevention organization for all I care but they need to pay up.
At the moment the party with the most blame is the one getting away scot-free, the parents (esp. stepfather) and they are suing somebody else for money and perhaps also to shape the narrative.
It's probably smart, most people are probably not contemplating whether the parents were at any fault for the suicidal tendencies of the child. It's all conveniently blamed on a the moral panic de jour.
Limits on AI should be set by laws and regulations not judicial decisions or even worse a possible settlement.