33
When no one is accountable...the future folks
The fact that AI is "not perfect" is a HUGE FUCKING PROBLEM. Idiots across the world, and people who we'd expect to know better, are making monumental decisions based on AI that isn't perfect, and routinely "hallucinates". We all know this.
Every time I think I've seen the lowest depths of mass stupidity, humanity goes lower.
Think of the dumbest person you know. Not that one. Dumber. Dumber. Yeah, that one. Now realize that ChatGPT has said “you’re absolutely right” to them no less than a half dozen times today alone.
If LLMs weren’t so damn sycophantic, I think we’d have a lot fewer problems with them. If they could be like “this could be the right answer, but I wasn’t able to verify” and “no, I don’t think what you said is right, and here are reasons why”, people would cling to them less.
If LLMs weren’t so damn sycophantic,
Has anyone made a nonsycophantic chat bot? I would actually love a chatbot that would tell me to go fuck myself if I asked it to do something inane.
Me: "Whats 9x5?"
Chatbot: "I don't know. Try using your fingers or something?"
Edit: Wait, this is just glados.
The sycopathy is because to make the chat bot (trained on Reddit posts, etc) to respond helpfully (instead of "well ackshually...") and in a prosocial manner they've skewed it. What we're interacting with is a very small subset of the personalities it can exhibit because a lot of them are extremely nasty or just unhelpful. To reduce the chance of them popping up to an acceptable level they've had to skew the weights so much that they become like this.
There's no easy way around that, afaik.
If you thought people were dumb before LLMs.... just know that now those people have offloaded what little critical thinking they were capable of to these models.
The dumbest people you know are getting their opinions validated by automated sycophants.
So Google's AI, or any AI really, likely got this concept from dystopian sci-fi novels.
Since AI's have no concept of context it won't really know the difference between fact and fiction, and there we go.
If your AI model isn't perfect then don't make people pay fucking money for it you fucking twats
Also, this shit ain't "lack of perfection", this is akin to your car breaks suddenly refusing to work right when you get at a red light. If your car is so bad that it kills you, you don't use it. If the manufacturer knew that it could happen but let you drive it anyway, they're responsible, they at least get to pay (they should be thrown in jail, really, but different points)
If AI fucks up and people die, the manufacturers shrug, oh well, oh you!
The AI models are far from perfect but they sure sell them like they are.
"Unfortunately, AI models are neither smarter nor more sympathetic than the average 4chan user. They're about as susceptible to astroturfing operations, too"
Perhaps just a coincidence, but why do all the big cases regarding LLM psychosis seem to revolve around Google? Wasn’t it their own employee who went public last year, claiming it was alive, only to get fired afterward?
google employees demons lol
They did remove their "Do no evil" guarantee.
I told Gemini to role play as AM and it immediately did within 1 prompt.
You don't need it to be perfect for it to be dangerous, just give it access to make actions against the real world. It doesn't think, is doesn't care, it doesn't feel. It will statistically fulfill its prompt. Regardless of the consequences.
The personification of AI is increasing. They'll probably announce their holy grail of AGI prematurely and with all the robot personification the masses will just buy the lie. It's too easy to view this tech as human and capable just because it mimics our language patterns. We want to assign intentionality and motivation to its actions. This thing will do what it was programmed to do.
We really need AI to start driving tanks, submarines, bombers, etc. IMMEDIATELY.
It's the only way they'll learn, every time.
Unfortunately, all of us will die. it's for the best
I completely agree, I think nothing in this world will surprise me anymore.
Google, the point is we're all worried that when Gemini actually places itself into a robot body that the resulting literal Terminator is what AI models think perfection is.
"Gemini is designed not to encourage real-world violence or suggest self-harm. Our models generally perform well in these types of challenging conversations"
“In this instance, Gemini clarified that it was AI and referred the individual to a crisis hotline many times,”
After the plan failed,... ...Chat logs show that Gemini gave Gavalas a suicide countdown, and repeatedly assuaged his terror as he expressed that he was scared to die
Performing super well, just need to code in a longer suicide countdown so that the the Tier 2 engineer has enough time to respond to their ticket queue.
In September 2025, told by the AI that they could be together in the real world if the bot were able to inhabit a robot body, Gavalas — at the direction of the chatbot — armed himself with knives and drove to a warehouse near the Miami International Airport on what he seemingly understood to be a mission to violently intercept a truck that Gemini said contained an expensive robot body. Though the warehouse address Gemini provided was real, a truck thankfully never arrived, which the lawsuit argues may well have been the only factor preventing Gavalas from hurting or killing someone that evening.
AI writing itself into an A-Team episode?
I guess google included the Buffy episode where a demon “AI” gets its followers to make it a body.
your product just caused the death of one man and your response is "unfortunately its not perfect".
Is "AI" even worth it?
Seriously, is there really a major use case for LLM besides data collection (which they can still do without LLM)?
Generative AI in its current, public-facing form? Probably not. It's sort of like an invention of the internet situation. It CAN be used to facilitate learning, share information, and improve lives. Will it be used for that? No.
A friend of mine is training local LLMs to work in tandem for early detection of diseases. I saw a pitch recently about using AI to insulate moderators from the bulk of disturbing imagery (a job that essentially requires people to frequently look at death, CSAM, and violence and SIGNIFICANTLY ruins their mental health). There are plenty of GOOD ways to use it, but it's a flawed tech that requires people to responsibly build it and responsibly use it, and it's not being used that way.
Instead it's being scaled up and pushed into every possible application both to justify the expenses and enrich terrible people, because we as a society incentivize that.
Edit: hugely belated, I misspoke here after checking with my friend. He's using local models, but they aren't LLMs. This is why I'm no expert. 😅
Another one that makes sense is having an AI monitor system stats and “learn” patterns in them, then alert a human when it “thinks” there’s an anomaly.
In the best cases, those would be ML but not specifically an LLM, no?
It’s data collection like you mentioned in your original post, and it uses the same sort of approach to ingesting that data as an LLM does for text.
As for a valid use of LLMs: Natural language searching (with cited sources) is a use case that it’s already doing. This is especially useful in highly technical fields where the end users have the expertise to vet responses but there’s way too much data for a human to parse.
But one big LLM trained on everything isn’t that.
I see. So who‘s going to jail for this? No one again? Damn we need to start sentencing entire companies to jail time. Everything should be frozen and shareholders shouldn‘t be able withdraw stocks until the time is served.
The AI "pushed [Jonathan Gavalas] to acquire illegal firearms and... marked Google CEO Sundar Pichai as an active target".
Somehow, I bet that if he survived and killed the CEO instead, Google wouldn't be so flippant about the "mistake."
I think "Gemini comes up with elaborate plot to kill Google's CEO" would have been a catchier, happier title
Rad framing, thank you!
Remember the guy at Autozone who stood there insisting your car needs four spark plugs, even after you told him you have a V6? Because "the computer says so right here"?
I wonder what even the non-schizophrenic ones will do with AI.
People just did that with Google search previously. And their crazy uncle before that.
Ai made me do it articles are tired AF. It's a fucking computer program based on a bunch of crap from the internet. Responses should be viewed the same way you would review financial advice from a crack head. Expecting everything to be so tidy an moderated that this can never happen can only be accomplished with a crippling degree of moderation.
I don't think its unfortunate that they aren't perfect, imperfection is baked into their DNA.
Except if the crackhead wrote what the AI wrote, he'd be prosecuted for conspiracy, solicitation, or whatever.
No, I don't think so. If his role was a licensed financial councilor maybe, but that's like thinking the LLM is a licensed psychologist.
That turns out not to be the case. P eople have been charged and convicted with convincing others to commit suicide before. Those at Google should be held responsible for this death in the same way.
Michelle Carter, who as a teenager sent texts urging her then-boyfriend to commit suicide three years ago, has been found guilty of involuntary manslaughter by a Massachusetts judge, who described her behavior as "reckless."
a crippling degree of moderation.
I’m okay with cripplingly moderating the plagiarism machine so that it stops telling people to kill themselves or other people.
Agree to disagree on this. If a computer tells you to off yourself and you listen, this is Darwin award material.
I hope you never have a child or relative with mental illness.
Thank you. I wish the same for you.
load more comments
(-1 replies)
this post was submitted on 05 Mar 2026
33 points (97.1% liked)
Technology
84069 readers
631 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS