100
“On Tuesday afternoon, ChatGPT encouraged me to cut my wrists.”
(www.theatlantic.com)
This is a most excellent place for technology news and articles.
That's called testing, and the companies behind these LLMs should, before launch, put a very important amount of their resources into testing.
"Product testing is a crucial process in product development where a product's functionality, performance, safety, and user experience are evaluated to identify potential issues and ensure it meets quality standards before release" (Gemini)
We are literally using alpha/beta software to deal with life altering issues, and these companies are, for some reason, being able to test their products on the public, without consequences.
It's like you bought a car and deliberately hit the wall to make a headline "cars make you disabled". Or bought a hammer, hit your thumb and blame hammers for this.
Guys, it's a TOOL. Every tool is both useful and harmful. It's up to you how you use it.
Car makers test exactly that, and for good measure since cars can and do crash!
What are you suggesting, that we buy cars that didn't pass crash tests?
To me it seems like you arguing something similar for AI.