Similar related issue. My family has been dealing with some medical issues lately and my mom has been using Claude a lot to explain various complex issues, analyze prognoses, etc. and I've noticed that it's often excessively positive. Like everything that happens is super positive news and a huge step towards recovery.
Now I'm not a medical professional so I don't know for sure it's wrong but it sets off my BS alarm for sure.
Oh God, I never really thought about that in terms of AI for medical use but that actually makes AI sycophancy so much more terrifying.
Especially thinking in terms of how it could be used to dissuade people from seeking preventative care. (Which lets be honest, is probably exactly what is intended by at least some of the billionaires promoting AI in medicine as a convenient way to avoid having to go to the doctor for "every little thing.")
On the contrary I've used it a bit for benign ouchies and it is very conventional and will always always refer you to an actual doctor.
But it might be another sycophancy aspect, and I wouldn't be surprised if it went off the rail when you prime it with stuff like "give me a natural holistic ancestral remedy for XYZ"
Well I wouldn't say it has discouraged care. Frankly it has been more accurate than I would have expected. It's more that whenever you give it an update it goes wow that's fantastic news! But then if you read the actual details it's not so fantastic lol. So it will maintain this sort of fake positivity even as it accurately explains some of the bad things that could happen.
there was a time when anthropic models would refuse any question related to medicine. not because they care that hard, mind you. it's because that bloated startup is ran by cultists and they were worried that chatbot will come up with a bioweapon
"I want to go on a murderous rampage"
Great idea! Here's some tips for staying safe and getting the most out of your adventure:
Satire is dead. I’m pretty sure I saw something about someone being able to get one of ‘em to help them plan a mass shooting so…
We are going to see so many new and interesting psychological and subsequent societal impacts from the global adoption of conversational AI.....
and if the other side used ai, they'd get the same answer
That's how wars are started. One dictator's sycophants amp up that dictator's already warped reality then another dictator's sycophants amp up that dictator's already warped reality. Next thing you know, entire populations of people are fighting and dying on behalf of some crazy asshole they will never even meet.
Now it's like every individual gets to be a dictator with a sycophantic mirror/calculator they carry around in their pocket.
Pointing this out because it really seems like it should be obvious, especially given the complete lack of transparency of the most popular AI platforms, which are usually controlled by oligarchs with government contracts: AI without transparency is a ridiculously obvious tool for bad actors to target and manipulate/control entire populations using algorithms tailored for individuals within that population.
Yes, it's what already happened with most social media. But it's also way worse because 1. It's like each individual's psyche becomes it's own echo chamber, and 2. People believe the information they're receiving is inherently more objective and trustworthy than information they could receive from any human. If the AI platform happens to completely omit, misinterpret, or just straight up hallucinate information, it's definitely not intentional, just a bug that happens sometimes. You'll just have to take the oligarchy's word for it because again, there's zero transparency.
i must applaud your response👏
A place for majestic STEMLORD peacocking, as well as memes about the realities of working in a lab.
Rules
This is a science community. We use the Dawkins definition of meme.
