347
submitted 4 days ago* (last edited 4 days ago) by AcidiclyBasicGlitch@sh.itjust.works to c/science_memes@mander.xyz
you are viewing a single comment's thread
view the rest of the comments
[-] LibertyLizard@slrpnk.net 54 points 4 days ago

Similar related issue. My family has been dealing with some medical issues lately and my mom has been using Claude a lot to explain various complex issues, analyze prognoses, etc. and I've noticed that it's often excessively positive. Like everything that happens is super positive news and a huge step towards recovery.

Now I'm not a medical professional so I don't know for sure it's wrong but it sets off my BS alarm for sure.

[-] AcidiclyBasicGlitch@sh.itjust.works 26 points 4 days ago* (last edited 4 days ago)

Oh God, I never really thought about that in terms of AI for medical use but that actually makes AI sycophancy so much more terrifying.

Especially thinking in terms of how it could be used to dissuade people from seeking preventative care. (Which lets be honest, is probably exactly what is intended by at least some of the billionaires promoting AI in medicine as a convenient way to avoid having to go to the doctor for "every little thing.")

[-] Zos_Kia@jlai.lu 7 points 4 days ago

On the contrary I've used it a bit for benign ouchies and it is very conventional and will always always refer you to an actual doctor.

But it might be another sycophancy aspect, and I wouldn't be surprised if it went off the rail when you prime it with stuff like "give me a natural holistic ancestral remedy for XYZ"

[-] LibertyLizard@slrpnk.net 3 points 4 days ago

Well I wouldn't say it has discouraged care. Frankly it has been more accurate than I would have expected. It's more that whenever you give it an update it goes wow that's fantastic news! But then if you read the actual details it's not so fantastic lol. So it will maintain this sort of fake positivity even as it accurately explains some of the bad things that could happen.

[-] fullsquare@awful.systems 3 points 4 days ago

there was a time when anthropic models would refuse any question related to medicine. not because they care that hard, mind you. it's because that bloated startup is ran by cultists and they were worried that chatbot will come up with a bioweapon

this post was submitted on 19 May 2026
347 points (98.9% liked)

Science Memes

20273 readers
640 users here now

Welcome to c/science_memes @ Mander.xyz!

A place for majestic STEMLORD peacocking, as well as memes about the realities of working in a lab.



Rules

  1. Don't throw mud. Behave like an intellectual and remember the human.
  2. Keep it rooted (on topic).
  3. No spam.
  4. Infographics welcome, get schooled.

This is a science community. We use the Dawkins definition of meme.



Research Committee

Other Mander Communities

Science and Research

Biology and Life Sciences

Physical Sciences

Humanities and Social Sciences

Practical and Applied Sciences

Memes

Miscellaneous

founded 3 years ago
MODERATORS