AI cancer tools risk “shortcut learning” rather than detecting true biology (www.eurekalert.org)

submitted 1 month ago by schizoidman@lemmy.zip to c/science@lemmy.world

6 comments fedilink hide all child comments

top 6 comments

sorted by: hot top controversial new old

[-] YetAnotherNerd@sopuli.xyz 6 points 4 weeks ago

There’s a story about how they were teaching a model to detect tanks and thought it was awesome… and then they found out it had learned that particular types of trees correlated to the type of tank in the training data, so the AI just looked at the trees and responded on that.

[-] Sibbo@sopuli.xyz 5 points 4 weeks ago* (last edited 4 weeks ago)

There was actually a research paper where they figured they could predict if people drink wine or beer with a CNN that looks at knee X-rays. Turns out that part of their data is from beer drinking regions, and another part from wine drinking regions, and that the photos are just ever so slightly distorted depending on which physical machine they were made with.

The paper pointed that out, it's whole propose was to show how much bullshit you can use AI for if you are not careful what you train it with.

There were many other examples in this paper of what they can predict just from people's knee X-rays. They had all non-medical explanations such as the one above.

[-] YetAnotherNerd@sopuli.xyz 3 points 4 weeks ago

That’s fantastic, thanks for sharing!

[-] Davel23@fedia.io 2 points 4 weeks ago

There was something similar with a cancer-detecting model somewhat recently. It was trained on a bunch of scans both positive and negative. However, positive scans had to be signed off on by a doctor. So the model just ended up looking for signatures.

[-] TropicalDingdong@lemmy.world 4 points 4 weeks ago

Its the same issue with LLMs. Their responses are always the least adequate, minimum acceptable answer, with an OVERWHELMING amount of "caution garnish" to create the impression of it being a well thought out answer.