There have been a number of articles about how GPT has been out-diagnosing doctors in various domains. To me, that isn't that surprising as diagnosis is a pattern matching problem, something a neuralnet will be very good at. Human doctors were seen to be discounting rare conditions just because they were rare and so "it was much more likely to be something else" even if the symptoms backed up the conclusion. A computer can be more objective about such things.

...but none of that needs AI/ML. We've had expert systems since the 60s.

It's also very different from constructing a treatment plan, which is what we're discussing here.

Or they could feed the current model with a reputable source of medical information.