A medical research paper entitled “Clinical knowledge in LLMs does not translate to human interactions” tested if LLMs can assist members of the public in identifying underlying conditions and choosing a course of action (disposition) in ten medical scenarios in a controlled study with 1,298 participants. The researchers reported: “Tested alone, LLMs complete the scenarios accurately, correctly identifying conditions in 94.9% of cases and disposition in 56.3% on average. However, participants using the same LLMs identified relevant conditions in less than 34.5% of cases and disposition in less than 44.2%, both no better than the control group.”
The researchers stated:
Despite LLMs alone having high proficiency in the task, the combination of LLMs and human users was no better than the control group in assessing clinical acuity, and worse at identifying relevant conditions. Previous work has shown that using LLMs does not improve clinical reasoning in physicians, and we find that this extends to the general public as well. We further identify the transmission of information between the LLM and the user as a particular point of failure, both users providing LLMs with incomplete information and LLMs suggesting correct answers but not effectively conveying this information to the users. We consider two common testing approaches for medical capabilities in LLMs, and find that although they may assess the medical information stored in the LLMs, they do not reflect the challenges of user interactions in deployment.
For a public-facing medical LLM to exist, we expect that it would need to be proactive in managing and requesting information rather than relying on the user to have the expertise to guide the interaction … We found that patients using LLMs have low accuracy in understanding the acuity of their symptoms and in identifying the etiology, comparable to participants using traditional approaches.
As a realistic assessment of using LLMs for direct patient care, we find that current models are not ready for deployment. Overall accuracy was low, with participants using LLMs scoring below 50% on both tasks, though in a real deployment the accuracy could change depending on the relative frequencies of the scenarios. Despite strong performance from the LLMs alone, both on existing benchmarks and on our scenarios, medical expertise was insufficient for effective patient care.
We identify user interactions as a challenge to the deployment of LLMs for medical advice. Standard benchmarks for medical knowledge and simulated patient interactions do not predict the failures we find with human participants. Moving forward, we recommend systematic human user testing to evaluate interactive capabilities prior to public deployments in healthcare.
If you or a loved one were injured (or worse) as a result of possible medical malpractice in the United States, you should promptly find a medical malpractice lawyer in your state who may investigate your medical malpractice claim for you and represent you or your loved one in a medical malpractice case, if appropriate.
Visit our website or call us on our toll-free number in the United States (800-295-3959) to find medical malpractice attorneys near you who may help you.
Turn to us when you don’t know where to turn.