Helen ZhangIn a recently published study, a team from the University of Alberta, Harvard, Stanford, and the Massachusetts Institute of Technology (MIT) tested the performance of OpenAI’s o1-preview model. They used a series of challenging clinical cases against physician assessments. They found that the model had a high degree of accuracy in diagnosis and was capable of providing helpful plans for further tests.
Liam McCoy, co-author of the study and U of A neurology resident, said the findings show that the integration of artificial intelligence (AI) tools within clinical settings will be a crucial part of medical advancement. He predicted that their use as a second opinion or diagnostic support for physicians will become extremely common across the field.
“I think by around 2030, it’s going to be seen as unethical for clinicians not to be using these tools regularly in clinical practice, in the same way it would be unethical not to order an MRI when a patient needs it,” McCoy added.
More trials needed before full integration
Given the power of AI tools and their rapid advancement, the potential benefits that come with their integration in medical settings are becoming “increasingly clear.” However, there are still risks and weaknesses that need to be investigated and alleviated before integration.
Previously research found that AI may struggle to reason and update judgement when faced with changing clinical information. The uneven effectiveness of AI could pose problems for physicians interacting with assistance tools.
McCoy pointed to a need for investigating how AI models interact cognitively with physicians. He also noted trials examining AI autonomously as a future area of research. The research would assess areas in clinical practice where AI may independently follow guidelines and provide care.
“Ultimately, as with any intervention in health care … the thing we need is randomized controlled trials that show us the teams using these tools are changing the things that we actually care about for our patients,” McCoy stated.
Missing pieces with AI assessments
Real world medical care also involves more than text-based reasoning. It incorporates physical examination, listening to patients, and understanding medical and social contexts. According to McCoy, physicians play a large role in listening to patients and advocating for their interests. These functions may be poorly handled by AI, potentially making the health-care system less human.
McCoy also cautioned against crystallizing current health-care practices within current AI systems, due to flaws embedded in training data used by these systems. However, he sounded an optimistic note in his assessment of the usefulness of this technology.
“I don’t think these risks are insurmountable, and I am very hopeful that this technology will allow us to achieve some really incredible things and improve a lot of patients’ care.”
