Leveraging AI for Multiple Sclerosis Diagnosis
A recent study explored the potential of large language models (LLMs), such as GPT-4, in diagnosing multiple sclerosis (MS) based on clinical notes. Given the frequent delays in MS diagnosis, the research aimed to assess whether artificial intelligence (AI) could effectively classify MS status using structured decision-making aligned with the 2017 McDonald criteria. The study’s findings were presented at the 2025 Americas Committee for Treatment and Research in Multiple Sclerosis (ACTRIMS) Forum.
Researchers analyzed the first neurology note from 125 patients (105 with MS, 10 with related disorders, and 10 healthy controls) from 2017 to 2023. To ensure objective assessment, they redacted the clinician’s final diagnosis and plan while retaining patient’s history and diagnostic test results. A decision tree based on the 2017 McDonald criteria guided GPT-4 in classifying patients into MS-confirmed, MS-not confirmed (with varying diagnostic gaps), related disorders, or healthy controls.
GPT-4 correctly classified 74% (93/125) patients, including 70% of patients with MS, 100% of related disorders, and 90% of healthy controls. The model’s accuracy varied across MS categories, ranging from 67% to 79%, depending on missing diagnostic criteria. However, 30% of MS cases were misclassified, with the most common errors being incongruence hallucinations (77%), overreliance on specific data points (48%), nonfactual (16%), irrelevance (19%), and reasoning (19%) hallucinations.
The findings highlight both the promise and challenges of AI-assisted MS diagnosis. While LLMs offer potential for streamlining early detection, researchers emphasize the need for rigorous validation and hallucination mitigation before clinical deployment. Future studies will reassess this approach using the newly adopted 2024 MS diagnostic criteria.
Reference
Venkatesh S, DelSignore M, Wu X, et al. Deconstructing Complex Diagnostic Criteria and Leveraging Generative Artificial Intelligence to Facilitate Multiple Sclerosis Diagnosis. Presented at: 2025 ACTRIMS Forum; February 27-March 1; West Palm, Beach, FL; Abstract P03.
