A new artificial intelligence system built by Microsoft correctly diagnosed complex medical cases far more often than doctors did in a controlled study, and the technology is now headed to a Boston hospital for testing on real patients.
In research published Oct. 22 in the journal Science, Microsoft’s AI diagnostic system, called the MAI Diagnostic Orchestrator or MAI-DxO, correctly identified 83% of 304 difficult medical cases drawn from The New England Journal of Medicine. Generalist physicians who could consult reference materials got 36% of a separate set of 56 cases right. Without those materials, physician accuracy dropped to 22%.
The system also performed well on historical emergency department cases, reaching about 80% accuracy compared with 47% for the physician comparison group.
How it works
MAI-DxO relies on multiple AI agents that function like a panel of doctors debating a tough case. Rather than settling on an immediate answer, the system starts with limited information, asks follow-up questions, weighs which tests might help, and has its component agents challenge each other’s conclusions as new details emerge.
The underlying AI models matter a great deal. Researchers tested systems from OpenAI, Google, Anthropic and others. OpenAI’s GPT-5, used with a simple prompt and no orchestration, already solved about 79% of complex cases on its own; adding Microsoft’s system pushed that to roughly 83%. With the newer GPT-5.5 model, Microsoft’s orchestration lifted accuracy from about 72% to nearly 87% on complex cases, though it added little benefit on emergency cases when paired with the strongest models.
Important caveats
The comparison was not doctor versus doctor in equivalent conditions. The physicians involved were generalists, including primary care doctors and general internists, evaluating cases individually. Hospitals facing genuinely difficult diagnoses typically bring in specialists who consult with one another, and Microsoft did not test its system against teams of specialists.
The physicians and the AI also were not scored on identical case sets: the AI worked through 304 cases while physicians worked through 56 held-out cases. The scenarios themselves were simulations built from existing case reports, with an AI system filling in gaps when case details weren’t available — a setup that cannot fully capture the messiness of real patient visits, incomplete records, or physical exam findings that never make it into a written summary.
The study’s authors are Microsoft employees, a point the company itself has acknowledged matters for independent verification before hospitals consider relying on the technology.
Cost savings, with caveats
Researchers also assigned estimated costs to diagnostic procedures to see whether the AI could reach accurate conclusions while limiting unnecessary testing. In one cost-conscious setup, MAI-DxO hit about 81% accuracy on complex cases while generating an estimated $2,241 in diagnostic costs per case, versus about $4,725 for physicians using reference materials. Other configurations tuned for maximum accuracy produced higher costs. These figures are simulation estimates, not real patient bills or a measure of what running the AI costs.
Next stop: a Massachusetts hospital
Microsoft has announced a partnership with Beth Israel Deaconess Medical Center in Boston to test MAI-DxO in an actual clinical setting. Under the planned research, consenting patients will continue to receive standard care while also interacting with the AI system, and researchers will compare its recommendations against physicians’ independent assessments. Doctors will remain responsible for treatment decisions throughout.
Microsoft CEO of AI Mustafa Suleyman has described the company’s broader goal as building
You must be logged in to post a comment Login