AI Diagnostic Tools Mira and Amie Match or Surpass Doctors in Simulated Tests

​Two new artificial intelligence systems, Mira and Amie, have matched or outperformed doctors in several diagnostic and treatment tasks, according to studies published in Nature. The findings highlight AI's growing capabilities in medicine, but researchers caution that these systems are not yet ready for use with real patients.

Key Highlights

• Mira and Amie AI systems matched or outperformed doctors in simulated diagnostic and treatment tasks.
• Mira achieved 87.1 percent diagnostic accuracy compared to 78.1 percent by physicians in emergency scenarios.
• Amie produced treatment plans closely aligned with clinical guidelines in 100 simulated patient cases.
• Researchers caution that AI performance in controlled tests may not reflect real-world clinical environments.

AI Systems Tested Against Doctors

Mira was developed by a research team in Germany, while Amie was created by Google and uses the Gemini AI model. Both systems were evaluated using simulated patient scenarios. In many cases, their results equaled or surpassed those of medical professionals, according to the Financial Times.

These results add to evidence that healthcare-specific AI models may provide more reliable support than general-purpose chatbots. However, scientists stress that strong performance in controlled experiments does not guarantee similar results in real clinical environments.

Jakob Kather, a researcher from TUD Dresden University of Technology and Heidelberg University, helped develop Mira. He compared AI assistants to aircraft autopilot systems, suggesting that such tools could ease the workload of healthcare professionals while keeping final decisions with humans.

Mira's Performance in Simulated Emergencies

Mira was designed to work with electronic health records and can recommend tests, medicines, and procedures. The system has access to over 85,000 possible clinical actions. Researchers tested Mira using information from more than 500 emergency department cases. Instead of real patients, Mira interacted with AI agents simulating patient behavior.

Mira was tested across eight medical conditions, including pancreatic cancer, pneumonia, appendicitis, and pulmonary embolism. It achieved a diagnostic accuracy rate of 87.1 percent. In comparison, a group of six physicians from various specialties scored 78.1 percent under the same conditions.

Amie's Strength in Treatment Planning

Google's Amie was evaluated differently. Researchers created 100 patient scenarios based on UK healthcare guidelines. Actors role-played patients during text-based consultations. Amie was then compared with 21 primary care physicians.

The study found that Amie matched doctors in patient management decisions. In several cases, it produced treatment and investigation plans more closely aligned with clinical guidelines. The AI also handled medication-related decisions well in complex cases.

Limitations and Expert Opinions

Despite promising results, both research teams acknowledged limitations. Mira sometimes recommended care that did not fully match accepted medical practice. Researchers noted that simulated patient information was likely more organized and complete than data in real emergency settings.

Google's team expressed similar concerns. They described the study as a significant step but said the testing environment did not reflect the unpredictability of real-world healthcare. Amie still needs further development to reduce reasoning mistakes and improve consistency before practical use.

Independent experts welcomed the studies but urged caution. Catherine Pope, Professor of Medical Sociology at the University of Oxford, noted that real healthcare settings are much messier than simulations. Patients may provide incomplete information or present multiple issues at once.

Julie Jacko, Professor of Health Informatics and Data Science at the University of Edinburgh, said AI could create detailed care plans but this does not mean it has better clinical judgment than experienced doctors. Researchers also noted that some of Amie's strong performance may reflect rapid improvements in modern AI models, not just innovations specific to healthcare.