A recent study published in NEJM AI has provided a clear look at how artificial intelligence models approach clinical decision-making compared to human physicians. By analyzing 200 deidentified outpatient and emergency department notes, researchers sought to determine whether AI systems would mirror the diagnostic habits of doctors or lean toward more aggressive testing protocols.

The findings suggest that when left to their default settings, large language models (LLMs) consistently favor a more intensive approach to patient care. This tendency toward over-testing highlights a distinct divergence between human clinical judgment and the algorithmic logic currently driving these systems.

Comparing AI and Physician Diagnostic Patterns

The study evaluated 24 different AI systems, including 20 general-purpose models and four specifically designed for medical applications. Researchers provided these systems with the pre-assessment portions of 200 real-world patient encounters. Each AI was tasked with generating a comprehensive care plan, which was then compared against the actual care plans documented by the treating physicians during those visits.

To quantify the differences, the researchers priced every diagnostic order using 2026 Medicare rates. The results revealed a significant discrepancy in how often testing was deemed necessary:

  • Physician behavior: In 66% of the encounters, the treating physicians ordered no diagnostic testing at all.
  • AI behavior: On every one of those encounters where physicians ordered no testing, every AI system ordered something, typically including CMP, TSH, A1c, and lipids.

When the AI models did order tests, they typically defaulted to a standard panel of diagnostics, including comprehensive metabolic panels (CMP), thyroid-stimulating hormone (TSH) tests, A1c levels, and lipid panels.

The Implications of Algorithmic Defensive Medicine

The data suggests that modern LLMs are effectively practicing a form of digital defensive medicine. While human doctors often rely on clinical intuition to forgo testing when it is not strictly indicated, AI models appear programmed to prioritize comprehensive data gathering. Robert Wachter noted that these models seem to have mastered the art of defensive medicine—ordering tests to cover all bases—without the external pressures of malpractice risk that typically influence human clinicians.

While this approach might offer a sense of thoroughness, it carries significant economic consequences. The study indicates that the AI models ordered approximately three times more care than the human physicians, a trend that could lead to substantially higher healthcare spending if integrated into standard clinical workflows.

Ultimately, the study serves as a critical benchmark for AI alignment. While the models currently appear to prioritize patient thoroughness over revenue-saving measures, the findings raise questions about how these systems will be tuned as they become more deeply integrated into the healthcare system. Balancing the benefits of comprehensive diagnostic screening against the costs of unnecessary testing remains a primary challenge for the future of AI in medicine.