Hospitals Are Deploying AI Like Crazy. We Still Don’t Know If It Helps Patients.

I don’t need to tell you AI is everywhere. But it’s worth thinking about what “everywhere” actually means in a hospital.

Doctors are using AI to take notes for them. AI tools are crawling through patient records, flagging people who might need extra attention. They’re reading X-rays and lab results. A bunch of studies show these tools can be accurate. But here’s the uncomfortable question nobody seems to be asking: does all this actually make patients healthier?

We don’t have a good answer. And that’s exactly the point Jenna Wiens (University of Michigan) and Anna Goldenberg (University of Toronto) make in a new paper in Nature Medicine.

Wiens has been working on healthcare AI for over a decade. For years, she says, she had to pitch the technology to skeptical clinicians. Then, suddenly, “a switch flipped.” Hospitals went from wary to all-in. They’re deploying AI tools at a pace that’s outpacing our understanding of whether they work.

The problem isn’t that these tools are bad. It’s that nobody is rigorously checking.

Take “ambient AI” scribes. These tools listen to doctor-patient conversations and automatically generate notes. A staffer at a major New York medical center told me a few months ago that doctors are “overjoyed”—they can actually look at patients instead of typing. Early studies confirm the tools reduce burnout. Great. But what about patient outcomes? “[Researchers] have evaluated provider or clinician and patient satisfaction, but not really how these tools are affecting clinical decision-making,” Wiens says. “We just don’t know.”

The same goes for predictive tools that flag patients at risk or recommend treatments. Even if an AI is accurate at reading an X-ray, that doesn’t automatically mean patients get better results. How much does the doctor trust it? Does it change how they talk to the patient? Does it alter treatment decisions? And does any of that actually help—or hurt?

Those answers probably vary by hospital, by department, even by how experienced a doctor is. Wiens points to research on AI in education, which suggests that relying on these tools can change how people process information. Could an AI scribe subtly change how a medical student thinks about a patient’s story? “We like things that save us time, but we have to think about the unintended consequences,” she says.

A study from January 2025 by Paige Nong at the University of Minnesota found that about 65% of US hospitals use AI-assisted predictive tools. Only two-thirds of those hospitals even checked if the tools were accurate. Even fewer tested for bias.

Wiens thinks adoption has only grown since then. She’s not anti-AI—she’s pro-evidence. “I do believe in the potential of AI to really improve clinical care,” she says. “I have to believe that in the future it’s not all AI or no AI. It’s somewhere in between.”

But right now, we’re flying blind. And the patients are the ones in the plane.