AI Scams Are Getting Smarter, But Healthcare AI Has a Blind Spot

I’ve been covering AI long enough to know that every shiny new capability comes with a dark mirror. The latest one? AI-powered scams that are faster, cheaper, and harder to spot than anything we’ve seen before.

When ChatGPT dropped in late 2022, it didn’t take long for cybercriminals to realize they could use it to write convincing phishing emails. That was just the beginning. Now they’re using AI for everything from automated vulnerability scans to hyperrealistic deepfakes. The volume of attacks is overwhelming organizations, and it’s only going to get worse as more criminals adopt these tools and the tech itself improves.

This isn’t speculation. MIT Technology Review just named “supercharged scams” one of the 10 Things That Matter in AI Right Now, and I can’t argue with that. The speed at which attackers are iterating is genuinely concerning. We’re in a new era, and it’s not a comfortable one.

But here’s the thing that really bothers me: while we’re scrambling to defend against AI-driven attacks, we’re also rushing to deploy AI in healthcare without knowing if it actually helps patients.

Doctors are using AI for notetaking, scanning patient records, flagging people who might need specific treatments, even interpreting X-rays and exam results. Multiple studies show these tools can deliver accurate results. But accuracy isn’t the same as better outcomes. Does using AI actually improve patient health? We don’t have a good answer yet.

That’s a massive blind spot. We’re putting AI into clinical workflows based on the assumption that faster, more accurate analysis will translate to better care. But assumptions in healthcare can be dangerous. What if the AI catches something the doctor would have caught anyway, but adds a layer of false confidence? What if it misses something subtle and the doctor trusts it too much?

Jessica Hamzelou flagged this exact problem in The Checkup newsletter, and it’s worth sitting with. We have all this hype about AI transforming medicine, but the evidence for actual patient benefit is surprisingly thin. That’s not to say it won’t work—just that we’re running before we can walk.

Meanwhile, DeepSeek just dropped preview versions of V4, claiming it’s the most powerful open-source platform out there, rivaling closed models from OpenAI and DeepMind. It’s also adapted for Huawei chips, which is an interesting geopolitical move. The AI arms race shows no signs of slowing.

And on the regulatory front, more countries are restricting children’s social media access—Norway and the Philippines joining the list. There’s also a push in the US to get AI out of schools. The backlash is real, and it’s growing.

So here’s where we are: AI is making scams more dangerous, healthcare deployments are lacking outcome data, new models keep dropping, and regulators are scrambling to catch up. It’s a lot to process in one week.

I don’t have tidy answers. But I do think we need to slow down on the healthcare AI rollout until we can measure what actually matters—patient outcomes, not just accuracy metrics. And on the scam front, we need to assume the worst and build defenses accordingly. The era of AI-driven threats isn’t coming. It’s here.