Vantage: Google’s New AI Experiment That Scores Your Future-Ready Skills

Google Research just dropped something interesting: a research experiment called Vantage that uses generative AI to assess what they call “future-ready” skills. Think critical thinking, collaboration, creative problem-solving — the stuff that’s supposed to matter even when AI takes over half our jobs.

They partnered with New York University to build it, and the results are surprisingly solid. The AI’s scoring was on par with human experts. That’s not nothing, especially when you consider how notoriously hard these skills are to measure.

The problem with measuring soft skills

Standardized tests are garbage for this kind of thing. You can’t gauge someone’s ability to navigate a disagreement or build on someone else’s idea with a multiple-choice question. Real human interactions would be ideal, but they’re expensive, inconsistent, and impossible to scale. How do you fairly assess conflict resolution if your group never disagrees? Or creative collaboration if everyone just nods at the first idea?

So Google’s team built a sandbox. You hop into a conversation with AI avatars — they’re working on a task together, like preparing for a debate or pitching a creative vision. The AI avatars are steered by an “Executive LLM” that watches the conversation and dynamically introduces challenges: someone pushes back on your idea, a conflict emerges, a teammate goes silent. The idea is to give you targeted opportunities to demonstrate your skills in a controlled but realistic environment.

How it actually works

It’s not just a chatbot. The Executive LLM uses a rubric to assess your responses in real time. It adapts the conversation to make sure it gathers enough data to evaluate you properly. By the end, it’s supposed to have a pretty good picture of your capabilities.

The study with NYU showed the AI’s assessments matched human expert evaluations. That’s a big deal because it means you could potentially scale this across classrooms without needing a team of trained evaluators. I’ve seen similar approaches tried before — automated essay scoring, for instance — but those always felt brittle. This feels more organic because it’s conversational.

My take

I’m cautiously optimistic. The concept is sound: if you want to teach future-ready skills, you need to measure them, and Vantage provides a way to do that at scale. But I’d like to see more transparency about the rubric and how the Executive LLM handles edge cases. Also, the fact that it’s a “research experiment” on Google Labs means it’s still early. I wouldn’t bet my curriculum on it just yet.

Still, it’s a step in the right direction. We’ve been talking about 21st-century skills for decades, but the assessment tools never caught up. Maybe generative AI can finally bridge that gap.

Vantage is available now in English on Google Labs. Sign up, play with it, and see if you think the AI gets it right.