Vibe Coding XR: Google’s Playful Take on AI-Powered XR Prototyping

I’ve been watching the “vibe coding” trend with a mix of amusement and genuine interest. The idea that you can just describe what you want and have an LLM spit out working code is powerful, but it’s mostly been stuck in 2D web land. XR prototyping has remained a painful mess of game engines, sensor pipelines, and fragmented SDKs.

Google Research just dropped something that might actually change that. They’re calling it Vibe Coding XR, and it’s exactly what it sounds like: you type a prompt, Gemini does the heavy lifting, and you get a physics-aware WebXR app running on Android XR headsets in under a minute.

I got my hands on the demo, and here’s the real talk.

How it actually works

The workflow is deceptively simple. You open the XR Blocks Gem in Chrome on an Android XR headset — say, the Galaxy XR — and type something like “create a beautiful dandelion.” Or you can do this from a desktop browser with a simulated reality environment, which is smart because not everyone wants to strap on a headset just to test an idea.

Gemini, using the open-source XR Blocks framework as its canvas, plans out the scene, configures perception and interaction logic, and generates a functional app. You pinch the “Enter XR” button, and there’s your dandelion, animated and ready to blow away when you interact with it. Hit share, and you get a public link.

What’s impressive is the speed. Google claims under 60 seconds from prompt to working experience. That’s faster than I can boot up Unity.

The technical bits that matter

Under the hood, this isn’t just a thin wrapper around an LLM. The team at Google Research has baked in specialized system prompts and curated code templates that teach Gemini how to structure XR-specific logic — spatial reasoning, physics, hand tracking, depth sensing. The model handles the boilerplate so you don’t have to.

The XR Blocks framework itself is open source, which is a smart move. It’s built on WebXR, so it runs in the browser without needing a native app install. That lowers the barrier to entry considerably.

One thing I appreciate: they didn’t ignore the desktop workflow. The simulated reality environment in Chrome lets you prototype and test interactions before deploying to the headset. Depth sensing and hand interactions are better on actual hardware, but for early iteration, the simulator is more than adequate.

What this means for prototyping

I’ve been through the XR prototyping cycle enough times to know that most ideas die before they reach a headset. The friction of setting up a development environment, wiring up perception pipelines, and debugging spatial interactions is real. Vibe Coding XR doesn’t solve every problem, but it does solve the “let me see if this even works” problem.

The team will be showing this at ACM CHI 2026, but you can try it now. I’d recommend starting with something simple — a floating cube, a physics ball — before trying anything complex. The model handles straightforward spatial scenarios well, but I’m curious how it handles multi-step interactions or complex animations.

My take

This is a genuine step forward for XR prototyping. It won’t replace traditional development for production apps, but for rapid exploration and validation, it’s a game changer. The open-source nature of XR Blocks means the community can build on this, and the integration with Gemini gives it a level of intelligence that earlier no-code XR tools lacked.

Is it perfect? No. The quality of the output depends heavily on the prompt quality, and complex scenarios will still require manual tweaking. But for a first iteration, Google has nailed the core experience. I’m looking forward to seeing what the community builds with this.