Genie 4: Google DeepMind adds sound, speech, and text-driven control to its world model

Google DeepMind unveiled Genie 4 this Tuesday, October 27 — the new generation of its “world model”: a system that doesn’t produce a fixed video, but an environment you explore in real time that reacts to what you do. The announcement dropped on X at 5:10 PM Paris time, alongside a blog post signed by Raia Hadsell, Diego Rivas, and Hang Qi.

No one can try it freely yet: Genie 4 is a research preview reserved for select testers chosen by Google. It is slated to arrive “soon” for Google AI Ultra subscribers in the Project Genie app, and for partners with early access to the Gemini API and Google AI Studio. A waitlist is open.

What’s changing: sound and speech

The clearest new feature is native audio. DeepMind presents Genie 4 as its first real-time world model that generates sound alongside the image: wind in the trees, waves, music playing in the scene. Above all, characters can speak and respond to the user or to events, with dialogue that adapts on the fly. Google itself acknowledges that real-time audio accuracy remains a research topic.

A world you steer with text

The user moves and orients the camera with the keyboard, and modifies the scene in natural language: make it rain, move the camera, bring in a person or an object. This is where Gemini comes in. Genie 4 builds on Gemini Omni to ground the images in realistic appearance and behavior, while other Gemini models serve as the “brain”: they translate requests into actions, plan events in multiple steps, and monitor the scene to keep it coherent between instructions.

DeepMind also highlights physics (aerodynamics, collisions) and how characters move and react, which it describes as the most dramatic progress. These claims rest for now on demonstrations chosen by Google.

What it’s for

Google cites testers in film (previewing a scene, multiple live “takes” by changing the angle, lighting, or dialogue), video games (going from an idea to an explorable prototype before writing code), education, and photojournalism, where an archive photo can become a simulated environment. The other use, more strategic, is training agents and robots in varied environments: the post shows Genie 4 paired with Gemini 4 Argon to simulate an Apptronik Apollo humanoid playing checkers.

Acknowledged limitations

Google calls Genie 4 an “early and experimental” model. Visual details and object permanence can drift as the session lengthens, after several complex modifications, or when moving through the scene. Complex physical interactions, readable text, spatial accuracy, and long-term coherence remain to be improved. Google provides no pricing, no specific date, no resolution, no maximum session duration. On safety, all produced images and sounds carry the invisible SynthID watermark, and the model applies the same content rules as Google’s other models.