Genie 4 Gives DeepMind's World Model a Soundtrack, Talking Characters and a Gemini Planner

Earlier interactive world models could show a scene and let a user walk through it. Google DeepMind’s newest one also lets the scene make noise and talk back. Genie 4, announced on October 27, 2026 in a post by Raia Hadsell, Diego Rivas and Hang Qi, is described as “natively multimodal.” Music, ambient sound and interactive dialogue are generated together with the world itself, in real time, as a user moves through it with keyboard controls and steers it with text prompts.

DeepMind calls Genie 4 “a major research milestone” but also “an early, experimental model.” It’s available only as a research preview for trusted testers. It’s “coming soon” to Google AI Ultra subscribers in Project Genie and to early-access partners of the Gemini API and Google AI Studio, and there’s a waitlist through Project Genie. DeepMind hasn’t published pricing, resolution or session-length limits.

Built on Gemini models

Genie 4 leans on the Gemini family in two ways. DeepMind says Gemini Omni “anchors the simulations in real-world appearance and behavior,” while other Gemini models supply the reasoning that keeps a simulation coherent from moment to moment. When a user types an instruction mid-session, such as changing the weather, moving the camera or adding a person or object, Gemini rewrites the prompt to fit the current scene. Between inputs, it “monitors and steers the scene to keep the world progressing cohesively.”

For longer events, DeepMind puts Gemini 3.8 Flash in the loop as what it calls a guiding agent. While Genie 4 renders, the Flash model watches the output, breaks a goal into steps, checks incoming frames to confirm each step worked, and re-plans if the world changes or the user intervenes. DeepMind says this gives “reliable causality, procedural behaviors, and multi-step reasoning.”

DeepMind also shows a robotics experiment. Gemini 4 Argon breaks a task into steps that are fed to Genie 4 as text captions while it renders. The demonstration has Apptronik’s Apollo humanoid playing checkers inside a simulated scene. DeepMind presents this as a path toward simulating many robot scenarios in imagined worlds before trying them on real hardware.

Sound, physics and people

The post groups the advances under four headings: real-time shaping of the simulation, realism in physics and human motion, sound, and Gemini-driven events. On realism, DeepMind says Genie 4 simulates aerodynamics, collisions and other physical behavior. It singles out people as “the most dramatic leap,” with characters that move naturally and can be steered in real time. The demo prompts include a watermelon on a stone plinth, a maple seed falling into grass and a crowded 1980s Japanese railway platform.

DeepMind calls Genie 4 its first real-time world model to generate audio natively. Every world carries its own ambient sound, and the company says “the true frontier” is speech. Characters hold conversations and react to the user’s inputs or to events in the scene.

Who has been testing it

DeepMind says it has worked with trusted testers across creative industries, journalism, education, gaming and engineering:

• Filmmakers and creative technologists are using it as a “draft mode” for previsualization, running several takes of a scene and changing camera, lighting, dialogue and action on the fly.
• Photojournalists are testing how a single archival photograph can become a grounded simulation of that place, and how short-lived events such as a flash flood or wildfire unfold.
• Game developers are going from an idea to an explorable prototype before writing engine code.
• Enterprise partners are evaluating it for quick “zero-to-one” scenario generation, alongside DeepMind’s own robotics research.

Limits DeepMind lists itself

The company is direct about the weaknesses. Visual details and object permanence “can still shift or drift over time,” especially in longer sessions, after several complex prompt edits or while moving around a scene. Complex physical interactions, real-time audio accuracy, legible text, spatial accuracy and long-term consistency are all listed as open research problems.

On safety, DeepMind says Genie 4 went through internal evaluations to block harmful content and follows the same policies as its other models. All generated video and audio carries SynthID, Google’s imperceptible watermark. DeepMind says it deployed to trusted testers first so it could gather feedback before a wider release.

DeepMind frames the longer-term goal as “hybrid world models.” Language and code models handle the structure and the narrative timeline, while a pixel-level simulator like Genie renders how the world looks, sounds and reacts.