Google releases Genie 4, the most powerful world model, which achieves simultaneous audio and video generation for the first time

On October 27, local time, Google DeepMind launched a research preview version of the world model Genie 4. This model can generate a highly realistic and interactive virtual world, achieve simultaneous audio and video generation for the first time, support free editing, and maintain a continuous picture for up to two minutes. ‌‌

The official Google DeepMind account stated, “Genie 4 is our most realistic and interactive world model to date.”

Create an interactive virtual world in one sentence

Unlike AI tools that only generate videos, Genie 4 generates an interactive virtual environment that can be explored in real time. After users create a scene through text prompts, they can use the keyboard to control character movement, adjust the camera, and continue to enter instructions during the simulation to change the weather, camera angle, or add new characters and objects in real time.

Google Research Director Tejas Kulkarni said that the future direction is a "hybrid world model" - Gemini is responsible for planning at the code and text levels, and Genie receives instructions in real time and simulates reality at different levels of abstraction.

Technology upgrades focus on four dimensions. Native audio generation is a capability achieved for the first time in the Genie series: while the model generates each frame, it simultaneously generates matching environmental sounds, background music and interactive dialogue, so characters can talk in real time about events happening in the scene.

In terms of physics and character simulation, Genie 4’s accuracy in complex interactions such as aerodynamics, collision and environmental resistance has been significantly improved, and the movements and facial expressions of digital characters have become more natural.

The real-time scene director allows users to "stop and change the scene" in the middle of the simulation using natural language. Gemini 3.8 Flash serves as an active agent in the loop, responsible for tracking picture frames, dismantling targets, verifying the success of actions, and adjusting plans instantly when the scene changes.

In addition, Genie 4 can handle coherent simulation of multi-step complex events, allowing developers to observe how AI agents respond to complex scenarios in a virtual environment, and then transfer strategies to reality.

Application of aim robot training and film and television prototypes

Google clearly lists robot training, game development, interactive learning and film and television production as the main application directions of Genie 4.

In the field of robotics, an experiment associated with Gemini Robotics showed that the Gemini 4 Argon model breaks down the task into multiple steps, and then simulates it in real time with Genie 4, providing a feasible path for testing scenarios in a virtual environment before the physical robot performs the task.

Film and television production has become a new direction of concern for early testers. Google DeepMind engineer Diego Rivas pointed out that real-time generation capabilities are most valuable in the creative conception and storyboarding stages to "allow creators to maintain a smooth creative state."

Wharton Business School professor Ethan Mollick generated a "neo-gothic brutalist city inhabited by robots and horses" in one minute with just one prompt during the experience, and then added dragons and NPC characters in real time.

However, Google repeatedly emphasized in its release that Genie 4 is still in the research preview stage and has not yet been opened to the public. The model is currently only available to "trusted testers," with plans to make it available to Google AI Ultra subscribers and early access partners of Gemini API and AI Studio through Project Genie in the coming weeks.

The team admitted that the model has clear limitations: as the simulation time increases, scene details and object stability are prone to drift, and audio accuracy and the readability of text in the virtual world have not yet been fully resolved.

In addition, all videos and audio generated by Genie 4 will come with SynthID invisible watermarks, and Google said the model has undergone rigorous security assessments to prevent the generation of harmful content.

The release of Genie 4 comes at a time when competition and consolidation in the world model field are heating up.

Just a few months ago, AI video startup Runway launched its second world model, GWM Worlds 2, which also focuses on interactive response and audio generation.

Industry consolidation is also accelerating. World Labs founded by Li Feifei was acquired by AMD, Amazon is negotiating to acquire Decart, and Rivian acquired Third Dimension to bring simulation capabilities into autonomous driving testing.

On a more macro level, Google puts its world model on the critical path to AGI. The official blog writes: “Understanding and simulating the world is a critical step in building AGI.”

Whether Genie 4 can move from a research preview to a scalable product depends on its stability in real workflows and the speed of developer adoption, but the direction is already clear: AI is moving from generating content to generating a world that can be trained, tried, and created by humans and machines together.