In its first major contribution to the public AI ecosystem, Netflix’s research team has open-sourced VOID (Video Object and Interaction Deletion), a specialized AI framework that doesn’t just erase objects from video—it rewrites the physics they left behind.
Developed in collaboration with researchers from Sofia University (INSAIT), VOID aims to solve one of the “dirty secrets” of video editing: making a scene look physically plausible after a major element has been removed.
1. What Makes VOID Different?
Most traditional video inpainting tools (like those in Runway or Adobe) focus on “pixel-filling”—cleaning up the visual hole left behind by removing an object. VOID, however, is a causal reasoning engine for video.
- Interaction Deletion: If you remove a person who was holding a guitar, VOID doesn’t just leave the guitar floating in mid-air. It predicts the “counterfactual” physics, showing the guitar falling naturally to the ground.
- Complex Dynamics: In a car crash sequence, VOID can remove one vehicle and “re-render” the scene so the remaining car continues down the road undisturbed, removing post-impact debris, smoke, and fire that no longer have a physical cause.
- Secondary Effects: Beyond shadows and reflections, it handles “downstream” interactions, such as water ripples that should no longer exist if a person jumping into a pool is deleted.
2. The Technical Stack: The “Quadmask”
The core innovation of VOID is its use of a 4-value “quadmask” rather than a simple binary (on/off) mask.
| Mask Value | Function |
| Primary | Identifies the main object to be removed. |
| Overlap | Detects regions where the object interacts with the environment. |
| Affected | Marks areas where physics must be “rewritten” (e.g., falling objects). |
| Background | Regions that must remain strictly untouched. |
The Architecture:
- Base Model: Built on Alibaba PAI’s CogVideoX-5b, a 3D Transformer-based video diffusion model.
- Reasoning Layer: Uses Google’s Gemini 3 Pro to analyze the scene logic and identify which surrounding regions need physical adjustment.
- Segmentation: Leverages Meta’s SAM2 for precise object isolation.
3. Training on “Counterfactual” Data
To teach the model how the world works without certain objects, Netflix generated a unique dataset using:
- Google’s Kubric: To simulate physically correct object-to-object collisions.
- Adobe’s HUMOTO: To provide motion-capture data of human-object interactions. By comparing videos with and without specific interactions, the model learned to “hallucinate” the correct physical outcome.
4. Performance vs. Industry Standards
In a blind survey of 25 participants comparing various state-of-the-art tools, VOID was the preferred choice for its physical consistency.
| Model | Preference Rate | Key Strength |
| Netflix VOID | 64.8% | Physical causality and “next-step” logic. |
| Runway Gen-3 | 18.4% | Visual texture and style consistency. |
| ProPainter | ~8% | Speed for simple background removals. |
5. Open Source Availability
Netflix has released VOID under the Apache 2.0 license, making it free for both academic and commercial use.
- Hugging Face: The weights and model checkpoints (5B parameters) are available for download.
- GitHub: The complete training and inference code has been published for the developer community.
- Strategic Play: Analysts suggest this release is a “talent magnet” and a way for Netflix to establish its influence in the Generative AI space, potentially paving the way for dynamic ad-insertion or localized “censorship-lite” edits in international markets.
“VOID is not just about erasing; it’s about imagining the truth of a scene,” said Saman Motamed, a lead researcher at Netflix. “It’s the difference between a photo-edit and a world-edit.”
