Netflix Introduces VOID AI Model for Editing Video Scenes Without Reshooting

by Admin
Netflix Introduces AI Model VOID

Netflix has released a video object removal model called VOID, developed by researchers at Netflix and Sofia University. The model is available on Hugging Face for public use and is not limited to Netflix productions.

VOID stands for Video Object and Interaction Deletion. Unlike typical video inpainting tools that remove objects and fill the gap with static background content, VOID also predicts how remaining objects in a scene would physically behave after the removed element is gone.

What the VOID Video Object Removal Model Does

Given a video of two vehicles colliding, VOID can remove one vehicle and generate footage showing the remaining vehicle continuing down the road, with post-impact debris, smoke, and fire replaced with an undisturbed road surface.

In another example from the paper, removing a person jumping into a pool results in a video where the pool surface appears untouched, with no splash in the water or on the ground nearby. The model takes both the video and a language description of the object to remove as input, making it a vision-language system rather than purely visual.

How VOID Compares to Other Tools and Where You Can Use It

Netflix researchers compared VOID to several existing tools, including Runway, Generative Omnimatte, DiffuEraser, ROSE, MiniMax-Remover, and ProPainter. In a survey with 25 participants across multiple scenarios, VOID was preferred 64.8 percent of the time, with Runway coming in second at 18.4 percent.

The authors mention in the preprint that “VOID excels at modeling complex dynamics which can follow on from object removal.” These are the researchers’ own descriptions, and the paper has not yet undergone peer review.

VOID is available on Hugging Face. Netflix has not announced any plans to incorporate the model into existing products or pipelines. The preprint paper is authored by Saman Motamed, William Harvey, Benjamin Klein, Zhuoning Yuan, and Ta-Ying Cheng from Netflix, along with Luc Van Gool from Sofia University.

Add Ghacks as a preferred source on Google

Related Posts

Leave a Comment