DeepMind Proposes GenCeption: Turning Video Generation Models into General-Purpose Vision Systems

Google DeepMind's latest paper, "Video Generation Models are General-Purpose Vision Learners," introduces GenCeption, which repurposes pre-trained text-to-video diffusion models into general-purpose video understanding systems capable of tasks such as depth estimation, surface normal prediction, segmentation, camera pose estimation, and 3D keypoint prediction. This work extends the idea of using image generators as general vision learners, with Kaiming He contributing to the research.

Core Method: A Paradigm Shift from Generation to Understanding

GenCeption directly reuses large-scale pre-trained text-to-video diffusion models (e.g., WAN 2.1), converting their spatiotemporal priors into visual understanding capabilities. While traditional diffusion models require multi-step denoising generation, GenCeption replaces multi-step diffusion with a single feedforward pass: it takes a noise-free video latent representation, fixes the diffusion timestep t=0, and produces outputs in just one forward pass. By changing the text instruction, the same model seamlessly switches tasks—for example, the instruction "output depth" generates a depth map, while "output 3D keypoints" predicts human pose.

Unified Multi-Task Architecture and Synthetic Data

GenCeption categorizes visual tasks into two types: dense tasks (depth, normals, segmentation, etc.) encode results into RGB space, while sparse tasks (2D/3D keypoints) use learnable tokens with an MLP decoder. All tasks are trained with a unified L2 loss, with task differences reflected only in data format, not model architecture. To address multi-task annotation alignment, the research team used 800 RenderPeople human assets and 200 actions to generate 7,500 synthetic human videos, simultaneously obtaining naturally aligned annotations for depth, normals, segmentation, DensePose, 2D/3D keypoints, and camera pose.

Performance and Data Efficiency

GenCeption approaches or exceeds specialized models such as Depth Anything V3, SAM 3, and D4RT on multiple benchmarks. The performance gap between the specialized version (single-task training) and the general version (multi-task joint training) is small, indicating no significant capability loss in the unified model. Under the same post-training data, the generative pre-trained backbone WAN 2.1 outperforms V-JEPA and VideoMAE V2 on depth estimation, with performance consistently improving as parameters scale from 1.3B to 14B. GenCeption uses only about 1.23 million frames of post-training data, while Depth Anything V3, D4RT, and VGGT-Ω use approximately 200 million, 86 million, and 600 million frames respectively—achieving 7x to 500x higher data efficiency.

Generalization and Significance

The model is primarily post-trained on synthetic human videos but can directly process real videos and generalize to unseen categories such as multi-person scenes, animals, and robots. The authors attribute this generalization to the broad world knowledge acquired during large-scale pre-training of the video generation model, with post-training only providing output format interfaces. This suggests that once a powerful video generation foundation model is available, the amount of specialized annotation data required for downstream vision tasks can be drastically reduced—especially valuable for annotation-expensive domains.