SenseTime Open-Sources Unified Vision Model SenseNova-Vision, Covering Four Vision Tasks with a Single Model

On July 13, SenseTime officially released and fully open-sourced the SenseNova-Vision understanding-generation unified vision large model. For the first time, this model natively covers classic vision tasks such as structured visual understanding, dense geometric prediction, image segmentation, and multi-view 3D geometry within a single shared representation space. With a single model, it matches or even surpasses specialized expert models on multiple authoritative benchmarks.

Technical Breakthrough

SenseNova-Vision breaks the traditional "one task, one model" architecture by unifying all vision tasks as multimodal generation problems. The model specifies tasks through natural language instructions and optional visual prompts, outputting native text, images, or mixed text-image results without requiring task-specific prediction heads or additional architectural branches. This design brings bidirectional benefits: SenseTime's accumulated visual data enhances the base model's visual understanding, while the reasoning capabilities of large language models enable vision tasks to be integrated, even allowing new tasks to be defined directly via language.

Performance

On multiple authoritative benchmarks, SenseNova-Vision leads in four core vision areas with a single model:

• Structured Visual Understanding: Outperforms similar general-purpose models in tasks such as object detection, referring expression comprehension, OCR, and keypoint localization, especially excelling in dense small-object detection and long-tail category recognition.
• Dense Geometric Prediction: Achieves accuracy comparable to geometry-specialized models in depth estimation and surface normal estimation, maintaining high stability across indoor and outdoor scenes.
• Segmentation Capabilities: Covers general segmentation, reasoning segmentation, and interactive segmentation, with impressive performance in reasoning and conversational segmentation.
• Multi-view 3D Geometry: A single model can perform high-quality multi-view point cloud reconstruction and camera pose estimation.

Generalization Ability

The model demonstrates remarkable generalization in extreme scenarios:

• Zero-shot Generalization: On game scenes not seen during training (e.g., Black Myth: Wukong), it can simultaneously perform surface normal estimation, instance segmentation, and keypoint detection.
• Ultra-dense Object Segmentation: For highly overlapping scenes like schools of fish or flocks of sheep, it can precisely isolate each individual.
• Seeing Through Reflections: Automatically filters mirror reflections to accurately estimate real-world orientation and depth.
• Overcoming Visual Illusions: In illusion images such as forced perspective, it correctly separates foreground and background.

Open Source and Ecosystem

SenseTime also open-sourced the SenseNova-Vision Corpus-50M, a visual instruction dataset containing 50 million high-quality samples. The model code, weights, and dataset are available on GitHub, Hugging Face, and ModelScope. SenseTime stated that this model significantly lowers the barrier to visual AI applications, as developers no longer need to maintain multiple model systems for different tasks—a single model can cover high-frequency vision needs.