VBench-2.0

Last updated: Oct 7, 2026

On this page

Dimensions Data composition Metrics Leaderboard How to run

VBench-2.0 (S-Lab, NTU and Shanghai AI Laboratory) is the intrinsic-faithfulness successor to VBench. It is not interchangeable with the 16-dimension quality suite. 18 fine-grained abilities group into five pillars that average into vbench2_total:

Creativity — composition, diversity.
Commonsense — instance preservation, motion rationality.
Controllability — camera, landscape, plot, dynamic, interaction, motion-order.
Human fidelity — anatomy, clothes, identity.
Physics — material, mechanics, multi-view consistency, thermotics.

Paper: VBench-2.0. Project: vchitect.github.io/VBench-2.0-project. Code: Vchitect/VBench. Snapshots: Vchitect/VBench-2.0_sampled_videos, _human_annotation, _human_anomaly.

WorldFoundry vendors the VBench-2.0 runtime. The runner checks prompt-suite coverage (vbench2_dataset_manifest.json, vbench2_video_coverage.json) before dispatching evaluators. A bounded diversity component is verified; the full suite is not. Some dims need RetinaFace or other heavyweight judges.

Dimensions

Creativity
KindCategory
Composition and diversity.
Commonsense
KindCategory
Instance preservation and motion rationality.
Controllability
KindCategory
Camera, landscape, plot, dynamic, interaction, motion-order.
Human fidelity
KindCategory
Anatomy, clothes, identity.
Physics
KindCategory
Material, mechanics, multi-view consistency, thermotics.

Data composition

Size
18 fine-grained abilities in five pillars (creativity, commonsense, controllability, human fidelity, physics). Not interchangeable with VBench's 16-dimension quality suite. HF snapshots: Vchitect/VBench-2.0_*.
Sources
Vchitect/VBench-2.0_sampled_videos · Vchitect/VBench-2.0_human_annotation · Vchitect/VBench-2.0_human_anomaly · github.com
Modalities
text · generated-video
License
unknown_pending_review
Notes
WorldFoundry vendors the VBench-2.0 runtime and writes coverage manifests. A bounded diversity component is verified; the full suite is not.

Metrics

Metric

vbench2_creativity
DirectionHigher is better
Official VBench-2.0 creativity group aggregated from Composition and Diversity dimensions.
vbench2_commonsense
DirectionHigher is better
Official VBench-2.0 commonsense group aggregated from Instance Preservation and Motion Rationality.
vbench2_controllability
DirectionHigher is better
Official VBench-2.0 controllability group aggregated from camera, landscape, plot, dynamic, interaction, and motion-order dimensions.
vbench2_human_fidelity
DirectionHigher is better
Official VBench-2.0 human fidelity group aggregated from anatomy, clothes, and identity dimensions.
vbench2_physics
DirectionHigher is better
Official VBench-2.0 physics group aggregated from material, mechanics, multi-view consistency, and thermotics dimensions.
vbench2_total
DirectionHigher is better
Mean of VBench-2.0 creativity, commonsense, controllability, human fidelity, and physics group scores.
PrimaryVBench-2.0 Total

Leaderboard

Official live board, embedded on this page. Not WorldFoundry-run scores. Hugging Face Space (Vchitect, VBench-2.0 tab).

huggingface.co · LiveOpen live board

How to run

${WORLDFOUNDRY_UNIFIED_PYTHON} \
  worldfoundry/evaluation/tasks/execution/runners/vbench_2_0/run_vbench_2_0_official_runner.py \
  --benchmark-id vbench-2.0 \
  --variant vbench2 \
  --dimension Motion_Order_Understanding \
  --videos-path "${WORLDFOUNDRY_GENERATED_ARTIFACT_DIR}" \
  --vbench2-dataset-root "${WORLDFOUNDRY_VBENCH2_DATASET_ROOT}" \
  --output-dir tmp/vbench-2.0/dimension \
  --json

Env: WORLDFOUNDRY_VBENCH2_DATASET_ROOT, WORLDFOUNDRY_VBENCH2_RESULTS_PATH, WORLDFOUNDRY_VBENCH2_ROOT, WORLDFOUNDRY_HF_CACHE_DIR. Shared stack: vbench_perception_metric_stack.

In-tree pointers: worldfoundry/data/benchmarks/tasks/external/vbench-2.0.yaml, worldfoundry/evaluation/tasks/execution/runners/vbench_2_0/runtime.

Outputs include scorecard.json (primary vbench2_total), dimension_scores.json, and coverage manifests.

← Benchmark Hub