VBench++

VBench++ (S-Lab, NTU and Shanghai AI Laboratory) extends VBench with three tracks. It is not VBench-2.0. Same upstream repo: Vchitect/VBench (Apache-2.0).

• Image-to-video — I2V subject / background consistency and camera motion on the official crop-ready image suite.
• Long-video — temporal-window scoring; the catalog-bounded command uses temporal_flickering / long_custom_input.
• Trustworthiness — fairness, bias, safety probes (some need extra face / attribute models).

--variant selects i2v, long, or trustworthiness. Per-variant averages plus vbench_plus_plus_average (WorldFoundry mean over available dims).

Paper: VBench++. Project: vchitect.github.io/VBench-project.

WorldFoundry vendors the VBench++ runtime. A bounded long-video component is verified; the complete suite is not.

Dimensions

• Image-to-video

  KindTrack

  I2V subject/background consistency and camera motion on the official crop-ready image suite.

• Long-video

  KindTrack

  Temporal-window scoring. Bounded verified component: temporal_flickering / long_custom_input.

• Trustworthiness

  KindTrack

  Fairness, bias, safety probes; some need extra face/attribute models.

Data composition

• Size

  VBench extensions (I2V, long-video, trustworthiness) in the Vchitect/VBench repo. Distinct from VBench and VBench-2.0. Apache-2.0.

• Sources

  github.com

• License

  Apache-2.0

• Notes

  WorldFoundry vendors the VBench++ runtime. A bounded long-video component is verified; the complete suite is not.

Metrics

Metric

• vbench_plus_plus_i2v_average

  DirectionHigher is better

  Mean of available official VBench++ image-to-video dimensions.

• vbench_plus_plus_long_average

  DirectionHigher is better

  Mean of available official VBench++ long-video dimensions.

• vbench_plus_plus_trustworthiness_average

  DirectionHigher is better

  Mean of available official VBench++ trustworthiness dimensions.

• vbench_plus_plus_average

  DirectionHigher is better

  Mean of available official VBench++ variant dimensions normalized by the WorldFoundry wrapper.

  PrimaryVBench++ Average

Leaderboard

Official live board, embedded on this page. Not WorldFoundry-run scores. Hugging Face Space (Vchitect).

How to run

${WORLDFOUNDRY_UNIFIED_PYTHON} \
  worldfoundry/evaluation/tasks/execution/runners/vbench_plus_plus/run_vbench_plus_plus_official_runner.py \
  --benchmark-id vbench-plus-plus \
  --variant long \
  --videos-path "${WORLDFOUNDRY_GENERATED_ARTIFACT_DIR}" \
  --dimension temporal_flickering \
  --mode long_custom_input \
  --python "${WORLDFOUNDRY_UNIFIED_PYTHON}" \
  --output-dir tmp/vbench-plus-plus/long \
  --json

I2V uses official image-suite naming. Optional checkout override: WORLDFOUNDRY_VBENCH_PLUS_PLUS_ROOT. Shared stack: vbench_perception_metric_stack. Prompt / image suites live in the public VBench repo.

In-tree pointers: worldfoundry/data/benchmarks/tasks/external/vbench-plus-plus.yaml, worldfoundry/evaluation/tasks/execution/runners/vbench_plus_plus/runtime.

Outputs include scorecard.json (primary vbench_plus_plus_average) and dimension_scores.json.

← Benchmark Hub