T2VWorldBench

T2VWorldBench (San Jose State, with UC Berkeley, UW-Madison, UW, and GUET; WACV 2026) asks whether text-to-video models can use world knowledge for factual generation. Official size: 6 categories, 60 subcategories, 1,200 prompts — physics, nature, activity, culture, causality, object knowledge. Scoring mixes human evaluation and VLM judging. The paper's 10-system study found most models fail at knowledge-correct video.

The released eval.py writes quality, realism, relevance, and consistency scores on 0–5 scales plus a final score on a 0–20 scale. The six knowledge categories describe the prompt suite; the released CSV does not contain separate category scores.

Paper: T2VWorldBench. Code: magiclinux/world_knowledge. Prompt CSV is vendored; full metadata / sample videos are Drive-hosted.

WorldFoundry vendors runtime source and normalizes official result files. Full official VLM evaluation still needs the upstream LLaVA runtime and staged Drive assets.

Dimensions

• Physics knowledge

  KindCategory

  Factual physics generation.

• Nature knowledge

  KindCategory

  Nature-world knowledge.

• Activity knowledge

  KindCategory

  Human/activity knowledge.

• Culture knowledge

  KindCategory

  Cultural facts.

• Causality knowledge

  KindCategory

  Causal world knowledge.

• Object knowledge

  KindCategory

  Object-centric world knowledge.

Data composition

• Size

  WACV 2026 world-knowledge T2V bench: 6 categories, 60 subcategories, 1,200 prompts. Prompt CSV is vendored; full metadata/sample videos are Drive-hosted.

• Sources

  github.com

• Modalities

  text · video

• License

  unknown_pending_review

• Notes

  WorldFoundry normalizes official result fields. Full raw-video/VLM evaluation (LLaVA + Drive assets) is pending.

Metrics

Metric

• quality

  DirectionHigher is better

  Minimum visual quality score (0–5) in the official CSV.

• realism

  DirectionHigher is better

  Minimum realism score (0–5) in the official CSV.

• relevance

  DirectionHigher is better

  Minimum prompt relevance score (0–5) in the official CSV.

• consistency

  DirectionHigher is better

  Minimum temporal consistency score (0–5) in the official CSV.

• final

  DirectionHigher is better

  Minimum total score (0–20) in the official CSV.

  PrimaryFinal

Leaderboard

Results from the paper (not a live board; not WorldFoundry-run scores). T2VWorldBench: A Benchmark for Evaluating World Knowledge in Text-to-Video Generation (2025).

Model	Average
Wan2.1	0.68
LTX Video	0.68
Kling1.6	0.67
Dreamina	0.67
Mochi-1	0.66
Sora	0.65
Hailuo	0.63
Pika2.2	0.60

How to run

worldfoundry-eval zoo benchmark-run \
  --benchmark-id t2vworldbench \
  --mode official-validation \
  --official-results-path <model_video_assessment_scores.csv> \
  --generated-artifact-dir <generated_videos> \
  --output-dir tmp/t2vworldbench/official-validation \
  --json

Default import: WORLDFOUNDRY_T2VWORLDBENCH_RESULTS_PATH. Optional root: WORLDFOUNDRY_T2VWORLDBENCH_ROOT. In-tree prompt: worldfoundry/data/benchmarks/assets/t2vworldbench/prompt/prompt.csv.

In-tree pointers: worldfoundry/data/benchmarks/tasks/external/t2vworldbench.yaml, worldfoundry/evaluation/tasks/execution/runners/t2vworldbench/runtime/t2vworldbench.

Outputs include scorecard.json (primary final) and raw_metric_table.jsonl.

← Benchmark Hub