VideoScience-Bench

Last updated: Oct 7, 2026

On this page

Dimensions Data composition Metrics Leaderboard How to run

VideoScience-Bench (Hao AI Lab, UC San Diego) asks whether video generators understand science. Prompts describe real experiments and phenomena. A VLM judge scores five dimensions on a 1–4 rubric; the official weighted overall normalizes to the unit interval as videoscience_average:

Prompt consistency
Phenomenon congruency — does the depicted outcome match the true science?
Correct dynamism
Immutability — do invariant properties stay invariant?
Spatio-temporal coherence

Paper: VideoScience-Bench. Space: lmgame/videoscience-bench. Code: hao-ai-lab/VideoScience. Catalog release status: experimental. No Hugging Face dataset is required by the contract.

WorldFoundry normalizes official result files and can drive judge runs over an experiments CSV when a provider / model / credential is configured. No complete credentialed upstream run is claimed.

Dimensions

Prompt consistency
KindAxis
1–4 rubric.
Phenomenon congruency
KindAxis
Does the depicted outcome match the true science?
Correct dynamism
KindAxis
1–4 rubric.
Immutability
KindAxis
Do invariant properties stay invariant?
Spatio-temporal coherence
KindAxis
1–4 rubric.

Data composition

Size
UCSD Hao AI Lab science-understanding T2V bench. Five dimensions on a 1–4 rubric; official weighted overall normalizes to videoscience_average. Catalog status: experimental.
Sources
github.com
Modalities
text · video
Notes
WorldFoundry wires the VLM judge and normalizes official files. No complete credentialed upstream run is claimed.

Metrics

Metric

prompt_consistency
DirectionHigher is better
prompt_consistency normalized benchmark score.
phenomenon_congruency
DirectionHigher is better
phenomenon_congruency normalized benchmark score.
correct_dynamism
DirectionHigher is better
correct_dynamism normalized benchmark score.
immutability
DirectionHigher is better
immutability normalized benchmark score.
spatio_temporal_coherence
DirectionHigher is better
spatio_temporal_coherence normalized benchmark score.
videoscience_average
DirectionHigher is better
Official weighted overall score normalized from the 1-4 rubric to the unit interval.
PrimaryVideoScience Weighted Overall

Leaderboard

Official live board, embedded on this page. Not WorldFoundry-run scores. Hugging Face Space (lmgame).

huggingface.co · LiveOpen live board

How to run

worldfoundry-eval zoo benchmark-run \
  --benchmark-id videoscience-bench \
  --mode official-validation \
  --official-results-path <official_results.json> \
  --generated-artifact-dir <generated_videos> \
  --output-dir tmp/videoscience-bench/official-validation \
  --json

Runtime CLI takes --experiments-csv, --evaluation-results-dir, optional --reference-videos-dir, and judge flags (--judge-provider, --judge-model, --judge-max-runs, --max-frames, --judge-fps). --ready-only scores only complete artifacts. Default import: WORLDFOUNDRY_VIDEOSCIENCE_BENCH_RESULTS_PATH.

In-tree pointers: worldfoundry/data/benchmarks/tasks/external/videoscience-bench.yaml, worldfoundry/evaluation/tasks/execution/runners/videoscience_bench/runtime/videoscience_bench.

Outputs include scorecard.json (primary videoscience_average).

← Benchmark Hub