WorldScore

WorldScore (Stanford) unifies evaluation of 3D/4D scene generation and video world models. A model receives a starting scene plus a sequence of camera-specified layout instructions. Three families normalize into worldscore_average:

• Controllability — camera and object / condition control
• Quality — visual and reconstruction fidelity
• Dynamics — temporal behavior

Paper: WorldScore. Project: haoyi-duan.github.io/WorldScore. Code: haoyi-duan/WorldScore (MIT). Data: Howieeeee/WorldScore (revision 42c4e267e1cc0529af1b0284ce018e04d43906e3).

WorldFoundry vendors the core runtime and converts evaluation.json trees into scorecards. A bounded dynamic component is verified; full metric-tree execution is not. Full metrics need DROID / lietorch, GroundingDINO, SAM, optical-flow / interpolation, Mamba, and matching checkpoints.

Dimensions

• Controllability

  KindAxis

  Camera and object/condition control.

• Quality

  KindAxis

  Visual and reconstruction fidelity.

• Dynamics

  KindAxis

  Temporal behavior of camera-specified rollouts.

Data composition

• Size

  Stanford unified world-generation bench. Starting scene + camera-specified layout instructions. Dataset Howieeeee/WorldScore (MIT), revision 42c4e267e1cc0529af1b0284ce018e04d43906e3.

• Sources

  Howieeeee/WorldScore · github.com

• Modalities

  image · video · 3d · 4d

• License

  MIT

• Notes

  A bounded dynamic component is verified. Full metric-tree execution (DROID/lietorch, GroundingDINO, SAM, flow, Mamba) is not.

Metrics

Metric

• controllability

  DirectionHigher is better

  WorldScore camera and condition controllability metric family.

• quality

  DirectionHigher is better

  WorldScore visual and reconstruction quality metric family.

• dynamics

  DirectionHigher is better

  WorldScore temporal dynamics metric family.

• worldscore_average

  DirectionHigher is better

  Normalized aggregate across WorldScore metric families.

  PrimaryWorldScore Average

Leaderboard

Official live board, embedded on this page. Not WorldFoundry-run scores. Hugging Face Space (Howieeeee / authors).

How to run

hf download Howieeeee/WorldScore --repo-type dataset \
  --revision 42c4e267e1cc0529af1b0284ce018e04d43906e3 \
  --local-dir cache/worldfoundry/data/hfd_datasets/Howieeeee/WorldScore

worldfoundry-eval zoo benchmark-run \
  --benchmark-id worldscore \
  --mode official-validation \
  --official-results-path <official_results.json> \
  --generated-artifact-dir <generated_videos> \
  --output-dir tmp/worldscore/official-validation \
  --json

GPU path: --generated-root, --data-path, --num-jobs. Staging helpers: --stage-dynamic-source, --stage-sample-index, --stage-target-frames. Env: WORLDFOUNDRY_WORLDSCORE_RESULTS_PATH, WORLDFOUNDRY_WORLDSCORE_OUTPUT_DIR, WORLDFOUNDRY_WORLDSCORE_CONFIG_ROOT. Shared stack: worldscore_spatial_metric_stack.

In-tree pointers: worldfoundry/data/benchmarks/tasks/external/worldscore.yaml, worldfoundry/evaluation/tasks/execution/runners/worldscore/runtime/worldscore.

Outputs include scorecard.json (primary worldscore_average) and per_sample_metrics.jsonl.

← Benchmark Hub