WorldFoundry
English中文

WorldArena

Edit this pageGitHubMarkdown

On this page
DimensionsData compositionMetricsLeaderboardHow to run

WorldArena (Tsinghua FIB Lab) evaluates embodied world models on two axes. Primary ewm_score aggregates the embodied-world-model surface. This catalog id is worldarena.

  • Perceptual — visual quality, motion quality, content consistency, physics adherence, 3D accuracy, controllability, human quality.
  • Functional — utility as a data engine (data_engine_success), policy evaluator (policy_evaluator_correlation), and action planner (action_planner_success).

Paper: WorldArena. Project: world-arena.ai. Code: tsinghua-fib-lab/WorldArena.

WorldFoundry integrates the official video-quality metric runtime (runners/worldarena/runtime/video_quality) and can score one --dimension at a time (catalog-bounded example: action_following). One real prepared-data PSNR component is verified. Full Track 1 and the embodied tracks are not. Functional-utility metrics need embodied task evidence outside the video-quality runtime.

Official paper teaser: sixteen video-perception metrics across six sub-dimensions.
Official paper teaser: sixteen video-perception metrics across six sub-dimensions.

Dimensions

  • Perceptual quality
    KindAxis

    Visual, motion, consistency, physics, 3D, controllability, human quality.

  • Functional utility
    KindAxis

    Data engine, policy evaluator, action planner.

Data composition

  • Size

    Tsinghua FIB Lab embodied world-model bench. Primary ewm_score.

  • Sources

    github.com

  • Modalities

    text · video · action

  • Notes

    One real prepared-data PSNR component is verified. Full Track 1 and embodied tracks are not.

Metrics

Metric

  • visual_quality
    DirectionHigher is better

    visual_quality normalized benchmark score.

  • motion_quality
    DirectionHigher is better

    motion_quality normalized benchmark score.

  • content_consistency
    DirectionHigher is better

    content_consistency normalized benchmark score.

  • physics_adherence
    DirectionHigher is better

    physics_adherence normalized benchmark score.

  • three_d_accuracy
    DirectionHigher is better

    three_d_accuracy normalized benchmark score.

  • controllability
    DirectionHigher is better

    controllability normalized benchmark score.

  • data_engine_success
    DirectionHigher is better

    data_engine_success normalized benchmark score.

  • policy_evaluator_correlation
    DirectionHigher is better

    policy_evaluator_correlation normalized benchmark score.

  • action_planner_success
    DirectionHigher is better

    action_planner_success normalized benchmark score.

  • human_quality
    DirectionHigher is better

    human_quality normalized benchmark score.

  • ewm_score
    DirectionHigher is better

    ewm_score normalized benchmark score.

    PrimaryEwm Score

Leaderboard

Official live board, embedded on this page. Not WorldFoundry-run scores. Hugging Face Space (WorldArena).

huggingface.co · LiveOpen live board

How to run

${WORLDFOUNDRY_UNIFIED_PYTHON} \
  worldfoundry/evaluation/tasks/execution/runners/worldarena/run_worldarena_official_runner.py \
  --run-official \
  --dimension action_following \
  --config-path <worldarena_config.yaml> \
  --generated-artifact-dir <generated_videos> \
  --output-dir tmp/worldarena/official \
  --json

Ground-truth comparisons take --gt-data-dir / --action-gt-data-dir. Weights resolve from --ckpt-root. --official-results-path imports an existing result file. Env: WORLDFOUNDRY_WORLDARENA_RESULTS_PATH, WORLDFOUNDRY_WORLDARENA_ROOT. No Hugging Face dataset is required by the contract.

In-tree pointers: worldfoundry/data/benchmarks/tasks/external/worldarena.yaml, worldfoundry/evaluation/tasks/execution/runners/worldarena.

Outputs include scorecard.json (primary ewm_score).

← Benchmark Hub

Related

OverviewBenchmark HubMetricsScorers & qualityEnvironmentCLILocal assets