WorldFoundry
English中文

4DWorldBench

Edit this pageGitHubMarkdown

On this page
DimensionsData compositionMetricsLeaderboardHow to run

4DWorldBench (USTC, with ZJU and BJZGCA; CVPR 2026) scores 3D/4D world generators — Image-to-3D/4D, Video-to-4D, Text-to-3D/4D — by mapping every conditioning modality into one evaluation space. Four axes roll into four_d_worldbench_average:

  • Perceptual quality — CLIP-IQA, CLIP-aesthetic, FastVQA.
  • Condition-4D alignment — Keye-VL yes/no judges for attribute / relationship / motion / event / scene control, plus DROID-SLAM camera error (raw error is lower-better, then inverted).
  • Physical realism — LLM-assisted physics reasoning.
  • 4D consistency — DROID-SLAM reprojection, VFIMamba interpolation, motion QA, VGG style.

Paper: 4DWorldBench. Project: yeppp27.github.io/4DWorldBench.github.io. Code: yeppp27/4dworldbench_code.

WorldFoundry hosts the official runtime (run_four_d_worldbench_official_runner.py) and scores one --dimension at a time over a dataset JSON in the official schema. Official dataset download is not published; you supply the JSON plus generated videos.

Official paper figure: 4DWorldBench’s unified evaluation framework for 3D/4D world generation.
Official paper figure: 4DWorldBench’s unified evaluation framework for 3D/4D world generation.

Dimensions

  • Perceptual quality
    KindAxis

    CLIP-IQA, CLIP-aesthetic, FastVQA.

  • Condition-4D alignment
    KindAxis

    Attribute, relationship, motion, event, scene, and camera-error control.

  • Physical realism
    KindAxis

    LLM-assisted physics reasoning.

  • 4D consistency
    KindAxis

    Viewpoint, motion smoothness, motion QA, and style consistency.

Data composition

  • Size

    Caller-supplied Condition-to-4D JSON plus generated videos. Official dataset download is not published. Four axes: perceptual quality, Condition-4D alignment, physical realism, 4D consistency.

  • Sources

    github.com

  • Modalities

    text · image · video · generated-video

  • Notes

    WorldFoundry hosts the official runtime and scores one dimension at a time. It does not invent a 4DWorldBench board.

Metrics

Metric

  • perceptual_clip_iqa_metrics
    DirectionHigher is better

    Official 4DWorldBench frame-level CLIP-IQA perceptual quality score.

  • perceptual_clip_aesthetic_metrics
    DirectionHigher is better

    Official 4DWorldBench CLIP-aesthetic perceptual quality score.

  • perceptual_fastvqa
    DirectionHigher is better

    Official 4DWorldBench temporal video-quality score.

  • alignment_attribute_control
    DirectionHigher is better

    Keye-VL yes/no judge score for dynamic attribute control.

  • alignment_relationship_control
    DirectionHigher is better

    Keye-VL yes/no judge score for dynamic spatial relationship control.

  • alignment_motion_control
    DirectionHigher is better

    Keye-VL yes/no judge score for motion order alignment.

  • alignment_event_control
    DirectionHigher is better

    Keye-VL yes/no judge score for complex plot/event alignment.

  • alignment_scene_control
    DirectionHigher is better

    Keye-VL yes/no judge score for complex landscape/scene alignment.

  • alignment_camera_error_metrics
    DirectionLower is better

    DROID-SLAM camera trajectory error for camera-control alignment; lower raw error is better and normalized inversely.

  • physics_realism
    DirectionHigher is better

    Official LLM-assisted physics-reasoning score.

  • consistency_viewpoint
    DirectionLower is better

    DROID-SLAM reprojection error for multi-view consistency; lower raw error is better and normalized inversely.

  • consistency_motion_smoothness
    DirectionHigher is better

    VFIMamba interpolation consistency score.

  • consistency_motion_qa
    DirectionHigher is better

    Keye-VL yes/no judge score for motion consistency questions.

  • consistency_style
    DirectionHigher is better

    VGG Gram-matrix style consistency score.

  • perceptual_quality
    DirectionHigher is better

    Mean normalized perceptual quality component.

  • condition_4d_alignment
    DirectionHigher is better

    Mean normalized Condition-4D alignment component.

  • physical_realism_score
    DirectionHigher is better

    Normalized physical realism component.

  • four_d_consistency
    DirectionHigher is better

    Mean normalized 4D consistency component.

  • four_d_worldbench_average
    DirectionHigher is better

    Mean normalized score across available 4DWorldBench components.

    Primary4DWorldBench Average

Leaderboard

Official live board, embedded on this page. Not WorldFoundry-run scores. yeppp27.github.io/4DWorldBench.

yeppp27.github.io · LiveOpen live board

How to run

${WORLDFOUNDRY_UNIFIED_PYTHON} \
  worldfoundry/evaluation/tasks/execution/runners/four_d_worldbench/run_four_d_worldbench_official_runner.py \
  --run-official \
  --dataset-json <dataset.json> \
  --model-name <model_name> \
  --dimension perceptual_clip_iqa_metrics \
  --generated-video-dir <generated_artifacts> \
  --output-dir tmp/4dworldbench/official \
  --json

Model-backed dims resolve WORLDFOUNDRY_4DWORLDBENCH_KEYE_MODEL, WORLDFOUNDRY_4DWORLDBENCH_DROID_CKPT, and OPENAI_API_KEY. Pixel-level dims run without them. Shared stacks: keye_vl, droid_slam, vfimamba, fastvqa. Default import path: WORLDFOUNDRY_4DWORLDBENCH_RESULTS_PATH.

In-tree pointers: worldfoundry/data/benchmarks/tasks/external/4dworldbench.yaml, worldfoundry/evaluation/tasks/execution/runners/four_d_worldbench.

Outputs include scorecard.json (primary four_d_worldbench_average) and raw_metric_table.jsonl.

← Benchmark Hub

Related

OverviewBenchmark HubMetricsScorers & qualityEnvironmentCLILocal assets