WorldFoundry
English中文

World-in-World

Edit this pageGitHubMarkdown

On this page
DimensionsData compositionMetricsLeaderboardHow to run

World-in-World (Johns Hopkins, with PKU, Princeton, MIT, and Harvard; ICLR 2026 Oral) benchmarks generative world models in a closed loop that mirrors agent–environment interaction. Task success beats visual quality. A unified online planner and a standardized action API cover four environments:

  • Active Recognition
  • Active Embodied QA
  • Image-Goal Navigation (success + SPL)
  • Robotic Manipulation

The paper's findings: visual quality alone does not guarantee success (controllability matters more); action-observation post-training beats upgrading the video generator; inference-time compute helps closed-loop performance.

world_in_world_average is the mean over available closed-loop metrics.

Paper: World-in-World. Project: world-in-world.github.io. Code: World-In-World/world-in-world.

WorldFoundry is a result importer. Closed-loop environments, planners, workers, and task scorers execute upstream. Bundled assets support parsing and coverage checks only.

Official paper teaser: closed-loop world-model evaluation in World-in-World.
Official paper teaser: closed-loop world-model evaluation in World-in-World.

Dimensions

  • Active Recognition
    KindTask

    Closed-loop recognition success.

  • Active Embodied QA
    KindTask

    Answer score plus exploration SPL.

  • Image-Goal Navigation
    KindTask

    Success rate and SPL.

  • Robotic Manipulation
    KindTask

    Closed-loop manipulation success.

Data composition

  • Size

    ICLR 2026 oral closed-loop utility bench. Four environments with a unified planner and action API. Task success over visual quality.

  • Sources

    github.com

  • Modalities

    video · action · observation

  • Notes

    WorldFoundry imports task summaries only. Closed-loop runtime parity is pending.

Metrics

Metric

  • active_recognition_success_rate
    DirectionHigher is better

    active_recognition_success_rate normalized benchmark score.

  • image_goal_navigation_success_rate
    DirectionHigher is better

    image_goal_navigation_success_rate normalized benchmark score.

  • image_goal_navigation_spl
    DirectionHigher is better

    image_goal_navigation_spl normalized benchmark score.

  • active_embodied_qa_score
    DirectionHigher is better

    active_embodied_qa_score normalized benchmark score.

  • active_embodied_qa_spl
    DirectionHigher is better

    active_embodied_qa_spl normalized benchmark score.

  • robotic_manipulation_success_rate
    DirectionHigher is better

    robotic_manipulation_success_rate normalized benchmark score.

  • interaction_trace_consistency
    DirectionHigher is better

    interaction_trace_consistency normalized benchmark score.

  • world_in_world_average
    DirectionHigher is better

    world_in_world_average normalized benchmark score.

    PrimaryWorld In World Average

Leaderboard

Results from the paper (not a live board; not WorldFoundry-run scores). Paper Active Recognition SR. † = video-generation post-train. World-in-World: World Models in a Closed-Loop World (2025).

ModelActive Recognition success
WAN2.1 †post-train62.98
SVD †post-train60.62
Wan2.158.26
SVD57.71
Hunyuan57.71
SE3DS57.53
Heuristic (w/o WM)39.02

How to run

worldfoundry-eval zoo benchmark-run \
  --benchmark-id world-in-world \
  --mode official-validation \
  --official-results-path <official_results.json> \
  --generated-artifact-dir <generated_videos> \
  --output-dir tmp/world-in-world/official-validation \
  --json

Optional --task / --exp-id scope the import. Default: WORLDFOUNDRY_WORLD_IN_WORLD_RESULTS_PATH.

In-tree pointers: worldfoundry/data/benchmarks/tasks/external/world-in-world.yaml, worldfoundry/data/benchmarks/assets/world-in-world, worldfoundry/evaluation/tasks/execution/runners/world_in_world.

Outputs include scorecard.json (primary world_in_world_average).

← Benchmark Hub

Related

OverviewBenchmark HubMetricsScorers & qualityEnvironmentCLILocal assets