WorldFoundry

WorldFoundry is open-source infrastructure for world intelligence. World intelligence is not a single generation task: a system has to infer structured state from observations, predict how that state evolves, express the consequences in task-specific outputs, and use those predictions during interaction. Video generators, neural scene representations, 4D models, and embodied policies each cover part of this problem.

Discovery, runtime and asset preparation, inference, artifact inspection, and benchmark evidence share one workflow across those families — without forcing them into one representation, task, or metric.

A first run

After the Quickstart environment is sourced, generate one inspectable artifact for matrix-game-2:

bash scripts/inference/prepare_model_infer.sh matrix-game-2 --download
worldfoundry-eval run matrix-game-2-universal-action-validation \
  --pipeline.task-profile official-universal-image

That run justifies one claim: this configuration produced an inspectable artifact. It does not mean a benchmark completed. Everyday commands use worldfoundry-eval run. Layers and inventory are in Design.

Operational definition

Visual plausibility measures observation quality. It is not sufficient evidence of world modeling. A photorealistic video can still lose object identity, violate dynamics, or ignore an intervention.

We therefore treat a generative world model as a learned, history- and control-conditioned latent dynamical system. It infers and maintains a persistent environmental state, evolves that state under endogenous dynamics and external interventions, and assimilates new observations so later predictions remain accountable to what just happened.

Three coupled operations make the definition usable:

1. State inference — estimate a task-relevant latent state from the interaction history.
2. Forward simulation — predict how that state evolves under autonomous dynamics, physical constraints, and actions.
3. Interactive response — produce distinct, coherent futures under alternative interventions, then fold observed outcomes back into the maintained state.

A compact, modality-agnostic form is

Here $h_t$ is the observation–control history, $c_t$ is an action or intervention (including a null control), $z_t$ is the inferred environment state, and $\hat{o}_{t+1}$ is a visual, geometric, or task-specific output. The formulation is functional rather than architectural: $z_t$ may be recurrent, object-centric, geometric, token-based, or otherwise structured.

Capability progression

WorldFoundry does not assign every model a single discrete level. It records a capability profile: which functions a system actually supports, and whether those functions are coupled.

Perceptual grounding supplies observational evidence. The world-model spine is the coupling that matters: persistent state, intervention-conditioned dynamics, and feedback-driven updating. Controllable transformation can sit beside any of those stages — editing a couch color or deleting a shelf does not, by itself, prove that the system can roll the scene forward or close an interaction loop.

Foundation models and generative world models

The taxonomy is functional, not a ranking of visual quality.

Foundation generative models learn visual, geometric, or spatiotemporal priors for synthesis, reconstruction, and controllable manipulation. They may be excellent at appearance, short-horizon coherence, or selective edits. They do not integrate state, dynamics, and feedback into a complete world-modeling loop.

Generative world models do integrate that loop. Intervention effects persist across subsequent transitions, and new observations revise the state that governs later predictions. Higher fidelity, broader prompting, or larger scale does not by itself promote a system from the first class to the second.

That is why WorldFoundry separates catalog inclusion, runnable inference, inspectable artifacts, and benchmark-eligible evidence. A convincing demo can still be only perceptual grounding. The model catalog and benchmark hub show current coverage and readiness. Catalog inclusion is not a runnable runtime. A demo is not benchmark evidence. An ingested score is not official leaderboard parity. Those states stay separate, with provenance.

Why a shared workflow

WorldFoundry is most useful when work spans more than one model, benchmark, or execution surface. The technical report calls this commensurability without homogenization: heterogeneous systems stay inspectable under shared operational and evidential contracts, without being forced into a single representation, task, or metric.

That is why the shared workflow exists:

• One operational vocabulary. Stable model and benchmark IDs, plus request, result, artifact, manifest, and scorecard records, replace repository nicknames and handwritten paths. You can ask whether a model was only cataloged or actually executed, which checkpoint and runtime ran, whether samples were covered, and whether a score was normalized, officially computed, or validated for a leaderboard.
• Shared glue. Environment bootstrap, checkpoint lookup, input shaping, output naming, job records, previews, and metric aggregation stay shared. Model-specific code lives in pipelines and operators; benchmark-specific code stays in runners and metrics.
• Reusable artifacts. Generated output is not trapped in a UI session. The same artifacts can be inspected, rescored, converted, compared, or audited later without loading the model again.
• Evidence beyond a command. The same command can change meaning after a checkpoint, package, dataset, or evaluator changes. Ledgers, identities, blockers, and scorecards record both what ran and what the result is allowed to claim.
• Multiple interfaces, one backend. TUI, CLI, Studio, Python, and MCP reuse the same catalogs and contracts. The interface can change; the run identity and durable outputs do not.

This fits teams that maintain or compare several world-model families, separate generation from evaluation, reuse expensive artifacts, or need stable machine-readable discovery for people and agents. The more models, benchmarks, machines, and reviewers you have, the more those contracts repay their setup cost.

Acknowledgements