Architecture

WorldFoundry separates model execution from benchmark scoring, and joins them with files you can still read after the process exits.

A model writes normalized artifacts. A benchmark scores those artifacts — never the checkpoint. The runner records what actually happened. That is why a video model and a vision-language-action (VLA) policy can share the same GenerationResult rows and scorecard.json, even though inference lives in different runtimes. It is also why you can generate once, inspect the files, and score them later without loading the model again.

The picture is one run: a surface turns intent into a request, execution either generates or scores (or both), and reporting writes the evidence. Catalog metadata, the native diffusion kernel, and Studio sit beside that path. They have their own ownership rules, but they are not extra boxes on the spine.

Read this after a first successful Quickstart run, when the commands need a mental model. The Design page is the coarser four-layer map; when a change is about ownership, use the list below.

After an evaluate run finishes, the output directory is the contract. You should be able to open requests.jsonl and results.jsonl (one row per sample), metrics/summary.json, scorecard.json (eligibility; report.md is derived from it), and run_manifest.json (identity and status). If a file is missing, Workflow shows which stage should have written it.

Who owns what

Layers talk only through serializable contracts — GenerationRequest, GenerationResult, BenchmarkSpec, and the scorecard fields. They do not share globals or machine-local paths. If a fix belongs in model runtime, scoring logic should not leak into a runner. If it belongs in reporting, the code should not assume a checkpoint layout.

• Surfaces and orchestration — Commands, run modes, resume, and output locations. Not model-specific inference.
• Model runtime — Pipelines, operators, local runtimes, hosted APIs, and normalized generation results. Not benchmark scoring.
• Native diffusion — The sampling loop and local diffusion contracts under worldfoundry/base_models/diffusion_model. Not an upstream pipeline framework, and not orchestration or evaluation.
• Benchmark runtime — Manifests, datasets, evaluators, metrics, and result normalization. Not model loading or checkpoint setup.
• Catalog and readiness — Metadata, provenance, assets, environment needs, blockers, and support status. Not runtime behavior that the code cannot actually run.
• Reporting — Manifests, summaries, report.md, scorecard.json, and leaderboard eligibility. Not hiding missing assets or skipped work.
• Studio — Interactive preview on the same catalog and runtime contracts. Not core logic that only works inside the UI.

Principles that settle arguments

These are the usual forks in a review: why a catalog row can describe intent before the code lands, why official validation is not the same as a benchmark run, and why even a simple artifact check still writes a manifest.

• One public runner contract — Every model path should end as the same result shape, so a benchmark does not care which runtime wrote the file.
• Operator plus pipeline — A runnable method needs both, so inputs stay explicit and repeatable.
• Evidence over claims — A catalog entry may describe intent. Support status has to match something that actually runs.
• External code is provenance first — Official repos and checkpoints are recorded. A machine-local checkout is not a runtime dependency.
• One native diffusion kernel — Upstream trees are implementation references, not backends. A reusable sampling algorithm has one WorldFoundry copy.
• Official validation is not benchmark parity — Importing official files proves the result pipeline. Benchmark-ready still needs real data, metrics, and a scorecard.

Where to go next

Most work crosses two pages. Adding a model means Model runtime and Evaluation core. Adding a benchmark means Benchmarks and data and Workflow.

Follow a run

Change a model or a benchmark

Look something up

If you came here with a task rather than a layer:

• Run an evaluation — Quickstart, then Evaluation and the CLI reference.
• Pick a model — Supported models, then worldfoundry-eval zoo model-show and preflight runtime --profile <benchmark-id>.
• Add a model — Add a model, then Model runtime.
• Add a benchmark — Add a benchmark, then Benchmarks and data.
• Read a run's leftovers — Evaluation core, then report.md, scorecard.json, and the metrics summary from a real output directory.
• Check support status — Supported models, then zoo model-show / zoo benchmark-show.