Run inference

Run model demos, inspect generated artifacts, and validate release readiness — scoring belongs in Evaluation.

Last updated: Oct 7, 2026

On this page

Quick path Environment Reproducible model setup CLI inference Workspace inference Output contract Next steps

WorldFoundry inference covers environment setup, checkpoint staging, model launch, and artifact review. Benchmark scoring and scorecards belong in Evaluation. To pick a model ID and read readiness labels, start with Models.

Quick path

bash scripts/setup/bootstrap_worldfoundry.sh
source tmp/worldfoundry_unified_env.sh
conda activate "${WORLDFOUNDRY_UNIFIED_ENV_PREFIX}"

worldfoundry-eval run <model-id> \
  --pipeline.task-profile <task-profile> \
  --output-dir tmp/worldfoundry_infer/<model-id>

Copy <model-id> and <task-profile> from the model page. Inspect the generated artifacts under tmp/worldfoundry_infer/<model-id>. That output is inference evidence only — not benchmark or leaderboard evidence. For checkpoints and cache layout, see Local assets; for the web workspace, see Studio. Run-status labels are in Run status.

Environment

Create and activate the unified inference environment. The official first-clone command is bash scripts/setup/bootstrap_worldfoundry.sh (same as Quickstart):

bash scripts/setup/bootstrap_worldfoundry.sh
source tmp/worldfoundry_unified_env.sh
conda activate "${WORLDFOUNDRY_UNIFIED_ENV_PREFIX}"

If a model has a dedicated conda profile, install or verify it through the profile resolver instead of guessing an environment name:

bash scripts/setup/model_env_install.sh --list
bash scripts/setup/model_env_install.sh --model <model-id>

Hugging Face checkpoints should normally load through native from_pretrained or snapshot_download cache behavior.

Use WORLDFOUNDRY_CKPT_DIR for non-HF weights and compatibility aliases required by upstream-style runtimes.

If the machine already has shared checkpoint directories, link them into the supported cache layouts instead of copying weights:

bash scripts/setup/link_hf_checkpoints.sh \
  --ckpt-dir /path/to/checkpoints \
  --hfd-root /path/to/checkpoints/hfd \
  --hf-hub-cache /path/to/huggingface/hub \
  --default-world

This creates both HFD aliases such as THUDM--CogVideoX-5b-I2V and native Hugging Face cache aliases such as models--THUDM--CogVideoX-5b-I2V/snapshots/<40-hex-local-revision>, with refs/main pointing at that local revision. New users do not need this step; they can let Hugging Face download into the default HF_HOME cache normally.

The public default is still the Hugging Face repo ID. Local symlinks only simulate that cache layout for machines that already have downloaded weights.

For a single model, the release-facing helper wires those steps together:

bash scripts/inference/prepare_model_infer.sh <model-id> --verify-env-only
bash scripts/inference/prepare_model_infer.sh <model-id> --download

It resolves the conda profile, checks or downloads declared Hugging Face checkpoints through the native HF cache or documented HFD aliases, runs zoo model-download --check-local, and prints the next inference command.

Reproducible model setup

Use the same sequence for local release validation and public user reproduction:

bash scripts/setup/bootstrap_worldfoundry.sh
source tmp/worldfoundry_unified_env.sh
conda activate "${WORLDFOUNDRY_UNIFIED_ENV_PREFIX}"

bash scripts/setup/model_env_install.sh --list
bash scripts/setup/model_env_install.sh --model <model-id> --verify-only

bash scripts/inference/prepare_model_infer.sh <model-id> --download

model_env_install.sh --list is the source of truth for environment routing. Most model profiles resolve to worldfoundry-unified-<tier>. A dedicated environment is expected only when runtime/environments records a real compatibility reason, for example pinned torch/CUDA ABI, JAX/TensorFlow isolation, or simulator dependencies.

Do not copy private cache paths into public guides. Public checkpoint paths should be Hugging Face repo IDs, native HF cache locations, or documented WORLDFOUNDRY_CKPT_DIR aliases.

CLI inference

The canonical command for everyday reproducible and documented runs is worldfoundry-eval run. This is the same shape as the commands on model pages:

worldfoundry-eval run <model-id> \
  --pipeline.task-profile <task-profile> \
  --output-dir tmp/worldfoundry_infer/<model-id>

Which command. Everyday reproducible / model-page commands use worldfoundry-eval run. Use python -m worldfoundry.studio.workspace_job infer only for Workspace visual smoke. See CLI entrypoints.

Workspace visual smoke (the same job path the Studio UI submits):

python -m worldfoundry.studio.workspace_job infer \
  --model-id <model-id> \
  --prompt "a cinematic scene, high quality" \
  --output-dir tmp/worldfoundry_infer/<model-id> \
  --device cuda

For image-to-video or video-to-video models, pass the relevant input path:

python -m worldfoundry.studio.workspace_job infer \
  --model-id <model-id> \
  --input-path /path/to/input.png \
  --prompt "camera moves forward through the scene" \
  --frames 81 \
  --steps 30 \
  --seed 42 \
  --output-dir tmp/worldfoundry_infer/<model-id>

For model-specific call parameters that are not first-class flags, use --call-json and --load-json:

python -m worldfoundry.studio.workspace_job infer \
  --model-id <model-id> \
  --call-json '{"num_frames": 17, "height": 480, "width": 832}' \
  --load-json '{"torch_dtype": "bfloat16"}' \
  --output-dir tmp/worldfoundry_infer/<model-id>

Echo-Memory Context-K1

WorldFoundry integrates echo-memory-context-k1 with a fixed model recipe, checkpoint filename, and sampler layout. Structural loading covers the released 1,185-tensor DiT; fresh two-chunk CUDA validation after the native pipeline cutover, official-sample parity, and benchmark scoring remain pending. Unreleased and retracted Echo-Memory variants are outside the integration scope.

The runtime extends WorldFoundry's canonical Wan 2.1 model directly and does not invoke DiffSynth. No architecture is ever selected implicitly from a checkpoint header: the fixed model recipe is constructed first, and loading requires full key and shape coverage. The public K=1 model uses the normal pipeline interface:

from worldfoundry.pipelines.echo_memory.pipeline_echo_memory import (
    EchoMemoryContextK1Pipeline,
)

pipe = EchoMemoryContextK1Pipeline.from_pretrained(device="cuda")
result = pipe(
    "The camera moves forward through a quiet forest trail.",
    images="/path/to/anchor.png",
    camera_trajectory="z*80",
    output_path="tmp/echo_memory/context_k1.mp4",
    return_dict=True,
)
print(result["generated_video_path"])

For research extensions, keep four concerns separate:

Add an immutable EchoMemoryRecipe and EchoMemoryModelSpec for model identity and reproducible comparison.
Implement checkpoint-specific tensor math below models/networks.
Put state-dict conversion and the shared Denoiser adapter in models/denoisers, then compose the framework-owned frozen-context execution strategy.
Implement history retention/retrieval through BaseMemory (the released rollout uses EchoRolloutMemory) without changing the canonical Wan path.

New recipes also need strict checkpoint-structure invariants, a fixed public model ID, catalog/binding/profile entries, and real CUDA validation before promotion. Public usage and checkpoint identity are on the Echo-Memory model card. Training and unvalidated FSDP/USP execution are outside this infer-only surface.

Matrix-Game 3.5 native models

Matrix-Game 3.5 is registered as two immutable model recipes:

matrix-game-3.5-first-person always loads first-person.safetensors.
matrix-game-3.5-third-person always loads third-person.safetensors. Its checkpoint architecture contains four subject-reference embedding slots; the released inference profile deliberately bounds each request to two references.

Both recipes use the same native DiffusionRequest → NativeDiffusionPipeline path and reuse canonical Wan components plus Depth Anything 3. They require an anchor image, a camera NPZ with extrinsics_c2w (or extrinsics) shaped (N,4,4), and pixel intrinsics shaped (4,), (N,4), (3,3), or (N,3,3). A one-block run needs at least 86 poses and exports the anchor plus 84 generated frames at 704×1280 and 16 FPS. The released base checkpoints use 25 denoising steps by default.

Their declarative inference profiles live under worldfoundry/data/models/runtime/configs/matrix_game_3p5; the runtime package contains the product inference path and injectable memory-research factories only. Profile overrides use num_inference_blocks, num_inference_steps, guidance_scale, inference_seed, and candidates_per_query_group. The upstream validation runner, GT/candidate panels, per-section debug videos, latent dumps, online YOLO branch, and alternate VGGT registration branch are not shipped in this infer-only runtime.

from worldfoundry.pipelines.matrix_game.pipeline_matrix_game_3p5 import (
    MatrixGame35FirstPersonPipeline,
)

pipe = MatrixGame35FirstPersonPipeline.from_pretrained(device="cuda")
result = pipe(
    image_path="/path/to/anchor.png",
    camera_path="/path/to/camera_c2w.npz",
    prompt="The camera tracks right through a cinematic city street.",
    output_path="tmp/matrix_game_3p5/first_person.mp4",
    return_dict=True,
)
print(result["artifact_path"])

For the third-person recipe, use MatrixGame35ThirdPersonPipeline and pass refs=["/path/to/subject_crop.png"] when subject memory is needed. Checkpoint layout is documented in Local assets. Memory-retrieval and fusion experiments should extend build_mosaic_inference_dataset(..., dataset_cls=...) and build_mosaic_pipeline_module(..., module_cls=...) instead of copying the runtime or native diffusion infrastructure.

Cosmos 1 and 2 native models

GEN3C/Cosmos Predict1 and Cosmos Predict2 are regular native recipes, not family backends. GEN3C maps a named camera trajectory through MoGe depth and core RGBD warping before the shared denoising loop. Predict2 2B and 14B share the native Predict backbone and Wan codec implementation, then select T5-11B and Rectified Flow AB2 in their recipes.

from worldfoundry.pipelines.cosmos.pipeline_cosmos_predict2 import CosmosPredict2Pipeline
from worldfoundry.pipelines.gen3c.pipeline_gen3c import Gen3CPipeline

predict2 = CosmosPredict2Pipeline.from_pretrained(model_id="cosmos-predict2", device="cuda")
predict2(
    "The camera advances through a robotics lab.",
    image="/path/to/first_frame.png",
    output_path="tmp/cosmos/predict2.mp4",
)

gen3c = Gen3CPipeline.from_pretrained(device="cuda")
gen3c(
    images="/path/to/first_frame.png",
    trajectory="left",
    output_path="tmp/cosmos/gen3c.mp4",
)

Both wrappers invoke NativeDiffusionPipeline. Predict2 checkpoint access is gated; registry and structural validation do not imply full-resolution GPU artifact parity.

Cosmos 2.5 native models

cosmos-predict2.5-2b, cosmos-predict2.5-14b, and cosmos-transfer2.5-2b-controlled-video are native recipes in the same registry. Predict accepts text, image, or short video conditioning. Transfer's video/control_video argument is the edge-control sequence; an optional image anchors the first generated frame. Both public pipelines use 93 frames, 35 Karras flow-UniPC steps, 16 FPS, and positive guidance by default.

from worldfoundry.pipelines.cosmos.pipeline_cosmos_transfer2p5 import (
    CosmosTransfer2p5Pipeline,
)

pipe = CosmosTransfer2p5Pipeline.from_pretrained(device="cuda")
result = pipe(
    "A robot moves through a bright workshop.",
    control_video="/path/to/edge_control.mp4",
    image="/path/to/first_frame.png",
    output_path="tmp/cosmos/transfer2p5.mp4",
    return_dict=True,
)
print(result["artifact_path"])

The native path does not import Diffusers or the deleted base_models/diffusion_model/video/cosmos2p5 package. Transfer currently pins the official general/edge checkpoint; other control variants must be added as separate, checkpoint-specific recipes after validation.

HunyuanVideo native models

The original and 1.5 T2V/I2V releases are four independent recipes on the same native pipeline. Selecting a model changes checkpoint-compatible components, not the runner architecture:

from worldfoundry.pipelines.hunyuan_video.pipeline_hunyuan_video import (
    HunyuanVideo15I2VPipeline,
)

pipe = HunyuanVideo15I2VPipeline.from_pretrained(device="cuda")
result = pipe(
    "Fireworks illuminate a city street at night.",
    image="/path/to/reference.png",
    output_path="tmp/hunyuanvideo-1.5-i2v.mp4",
    return_dict=True,
)
print(result["artifact_path"])

hunyuanvideo-t2v, hunyuanvideo-i2v, hunyuanvideo-1.5-t2v, and hunyuanvideo-1.5-i2v reuse the framework loader, scheduler, offload policy, standard denoising runner, and media writer. The old upstream CLI runtime is not a backend. To run generated artifacts through an integrated benchmark, use the public CLI with the model catalog ID and a benchmark ID:

worldfoundry run --benchmark vbench \
  --model hunyuanvideo-1.5-t2v \
  --generated-artifact-dir /path/to/generated-videos \
  --mode official-run \
  --output-dir tmp/bench/hunyuanvideo-1.5-t2v

Registry and checkpoint-structure validation pass; fresh native CUDA artifact parity is still required and is not implied by the historical CLI artifacts.

The compatibility bridge also exists:

bash scripts/inference/run_infer.sh \
  --model-id <model-id> \
  --prompt "a cinematic scene, high quality" \
  --output-dir tmp/worldfoundry_infer/<model-id>

Workspace inference

For multi-model validation, run the workspace UI and submit jobs from the browser:

bash scripts/workspace/run_workspace.sh

Open:

http://127.0.0.1:7870/

The workspace is the preferred surface for checking release readiness. Use it to inspect model-specific input forms, official-demo defaults, generated files, gallery previews, and visual regressions before recording model readiness.

Per-model viewer routing and the Visualizers tab are documented in the Studio guide.

Create Job fields, runtime checks, default values, and optional persisted defaults are documented in the Studio guide.

Output contract

Each inference run should produce a manifest-like JSON record and one or more artifacts:

Generated video/image/3D/action artifact — Primary demo output to inspect visually or structurally.
Job manifest or run record — Model ID, variant, backend, parameters, paths, timings, status, and errors.
Logs — Runtime import, checkpoint, CUDA, and model-specific call details.
Gallery/files entry in Studio — Human review surface for release validation.

A video file existing is not success on its own. Check that the input contract, prompt/camera/action controls, frame count, resolution, and artifact content match the intended official demo or model profile.