WorldFoundry
English中文

Runtime assembly

The base-class contracts and directory roles for pipelines, operators, synthesis, and base_models — how a model call is assembled before it reaches a GenerationResult.

Edit this pageGitHubMarkdown

On this page
What this page is forBase-class contractsDirectory rolesHow to decide where a change belongs

What this page is for

This is the directory-level and base-class map of the model-execution side. It answers which file owns input shaping, inference, geometry, and state, and what contract each implements. The narrative of how to author or migrate a model lives in Model runtime; the shared primitives those implementations reuse live in Core infrastructure.

The one chain to remember:

Call graph
Base-class assembly
EntrypointPipelineABCpipelines/pipeline_utils.py:11
shape inputs
InputBaseOperatoroperators/base_operator.py:4
predict() / from_pretrained()
InferenceBaseSynthesissynthesis/base_synthesis.py:38
Local diffusionNativeDiffusionPipeline
optional post-inference roles
Geometry3D model runtimethree_dimensions/
StateBaseMemorymemory/base.py:9

Optional geometry and multi-turn state

ArtifactGenerationResult
Every runnable method needs both a pipeline and an operator. Operators may be thin; they must exist.

Every runnable method needs both a pipeline and an operator. Operators may be thin, but they must exist.

Base-class contracts

Base class

  • PipelineABC

    The minimal public entrypoint: from_pretrained() :118, process() :173, and stream() when incremental output is supported. Owns model construction, checkpoint loading, device placement, normalized execution, and GenerationResult artifact semantics.

    First file pipelines/pipeline_utils.py:11

  • BaseOperator

    Input and interaction shaping only: interaction_template_init, get_interaction, check_interaction, process_interaction, process_perception. Must not own generation or scoring.

    First file operators/base_operator.py:4

  • BaseSynthesis

    Connects public artifact handling to either a local native runtime or a hosted API: from_pretrained :57, api_init :76, predict :93.

    First file synthesis/base_synthesis.py:38

  • ActionModelSynthesis

    Embodied-action synthesis surface (extends RuntimeProfileSynthesis); used by VLA-style action policies.

    First file synthesis/action_generation/base_action_synthesis.py:14

  • BaseMemory

    Stores and transforms prior multimodal state for streaming, long-context, or interactive flows. Defined in core, implemented per-family in synthesis/*/memory.

    First file core/memory/base.py:9

Directory roles

worldfoundry/pipelines/

One thin public wrapper per model family (pipelines/<family>/pipeline_*.py). from_pretrained() stages checkpoints or initializes API clients; process() / __call__() turns user input into artifacts; stream() is implemented only when Studio live controls or incremental output require it. 88 family directories. Hosted-provider wrappers (Wan hosted, Kling, Runway, Luma, Hailuo, Sora/Veo, World Labs) live separately from local checkpoint runtimes.

worldfoundry/operators/

Per-family *_operator.py files that own validation, image/video loading, camera/action parsing, interaction shaping, and perception preprocessing. Inspect here first when inputs fail before model execution. Shared media helpers live in operators/_media.py.

worldfoundry/synthesis/

In-tree runtime implementation only. External repos or checkpoints remain metadata until ported.

  • synthesis/visual_generation/** — visual and world-generation checkpoint calls, hosted APIs, subprocess bridges, runtime env helpers, and per-family synthesis.py (for example cameractrl/synthesis.py, motionctrl/synthesis.py, echo_memory/, openvla/).
  • synthesis/action_generation/** — embodied-action surfaces, including the OpenVLA runtime, _native_policy_runtime.py, _official_action_trace.py, and base_action_synthesis.py.

CameraCtrl and MotionCtrl do not share one synthesis package; they live in their own synthesis.py files, while shared camera-control YAML lives under data/models/runtime/configs/camera_control.

worldfoundry/base_models/

Four inference-only packages; no training, datasets, dataloaders, callbacks, losses, augmentation-only modules, or experiment scripts.

  • diffusion_model — the single canonical diffusion kernel and the only diffusion package boundary. See Native diffusion.
  • llm_mllm_core — language/multimodal-language inference.
  • perception_core — perception inference.
  • three_dimensions — 3D model code, checkpoint loading, depth/point-cloud inference, and geometry postprocessing. Each family owns its runtime beside its model implementation (depth/depth_anything/runtime_v*.py, point_clouds/<family>/runtime.py). Pipelines import these modules directly; there is no separate representation layer or marker base class.

How to decide where a change belongs

Ask where the failure happens. Inputs malformed before model execution → operators/. Native inference, checkpoint calls, hosted APIs, subprocess bridges → synthesis/. 3D/4D inference and geometric artifacts → the matching family in base_models/three_dimensions/; artifact display → studio/visualization/. Streaming or multi-turn state → core/memory plus the relevant synthesis/*/memory. Public load path, input signature, or output artifact contract → pipelines/. A reusable primitive that helps more than one family → core, never the family package.

For the directory map of every package, see Codebase map. For the full run flow past this assembly, see Workflow.