Model recipes

Pick a model, inspect its runtime, and copy a command that matches your machine.

On this page
270 modelsruntime data from repository manifests

270 recipes

AnimateDiffShanghai AI Laboratory / CUHK MMLabAnimateDiff is the classic motion-module approach to text-to-video: it animates existing Stable Diffusion image models by inserting a trained temporal module, rather than training a video model from scratch.Text to videoPython 3.11 · 12.8AstraOpen-source research release (institution not recorded)Astra is an interactive world model with camera control built over a Wan2.1 backbone: from a single image it rolls out navigable world video following camera actions.Image to videoPython 3.11 · 12.8Being-H0.5BeingBeyondBeing-H0.5 is BeingBeyond's 2B-parameter cross-embodiment VLA policy.VlaPython 3.10 · CUDA 11.8+CameraCtrlThe Chinese University of Hong Kong / Shanghai AI LaboratoryCameraCtrl pioneered plug-and-play camera control for text-to-video diffusion: it trains a camera encoder over Plücker embeddings so trajectories can steer generation without retraining the base model.WorldPython 3.11 · 11.3CogVideoXZhipu AI (THUDM)CogVideoX is Zhipu AI's open video generation family, one of the most widely used open T2V/I2V baselines. This catalog entry serves three runnable variants — 2B text-to-video, 5B text-to-video, and 5B image-to-video — through in-tree runti…Text to videoPython 3.11 · 12.8Cosmos Transfer 2.5NVIDIACosmos Transfer 2.5 does controlled world-to-world translation: it takes structured control inputs (e.g. edge maps or simulation renders) and generates photoreal world video that follows them — the sim-to-real arm of the Cosmos family.Controlled videoPython 3.11 · 12.8Depth Any PanoramasInsta360 (Research Team)Depth Any Panoramas (DAP) is Insta360's metric depth estimator for panoramic images — full 360° depth from a single equirectangular input.Panoramic depth estimationPython 3.11 · 12.8Depth Anything 3 Metric PriorByteDance (Seed)Depth Anything 3 Metric Prior is the metric-depth head of ByteDance Seed's Depth Anything 3, integrated as an independent geometry prior with verified runner status. The recorded checkpoint is depth-anything/DA3METRIC-LARGE, and it is the…Monocular depth estimationPython 3.11 · 12.8Depth Anything V1TikTok / The University of Hong KongDepth Anything V1 is the foundation model for relative monocular depth that popularized large-scale unlabeled-data depth training.DepthPython 3.11 · 12.8Depth Anything V2TikTok / The University of Hong KongDepth Anything V2 is the sharper, synthetic-data-trained successor to V1, integrated here as a metric/monocular depth prior with verified runner status. The recorded checkpoint is depth-anything/Depth-Anything-V2-Large; the shared priors f…Monocular depth estimationPython 3.11 · 12.8DIAMONDUniversity of Geneva / University of Edinburgh / Microsoft ResearchDIAMOND (DIffusion As a Model Of eNvironment Dreams) trains reinforcement-learning agents entirely inside a diffusion world model — its Atari agents and playable CS:GO world model made diffusion world models famous (NeurIPS 2024).Diffusion world modelPython 3.11 · 12.8DualCamCtrlThe Hong Kong University of Science and Technology (Guangzhou) (EnVision Lab)DualCamCtrl controls video generation with a camera trajectory plus a depth image: the RGB and depth conditions together pin down scene geometry so the generated camera move stays physically grounded.WorldPython 3.11 · 12.8DUSt3RNAVER Labs EuropeDUSt3R made unconstrained two-view 3D reconstruction easy: feed it two images — no calibration, no poses — and it directly regresses aligned pointmaps.Feed forward multi view 3D reconstruction and geometry estimationPython 3.11 · 12.8FantasyWorldAlibaba (Amap, Fantasy-AMAP)FantasyWorld does geometry-consistent world modeling by unifying video and 3D prediction: from an image it produces camera-controllable world video that stays geometrically coherent. Two variants are recorded — the verified Wan2.1 pipeline…Geometry consistent world modelingPython 3.11 · 12.8FlashWorldXiamen University / TencentFlashWorld generates 3D scenes in seconds: from one image (plus optional prompt) it produces a camera-navigable 3D scene rather than a flat clip.Fast 3D scene generation from image or textPython 3.11 · 12.8GeoCalibETH Zurich (CVG)GeoCalib estimates camera calibration and the gravity direction from a single image by combining geometric optimization with deep learning.Camera calibrationPython 3.11 · 12.8GigaBrain-0GigaAIGigaBrain-0 is GigaAI's 3.5B VLA foundation policy with subtask prediction, released as GigaBrain-0-3.5B-Base and the 0.1 refresh under Apache-2.0.VlaPython 3.11 · 12.8HunyuanWorld-MirrorTencent (Hunyuan)HunyuanWorld-Mirror is Tencent's universal 3D reconstruction model: feed it images and it returns point clouds, Gaussian splats, depth, normals, and camera parameters in one pass.Three dimensionPython 3.11 · 12.8HY-WorldPlayTencent (Hunyuan)HY-WorldPlay is Tencent's real-time interactive world video model: starting from an image, it follows pose/action input to roll the world forward — the interactive arm of the Hunyuan world stack.WorldPython 3.11 · 12.8Infinite-WorldMeituan (MeiGen-AI)Infinite-World is MeiGen-AI's long-horizon world video model: from one image it navigates and extends a world over long durations.WorldPython 3.11 · 12.8InSpatio World 1.5inspatioNative inference reuses the existing causal Wan graph, UMT5 encoder, Wan VAE, causal sampler and in-tree DA3.Image to videoPython 3.11 · 12.8InSpatio-WorldInSpatio (formal institution not recorded)InSpatio-World is a 1.3B video-conditioned world generator: given a video and a target trajectory it synthesizes trajectory-conditioned novel views of the same world.Video conditioned world generationPython 3.11 · 12.8InSpatio-WorldFMInSpatioInSpatio-WorldFM is a 3D world foundation model for spatial intelligence and frame generation, released with confirmed code and weights (inspatio/WorldFM on Hugging Face, Apache-2.0).WorldPython 3.11 · 12.8Kairos SensenovaSenseTime (SenseNova)Kairos Sensenova is SenseTime's embodied world-model video generator: a 4B model covering text-, image-, and text+image-to-video with a robot-oriented distilled recipe.Text to videoPython 3.10 · 12.8LingBot-WorldRobbyant (Ant Group)LingBot-World is Robbyant's camera-controlled world video model.WorldPython 3.10 · 12.1LoGeROpen-source research release (institution not recorded)LoGeR reconstructs 3D scenes with camera trajectories and point clouds from input views, and is one of the verified 3D routes in this catalog: checkpoint-backed Studio inference produced finite poses, depth, and a 601,570-vertex point clou…3d reconstructionPython 3.11 · 12.8LongVie 2Shanghai AI Laboratory (Vchitect)LongVie 2 is the second-generation controllable long-video framework from the Vchitect team, and the verified member of the pair in this catalog: Studio's default four-GPU torchrun USP route completed a checkpoint-backed run on 2026-07-28,…Image to videoPython 3.10 · 12.8LyraNVIDIALyra is NVIDIA's generative world model family for exploration: from an image or video (Lyra-1) or from actions (Lyra-2) it produces an explorable world video.Open generative 3D and 4D world modelsPython 3.10 · 12.4Matrix-Game 2.0Skywork AI (Kunlun Tech)Matrix-Game 2.0 is Skywork's real-time-oriented interactive world model generation: an auto-regressive diffusion design that follows keyboard/mouse actions across game scenes. The in-tree runner was checked with a 15-latent-frame short run…WorldPython 3.11 · 12.8Matrix-Game 3.0Skywork AI (Kunlun Tech)Matrix-Game 3.0 is the third Skywork generation, pushing interactive world video to higher resolution with few-step inference.WorldPython 3.11 · 12.8Metric3DDJI / Zhejiang UniversityMetric3D recovers metric-scale depth from a single image by normalizing images into a canonical camera space, which lets one model generalize across cameras.Monocular depth estimationPython 3.11 · 12.8MolmoAct2Allen Institute for AI (AI2)MolmoAct2 is AI2's second-generation action reasoning model with an unusually broad Apache-2.0 checkpoint zoo: base, Think (depth-reasoning), DROID (Franka), Bimanual YAM, SO100/101, and LIBERO variants are all confirmed. WorldFoundry port…VlaPython 3.11 · 12.8MonST3RGoogle DeepMind / UC BerkeleyMonST3R extends the DUSt3R pointmap approach to dynamic scenes: it estimates per-timestep geometry directly from video, recovering shape and motion without a static-scene assumption.Geometry estimation in the presence of motionPython 3.11 · 12.8MotionCtrlTencent (ARC Lab)MotionCtrl is a unified, flexible motion controller for video generation: it controls camera motion and object motion independently in one model (SIGGRAPH 2024).Python 3.11 · 12.8NeoVerseCreateAI (formerly TuSimple)NeoVerse is a 4D world model built as a native assembly over canonical Wan components plus model-specific control and reconstruction roles: it enhances world generation with 4D-aware conditioning.4D world modeling from in the wild monocular videosPython 3.11 · 12.8OpenVLAOpen-source research release (institution not recorded)OpenVLA is the widely used open 7B vision-language-action model (MIT license): a Prismatic-based VLM fine-tuned on Open X-Embodiment robot demonstrations to predict discretized actions. Both openvla-7b and the LIBERO-10 fine-tune are confi…VlaPython 3.11 · 11.3Prior Depth AnythingOpen-source research release (institution not recorded)Prior Depth Anything is a depth-completion prior: it takes any coarse or sparse metric-depth prior plus an image and refines them into a dense, sharper metric depth map.Depth completionPython 3.11 · 12.8ReCamMasterKuaishou (Kling Team)ReCamMaster re-renders an existing video along a new camera trajectory — camera-controlled generative rendering from a single video.Camera controlled video rerenderingPython 3.11 · 12.8RoboFlamingoOpen-source research release (institution not recorded)RoboFlamingo adapts the OpenFlamingo vision-language model into a robot imitation policy: a lightweight policy head turns the VLM's visual-language features into actions.VlaPython 3.11 · Prepare-onlySCOPEOpen-source research release (institution not recorded)SCOPE is an interactive game world model over a Wan backbone: image plus per-step actions in, game-world video out (paper arXiv 2605.23345).Interactive world modelPython 3.11 · 12.8Show-oNational University of Singapore (Show Lab)Show-o (Xie et al., Show Lab at NUS with ByteDance, arXiv 2408.12528) is a single 1.3B transformer that unifies multimodal understanding and generation. It is initialized from Phi-1.5 and, unlike systems that attach a separate diffusion ex…Multimodal generationPython 3.11 · 11.3SolarisNew York University (VisionX)Solaris is a multi-agent interactive video world model from NYU's VisionX group (paper arXiv 2602.22208): multiple agents act inside one generated world.Video generationPython 3.11 · 12.8Splatt3RUniversity of Oxford (Active Vision Lab)Splatt3R predicts a 3D Gaussian splat scene directly from an uncalibrated image pair — zero shot, no camera parameters, no per-scene optimization.3d reconstructionPython 3.11 · 11.3StarVLAOpen-source research release (institution not recorded)StarVLA is a Lego-like modular VLA framework; this catalog entry is GPU-validated: the official QwenOFT predict_action ran on an 80GB CUDA GPU with the local Qwen3-VL-OFT-LIBERO-4in1 checkpoint and Qwen3-VL-4B base VLM. A WM4A world-action…VlaPython 3.11 · 12.8Step-Video-T2VStepFunStep-Video-T2V is StepFun's large open text-to-video model (30B parameters upstream, MIT weights at stepfun-ai/stepvideo-t2v). In WorldFoundry the public pipeline is assembled from native role-based components with the shared diffusion run…Text to videoPython 3.11 · 12.8VGGT-OmegaMeta AIVGGT-Omega is the extended VGGT release published under facebook/VGGT-Omega, covering 3D reconstruction, Gaussian splatting, and novel view synthesis.3d reconstructionPython 3.11 · 12.8VMemUniversity of OxfordVMem is an interactive video scene generation model with a surfel-indexed view memory: as you navigate a scene, it remembers which surfaces were seen from where, keeping new views consistent with the past.Video generationPython 3.11 · 12.8Wan2.1Alibaba (Tongyi Lab)Wan2.1 is Alibaba Tongyi Lab's open video generation family and the backbone of a large part of this catalog — many world-model and control entries build on it. Four variants are recorded here: T2V 1.3B, T2V 14B, and I2V 14B at 480p and 72…Text to videoPython 3.11 · 12.8Wan2.2Alibaba (Tongyi Lab)Wan2.2 is the next Wan generation, tracked separately from Wan2.1 because upstream versions code and checkpoints separately. The recorded runnable surface is the TI2V 5B variant — a compact text+image-to-video model. WorldFoundry wraps the…Text to videoPython 3.11 · 12.8Warp-as-HistoryOpen-source research release (institution not recorded)Warp-as-History (WAH) is a camera-controlled image-to-video method: it warps the input image along the requested camera path and treats those warps as the generation history that a video model then refines.Image to videoPython 3.10 · 12.1WorldCamKAIST / Adobe Research / MAUM AIWorldCam, from KAIST's CV Lab with Adobe Research and MAUM AI, is an interactive world model with camera-pose control for long-horizon game video.WorldPython 3.11 · 12.8WorldCrafterTencentarcReuses shared Helios transformer, scheduler, latent preparation and VGGT layers.Text to videoPython 3.11 · 12.8WorldCrafter BaseTencentarcWorldCrafter Base is a world model from Tencentarc. The catalog records it for text-to-video, image-to-video, camera-controlled-video, and world-model.Text to videoPython 3.11 · 12.8WorldCrafter FastTencentarcWorldCrafter Fast is a world model from Tencentarc. The catalog records it for text-to-video, image-to-video, camera-controlled-video, and world-model.Text to videoPython 3.11 · 12.8YUMEOpen-source research release (institution not recorded)YUME is an interactive world-video generator: from an image (and for YUME-1.5, text) it produces explorable first-person world videos that respond to keyboard-style motion input. Both recorded variants are verified in this catalog — YUME I…Image/video conditioned interactive world video generationPython 3.11 · 12.8ZeroScopeOpen-source community (cerspense)ZeroScope is a community fine-tune of ModelScopeT2V optimized for watermark-free 16:9 video, released in 576w and XL upscale checkpoints by the creator cerspense.Text to videoPython 3.11 · 12.8AC3DSnap Inc. (Snap Research)AC3D analyzes and improves camera control in video diffusion: it is a ControlNet-style plugin for CogVideoX. WorldFoundry includes the official inference adapter; checkpoint-backed inference requires the AC3D ControlNet and matching CogVid…Camera control videoPython 3.10 · 12.4Action Chunking TransformerOpen-source research release (institution not recorded)ACT (Action Chunking Transformer) is the classic imitation-learning policy behind the ALOHA line of work: instead of predicting one action at a time it predicts chunks of future actions, smoothing control and reducing compounding error. Wo…Action chunking policyPython 3.10 · CUDA 11.8+AHA-WAMOpen-source research release (institution not recorded)AHA-WAM is a world-action model for manipulation: paired video and action experts share a mixture cache so the policy benefits from predictive world modeling. The MIT-licensed SereneC/AHA-WAM-RoboTwin2.0 checkpoint (robotwin_ahawam.pt) is…VlaPython 3.11 · 12.8AlayaWorldOpen-source research release (institution not recorded)AlayaWorld is a camera-controlled interactive world model for long video: from an image it rolls out world video autoregressively with action control and DA3-backed spatial memory.Image to videoPython 3.11 · 12.8

How to read readiness

Every model carries one readiness badge. The badge is computed from independent manifest records and never rounds missing evidence up to a stronger claim.

  • Runner verified — A WorldFoundry runner-parity artifact is recorded for at least one route. Select the variant and configuration named in that evidence.
  • Integrated — A runnable pipeline is bound, but end-to-end GPU parity is still pending. Run it; treat outputs as unvalidated until parity is recorded.
  • Runtime ported — Runtime code exists in-tree; the route is still being wired up. Inspect the manifest; running may require manual steps.
  • Profile only — Only a runtime profile is recorded — no bound pipeline. Use the page for provenance and planning; no run command is shown.
  • Planned — The entry records intent and upstream provenance only. Track it; nothing is runnable yet.
  • Blocked — A recorded blocker (license, weights, upstream) prevents integration. Read the manifest notes for the specific blocker.

How to choose a model

  1. Filter by family and task. Use the category tabs (Video, World models, 3D & 4D, Embodied, Hosted API) and the search box — it matches model names, aliases, providers, and task ids.
  2. Prefer verified or integrated entries. These have a bound pipeline; everything else is reference material.
  3. Check the runtime line. Dedicated environments need their own install; unified-environment models share one conda environment. Python and CUDA tell you whether your driver stack matches before you install anything.
  4. Open the model page. Each page explains what the model is, where it comes from, its input/output contract, exact environment, weight repositories with pinned revisions, and a command builder that emits the copy-paste run command for a chosen variant.

What each model page contains

Model pages combine generated runtime records with model-specific articles. docs/fumadocs/scripts/generate-model-recipes.py refreshes recipe data and pages marked pageSource: generated; authored articles are maintained separately. Each page keeps these claims separate:

Catalog
Stable model ID, tasks, official sources (paper, project, code, weights), aliases, and integration status.
Publisher
The formal publishing institution (company first, then university or lab) with its kind. GitHub handles are never presented as brands; when the repository records no formal institution, the page says so explicitly.
Runtime profile
Input contract, artifact contract, backend stage, and runtime readiness.
Environment
Python, CUDA tier, pinned packages, validation imports, and dedicated vs. unified routing.
Pipeline binding
The runner and concrete pipeline target used by WorldFoundry, per variant.
Benchmarks
Evaluation benchmarks recommended by the manifest or referenced in its evidence notes, linked to the Benchmark Hub.
Evidence
Native-demo and WorldFoundry-runner parity; a catalog entry alone is not proof of a successful GPU run.

Unknown values remain Not recorded. Catalog manifests can carry a curated bilingual docs: block with overview, architecture, publisher attribution, and recommended benchmarks. Entries without that block use recorded catalog and runtime metadata. Usage notes and blockers live in the Runtime contract on each model page. Catalog tasks describe model capabilities; the command builder uses the CLI's accepted task-profile IDs and options. Recorded validation applies to the stated route and configuration.

Common commands

# Inspect the complete catalog manifest.
worldfoundry-eval zoo model-show \
  --model-id <model-id> \
  --include-manifest \
  --json

# Resolve checkpoints, credentials, and local asset blockers without downloading.
worldfoundry-eval zoo model-download \
  --model-id <model-id> \
  --check-local \
  --json

# Resolve and install the environment manifest for one runtime profile.
bash scripts/setup/model_env_install.sh --model <model-id>

# Download declared checkpoints for a runtime profile.
bash scripts/inference/prepare_model_infer.sh <model-id> --download

# Run a recorded task profile (each model page generates the exact flags).
worldfoundry-eval run <model-id> --pipeline.task-profile <task> --json

Where this fits in the docs

  • Run inference — the full request-file and artifact flow once you have picked a model.
  • Environment reference — how unified vs. dedicated environment routing is decided.
  • Evaluation and the Benchmark Hub — score a model's outputs; model pages link to benchmarks their manifests recommend or reference.
  • Add a model — how new catalog manifests (and their optional docs: block) are written.