Environments
Choose the dependency stack for each model family, benchmark runtime, and validation stage.
On this page
Introduction
An environment is the dependency and launch context expected by a model or benchmark: Python packages, CUDA/Torch bucket, optional system tools, checkpoint layout, device assumptions, and command shape.
The repository still uses runtime_profile as a metadata field because it points to a concrete environment record in worldfoundry/data/models/runtime/profiles/<model-id>.yaml. In reader-facing docs, this page calls the same concept an environment.
WorldFoundry uses conda as the single supported environment path. The documented workflow creates one named conda environment for the main framework and separate conda environments only when an upstream simulator or official runtime requires isolation.
- Start with
bash scripts/setup/bootstrap_worldfoundry.shfor model inference. - Use
bash scripts/setup/model_env_install.sh --listto inspect every model-to-env mapping, or--model <model-id>when you need the exact env for one model. - Stay in the unified env when the resolved profile is
worldfoundry-unified-<tier>. - Switch to a dedicated env only when
runtime/environments/<category>/<model-id>.yamlrecords an ABI or simulator constraint. - Use benchmark-specific scripts only for official metrics, reference metrics, or physics stacks.
Python install surfaces
pyproject.toml exposes two intentionally different dependency surfaces:
- Optional dependencies (extras) are the public, wheel-visible install surface for users and contributors. Install the smallest extra that matches the feature you need, for example
python -m pip install -e ".[ui]". - Dependency groups are source-checkout tooling and resolver buckets. They are not wheel extras and are not selected with
.[name]. With a PEP 735-capable pip, use a command such aspython -m pip install --group lint; environment tooling may consume the same groups directly.
all is a curated general-runtime bundle, not the literal union of every extra. It deliberately excludes development tools, proprietary NVIDIA VFX bindings, training stacks, and model-specific ABI islands. In the catalog below, “In all” means that every package name declared by that extra is represented in all; install the named extra as well when its tighter version constraints are part of the runtime contract.
Extra
apiHosted model API clients and image/request helpers.
In allYes
Install
.[api]apple_piApple-PI evaluation runtime, judge clients, and media helpers.
In allYes
Install
.[apple_pi]hfHugging Face and ModelScope asset download support.
In allYes
Install
.[hf]videoBroad video, media, data, and 3D runtime dependencies.
In allNo
Install
.[video]vipeViPE geometry/depth runtime and visualization stack.
In allNo
Install
.[vipe]molmoact2MolmoAct2 model serving stack with its narrow Transformers contract.
In allNo
Install
.[molmoact2]molmobot_pi0Pinned MolmoBot Pi0 policy runtime.
In allNo
Install
.[molmobot_pi0]metricsGeneral evaluation metrics and dataset I/O.
In allNo
Install
.[metrics]distribution_metricsPerceptual and distribution metric models.
In allNo
Install
.[distribution_metrics]dvltDVLT 3D/visualization runtime.
In allNo
Install
.[dvlt]uiComplete Studio/browser UI, including realtime transport dependencies.
In allYes
Install
.[ui]studio_realtimeMinimal WebRTC/realtime Studio transport stack.
In allYes
Install
.[studio_realtime]rtx_postprocessProprietary NVIDIA VFX post-processing; explicit opt-in only.
In allNo
Install
.[rtx_postprocess]tuiTextual terminal UI.
In allYes
Install
.[tui]optimized_coreOptional Torch-based core acceleration dependencies.
In allYes
Install
.[optimized_core]loraLoad and merge released PEFT LoRA adapters for inference.
In allNo
Install
.[lora]profilingModel profiling helpers.
In allNo
Install
.[profiling]webMinimal FastAPI web service surface.
In allYes
Install
.[web]mcpModel Context Protocol server/client support.
In allYes
Install
.[mcp]worldscoreWorldScore evaluation runtime.
In allYes
Install
.[worldscore]studio_pointcloudPoint-cloud Studio visualization.
In allYes
Install
.[studio_pointcloud]studio_rerunRerun-based Studio visualization.
In allYes
Install
.[studio_rerun]studio_geometryGeometry and Open3D Studio visualization.
In allYes
Install
.[studio_geometry]studio_plotPlotting and color utilities for Studio.
In allNo
Install
.[studio_plot]devBuild, pre-commit, and repository-pinned lint tooling.
In allNo
Install
.[dev]testPublic CPU inference and packaging tests; no Torch or model downloads required.
In allNo
Install
.[test]eval_coreOptional CPU evaluation dependencies.
In allNo
Install
.[eval_core]allCurated general runtime bundle described above.
Install
.[all]
The source-only dependency groups are narrower resolver building blocks:
lint— Repository-pinned pre-commit and Ruff toolchain.flash_attn_fa25— Flash-attention bucket for FA 2.5-compatible environments.flash_attn_fa28— Flash-attention bucket for FA 2.8-compatible environments.runtime_clip_bundle— CLIP-oriented runtime compatibility bundle.runtime_timm_bundle— Timm/Torch runtime compatibility bundle.three_d_core— Shared point-cloud and 3D geometry packages.transformers_high— Current Transformers/Diffusers compatibility bucket.transformers_low— Legacy or model-constrained Transformers compatibility bucket.
These groups select Python requirements only. CUDA/Torch wheel selection, native extension builds, and isolated model environments still go through the setup scripts and runtime profiles documented below.
Clone-to-run setup scripts
Run these scripts from the repository root. They are part of the public release surface, not local machine notes.
scripts/setup/bootstrap_worldfoundry.sh— First command after cloning the repo. Creates/verifies the unified env, writestmp/worldfoundry_unified_env.sh, optionally installs model-specific envs, optionally prepares model assets, and prints the Workspace launch command.scripts/setup/unified_install.sh— Main GPU runtime only. Createsworldfoundry-unified-<cuda-tier>and installsrequirements/worldfoundry-unified.txt, PyTorch CUDA wheels, optional flash-attn, and import/CUDA checks.scripts/setup/model_env_install.sh— Per-model environment resolution. Readsworldfoundry/data/models/runtime/environments/**; routes compatible profiles back to the unified env and creates dedicated envs only for hard ABI conflicts.scripts/inference/prepare_model_infer.sh— Checkpoint/Hugging Face preparation per model. Defaults to a plan/check-local flow. Add--downloadonly after reviewing license, gated access, and storage requirements.scripts/workspace/run_workspace.sh— Start the browser Workspace from an activated runtime. KeepsPYTHONPATH=.,WORLDFOUNDRY_WORKSPACE_MAX_JOBS, andWORLDFOUNDRY_CKPT_DIRexplicit.
The unified installer selects a Torch range that exists on the resolved wheel
index: cu121 uses Torch 2.4–2.5, cu124 uses 2.4–2.6, and cu128 uses 2.7 or
newer within the repository upper bound. After the CUDA-index install, the
exact Torch/Torchvision/Torchaudio versions constrain the remaining PyPI pass
so indirect dependencies cannot replace the selected CUDA build. On hosts
without nvidia-smi, bootstrap explicitly uses the cu128 wheel index while
enabling --allow-no-cuda and skipping flash-attn; direct installer calls with
--cuda auto instead fail fast and ask for an explicit tier. If another
selected extra requires a newer Torch than that tier publishes, dependency
resolution fails explicitly instead of falling back to a PyPI CPU/different
CUDA wheel.
Recommended fresh-clone path:
git clone https://github.com/OpenEnvision/WorldFoundry.git
cd WorldFoundry
bash scripts/setup/bootstrap_worldfoundry.sh
source tmp/worldfoundry_unified_env.sh
conda activate "${WORLDFOUNDRY_UNIFIED_ENV_PREFIX}"Environment prefixes are machine-local artifacts. If an existing checkout reports a missing python3.x, missing native module, or
libssl/libcrypto error, recreate the environment or run the public verifier:
bash scripts/setup/bootstrap_worldfoundry.sh --verify-onlyReview a model before downloading checkpoints:
bash scripts/setup/bootstrap_worldfoundry.sh \
--with-model matrix-game-2 \
--prepare-model matrix-game-2Download public assets only when ready:
export HF_TOKEN=<your-token-if-needed>
bash scripts/setup/bootstrap_worldfoundry.sh \
--with-model matrix-game-2 \
--prepare-model matrix-game-2 \
--download-model-assets--download-model-assets forwards to each model's preparation script. It does
not bypass Hugging Face gates; users must accept model terms upstream and
provide HF_TOKEN or HUGGING_FACE_HUB_TOKEN themselves.
For a multi-model workstation, install the unified env once and then verify exceptions explicitly:
bash scripts/setup/bootstrap_worldfoundry.sh \
--with-model lingbot-world \
--with-model octo \
--prepare-model lingbot-world \
--prepare-model octoStart the Workspace after setup:
source tmp/worldfoundry_unified_env.sh
conda activate "${WORLDFOUNDRY_UNIFIED_ENV_PREFIX}"
PYTHONPATH=. WORLDFOUNDRY_WORKSPACE_MAX_JOBS=8 \
python -m worldfoundry.studio.serving.workspace --host 127.0.0.1 --port 7870Use explicit roots when datasets or checkpoints should live on shared storage:
bash scripts/setup/bootstrap_worldfoundry.sh \
--home /path/to/worldfoundry-home \
--data-root /path/to/worldfoundry-data \
--model-root /path/to/worldfoundry-modelsRuntime variables
bootstrap_worldfoundry.sh and unified_install.sh write the same sourceable env
file. Keep this file local to the machine; it contains paths but no secrets.
Variable
WORLDFOUNDRY_HOMEWritten by setup yes
Root for local runtime state.
WORLDFOUNDRY_REPO_ROOTWritten by setup yes
Absolute path to the checked-out WorldFoundry repository.
WORLDFOUNDRY_CACHE_DIRWritten by setup yes
General cache root.
WORLDFOUNDRY_DATA_DIRWritten by setup yes
Dataset and benchmark data root.
WORLDFOUNDRY_MODEL_DIRWritten by setup yes
Model asset root.
WORLDFOUNDRY_CKPT_DIRWritten by setup yes
Compatibility checkpoint root for non-HF layouts and soft links.
WORLDFOUNDRY_ARTIFACT_DIRWritten by setup yes
Generated outputs and validation artifacts.
WORLDFOUNDRY_CONDA_ENVS_ROOTWritten by setup yes
Parent directory for unified and dedicated conda envs.
WORLDFOUNDRY_UNIFIED_ENV_PREFIXWritten by setup yes
Absolute path of the main conda env.
WORLDFOUNDRY_CUDA_PROFILEWritten by setup yes
Resolved CUDA wheel tier, normally
cu128on CUDA 12.8+ hosts.HF_HOME,HF_HUB_CACHE,HF_DATASETS_CACHEWritten by setup yes
Hugging Face cache roots when using a WorldFoundry-managed cache.
HF_TOKEN,HUGGING_FACE_HUB_TOKENWritten by setup no
Optional user-provided credentials for gated assets. Never commit these.
Prefer native Hugging Face loading (from_pretrained, snapshot_download, and
standard HF cache variables) for public checkpoints. Use WORLDFOUNDRY_CKPT_DIR
for upstream runtimes that require a fixed directory layout or for local soft
links that preserve official paths without duplicating large files.
Benchmark dataset layouts
worldfoundry-eval dataset locate <org/name> resolves an existing dataset
without network access. It checks the per-dataset environment override first,
then local asset/data manifests, a direct data root, and finally Hugging Face
cache snapshots. Direct roots accept org/name, org--name, and nested
hfd_datasets layouts; hub caches use the standard
datasets--org--name/snapshots/<revision> layout. Use
worldfoundry-eval dataset plan <org/name> [...] to print deduplicated download
commands without executing them.
Compiler caches
worldfoundry.core.execution.compile_cache configures persistent torch.compile caches
for Inductor, Triton, the CUDA driver JIT, and torch C++/CUDA extensions. The
compatibility import worldfoundry.runtime.compile_cache exposes the same API.
The root is WORLDFOUNDRY_COMPILE_CACHE_DIR when set, otherwise
${WORLDFOUNDRY_CACHE_DIR}/compile. Generated-code caches are partitioned by a
Torch/Triton/cuDNN/Python/accelerator fingerprint. Explicit
TORCHINDUCTOR_CACHE_DIR, TRITON_CACHE_DIR, TORCH_EXTENSIONS_DIR, and
CUDA_CACHE_PATH values take precedence when writable; an unwritable cache
falls back to a process-local temporary directory with a warning.
On a multi-node cluster, use node-local storage rather than a shared NFS or project mount. The fingerprint separates incompatible software and accelerator stacks, but it does not isolate concurrent writers on different nodes:
export WORLDFOUNDRY_COMPILE_CACHE_DIR=/local-ssd/${USER}/worldfoundry-compileAutotune defaults are applied with setdefault, so values exported before
launch win. WorldFoundry defaults to
TORCHINDUCTOR_AUTOTUNE_LOCAL_CACHE=1 and
TORCHINDUCTOR_AUTOTUNE_REMOTE_CACHE=0; opt in to a remote autotune cache only
when the cluster provides a compatible service.
Logging
The CLI flags and environment variables below feed the same opt-in logging
pipeline, worldfoundry.core.observability.logging_setup.configure_logging. Explicit CLI
flags take precedence over inherited environment values.
Setting
- Level
Console and file threshold; the CLI accepts
DEBUG,INFO,WARNING,ERROR, orCRITICAL.CLI
--log-levelEnvironment
WORLDFOUNDRY_LOG_LEVEL - File sink
Optional rotating file path. Output-owning CLI commands create a per-run
events.jsonlwhen neither is set.CLI
--log-fileEnvironment
WORLDFOUNDRY_LOG_FILE - JSON Lines
Makes the file sink JSON Lines; console output remains text.
CLI
--log-jsonEnvironment
WORLDFOUNDRY_LOG_JSON - Child context
JSON object containing structured correlation fields such as
run_id,job_id,benchmark_id, andmodel_id.Environment
WORLDFOUNDRY_LOG_CONTEXT
Framework-owned workers receive structured context through
log_context_environment(). WORLDFOUNDRY_RUN_ID is also exported by
output-owning CLI operations for operator visibility, but it is not a
replacement for WORLDFOUNDRY_LOG_CONTEXT.
Structured log messages and fields redact sensitive key names, bearer tokens, common OpenAI/Hugging Face/GitHub/Slack/AWS token formats, and compact JWTs. Raw stdout/stderr capture remains fidelity-preserving and may use a separate line or argument scrubber; do not assume every captured byte passed through the structured logger. Loguru keeps native time/count retention semantics, while the stdlib fallback interprets retention as a rotated backup-file count.
Dedicated model environment exceptions
Start from worldfoundry-unified-<cuda-tier> for every model. Create a
dedicated environment only for the profiles below, where the upstream runtime
has a real ABI or framework-family conflict with the unified CUDA stack.
Install one exception with:
bash scripts/setup/model_env_install.sh --model <model-id>For model setup, prefer scripts/inference/prepare_model_infer.sh; it installs
the resolved environment first and then checks or downloads assets. Use
scripts/setup/model_env_install.sh directly only when you want to prepare the
conda env without touching checkpoints.
Visual & world12
ac3dac3d
bash scripts/inference/prepare_model_infer.sh ac3d --downloadNotesOfficial runtime pins Python 3.10, PyTorch 2.4, and CUDA 12.4. Runtime code is in tree under worldfoundry/synthesis/visual_generation/ac3d/ac3d_runtime; only checkpoints/assets are external.
cosmos3worldfoundry-cosmos3-cu128
bash scripts/inference/prepare_model_infer.sh cosmos3 --downloadNotesCosmos3 uses the native diffusion runner, checkpoint-shaped transformer, Wan VAE38, AVAE decoder, and core runtime policies. Guardrails remain an optional orchestration layer; Diffusers is not an inference backend.
gen3cgen3c
bash scripts/inference/prepare_model_infer.sh gen3c --downloadNotesInference-only Cosmos Predict1 stack with Python 3.10, CUDA 12.4, torch 2.6, Transformer Engine, and MoGe. Setup applies the Transformer Engine link patch from released packages; it does not clone or build the training-only Apex repository.
lyra-1lyra
bash scripts/inference/prepare_model_infer.sh lyra-1 --downloadNotesUses the same inference-only Cosmos Predict1 / Transformer Engine family as GEN3C.
magi-1magi-official-cu124
bash scripts/inference/prepare_model_infer.sh magi-1 --downloadNotesOfficial stack is Python 3.10.12 with PyTorch 2.4 / CUDA 12.4. Keep flash-attn and flashinfer in this env because their wheels are ABI-sensitive.
warp-as-historywarp-as-history
bash scripts/inference/prepare_model_infer.sh warp-as-history --downloadNotesPins Python 3.10, torch 2.5.0+cu121, diffusers 0.36, transformers 4.51, and peft 0.17.
hunyuanvideo-t2vhunyuanvideo-i2vhunyuanvideo-1.5-t2vhunyuanvideo-1.5-i2vworldfoundry-unified-cu128
bash scripts/inference/prepare_model_infer.sh hunyuanvideo-t2v --downloadNotesAll four IDs use the shared native diffusion runner and role-based Hunyuan components. The former Diffusers/xFuser CLI environment is no longer an inference backend.
kairos-sensenovakairos-sensenova
bash scripts/inference/prepare_model_infer.sh kairos-sensenova --downloadNotesUses the official torch/CUDA 12.6-class stack and native attention packages such as SageAttention. Use this env when unified CUDA does not provide the Kairos attention stack.
lingbot-worldlingbot-world-actlingbot-world
bash scripts/inference/prepare_model_infer.sh lingbot-world --downloadNotesBase-Cam and the independently cataloged public Base-Act checkpoint share the in-tree runtime. Parity requires exact torch and flash-attn pins; the unified env produced invalid near-black act2cam output in upstream-style tests.
wonderworldwonderworld
bash scripts/inference/prepare_model_infer.sh wonderworld --downloadNotesInference imports PyTorch3D CUDA extensions. Build or install PyTorch3D wheels compatible with this env before running the official path.
wonderjourneywonderworld
bash scripts/inference/prepare_model_infer.sh wonderjourney --downloadNotesShares the WonderWorld PyTorch3D extension environment and uses in-tree WonderJourney runtime code.
hunyuan-game-craftHYGameCraft
bash scripts/inference/prepare_model_infer.sh hunyuan-game-craft --downloadNotesOfficial inference relies on HYGameCraft plus torchrun sequence parallelism. The dedicated env includes the CUDA 12.4 toolkit and builds the pinned flash-attn 2.6.3 PyPI release without cloning its Git repository. Set cuda_visible_devices / torchrun_nproc_per_node explicitly on shared GPU machines.
Embodied & JAX4
octoworldfoundry-octo-jax
bash scripts/inference/prepare_model_infer.sh octo --downloadNotesJAX CUDA 11.8, Flax 0.7.5, TensorFlow, dlimp, and Octo tokenizer assets are isolated from the PyTorch CUDA env.
lapaworldfoundry-lapa-jax-cu118
bash scripts/inference/prepare_model_infer.sh lapa --downloadNotesUses the official Python 3.10, JAX 0.4.25 CUDA 11/cuDNN 8.6, Flax 0.8.2, Tux 0.0.2, and Transformers 4.40 inference stack. It stays isolated because Transformers 5 removed the Flax generation modules required by LAPA.
openvla-oftworldfoundry-vla-openvla-compat-cu121
bash scripts/inference/prepare_model_infer.sh openvla-oft --downloadNotesPrismatic/OpenVLA-compatible stack with older transformers/timm/tokenizers. Use this env for LIBERO/ALOHA OpenVLA-OFT inference.
cogactworldfoundry-vla-openvla-compat-cu121
bash scripts/inference/prepare_model_infer.sh cogact --downloadNotesShares the OpenVLA compatibility env and reuses a staged OpenVLA Llama tokenizer for inference-only loading. A token is only needed when no compatible local tokenizer or complete Llama-2 mirror exists.
Prepare-only3
lingbot-valingbot-va
bash scripts/inference/prepare_model_infer.sh lingbot-va --downloadNotesPrepare-only profile for the official Python 3.10, torch 2.9, CUDA 12.6, flash-attn stack; stage the LingBot-VA base/posttrain checkpoints before rollout.
being-h05beingh
bash scripts/inference/prepare_model_infer.sh being-h05 --downloadNotesPrepare-only profile. Stage all Being-H0.5 checkpoints; full get_action parity needs the upstream policy stack and more headroom than a single 80GB GPU profile.
dreamdojodreamdojo
bash scripts/inference/prepare_model_infer.sh dreamdojo --downloadNotesPrepare-only profile for the official torchcodec/CUDA 12.8 setup on H100-class hosts. Stage the GR-1 eval dataset before enabling real command templates.
EvalCrafter is the only benchmark with a WorldFoundry-managed dedicated metric profile. Other benchmark-specific native stacks are listed below and are not claimed as configured until their own preflight passes.
Unified environment with extra runtime requirements
The models below should still use worldfoundry-unified-<cuda-tier>, but their
docs should call out extra local assets, compiled wheels, or external services.
These are not reasons to clone an external repository; the code path is in-tree.
- Unified-env WRBench camera/world integrations (EasyAnimate, Hunyuan WorldPlay, HunyuanWorld-Voyager, InSpatio-World, Matrix-Game 2/3, ReCamMaster, SANA-WM, ATI, HyDRA, Spatia, minWM, MagicWorld, LiveWorld, VerseCrafter, VideoX-Fun camera variants, and the HY-World renderer) — Extra requirement: Hugging Face checkpoints declared by each profile; Spatia's default MapAnything checkpoint is non-commercial research only. Notes: These resolve to
worldfoundry-unified-<cuda-tier>. Every model keeps an independent in-tree inference binding. Reusable Wan2.1, MapAnything, UniCeption, Uni3C, Depth Pro, STream3R, SAM3 video segmentation, Qwen3-VL entity extraction, and point-cloud projection components live underbase_models; runtime config lives underworldfoundry/data. Runtime setup never clones source repositories. AC3D, Cosmos3, GEN3C, Hunyuan GameCraft, and both LingBot routes use the dedicated environments listed above. - SkyReels V3 — Extra requirement: CUDA compiler, flash-attn, and torchao for the official 8-GPU USP path. Notes: Use
unified_install.shwithout--skip-flash-attnor build flash-attn throughscripts/setup/install_flash_attn.sh. - Matrix-Game-3 — Extra requirement: Flash-attn. Notes: FA3 may be requested by config; FA2 fallback is acceptable on A100 validation hosts.
- HunyuanWorld-Mirror — Extra requirement: CUDA/PyTorch-matched
gsplatwheel. Notes: Remains a unified-env model when the wheel matches the active torch stack. - pixelSplat — Extra requirement: Local Gaussian rasterization extension and
re10k.ckpt. Notes: The model wrapper is in tree; only the extension/checkpoint are local prerequisites. - LingBot-Map — Extra requirement: Optional FlashInfer. Notes: The wrapper defaults to SDPA, so FlashInfer is not mandatory for baseline inference.
- OpenPI / MME-VLA — Extra requirement: JAX policy dependencies,
gsutilfor some OpenPI checkpoint shards, and optional OpenPI-style websocket or robot-simulator services for full policy rollout. Notes: WorldFoundry keeps code in tree; these requirements are runtime services or checkpoint-transfer tools, not external repo clones. LAPA uses the isolated environment listed above. - Solaris — Extra requirement: JAX/Orbax checkpoint loading and official HF evaluation fixtures. Notes: Uses the unified env in WorldFoundry; document the dataset/checkpoint layout rather than a separate conda env.
Benchmark metric environments
Most benchmark official validation runs and official-result normalizers run in the unified environment. Full official metric suites can require a separate conda profile when their dependency stack would disturb model inference. Install those profiles through the same public setup entrypoint:
The complete per-benchmark mapping for all 42 Video Generation and World Model
entries is in the benchmark runtime environment matrix.
That matrix also records external API services, native extensions, checkpoint
stacks, and the currently verified scope; needs_new_env: false alone is not a
full-suite readiness claim.
bash scripts/setup/model_env_install.sh --model evalcrafterEvalCrafter's supported three-metric CLIP path and final_result.txt import run
in worldfoundry-unified-<tier>. The separate
worldfoundry-evalcrafter-metrics profile mirrors the official revision's
Python 3.9.17, pip 23.2.1, Torch 2.0.1, torchvision 0.15.2, TensorFlow 2.12.0,
PaddlePaddle-GPU 2.5.1, MMCV 2.0.1, and MMAction2 1.1.0 package pins. It exists
to prepare the upstream dependency surface, not to imply that WorldFoundry
orchestrates the full scorer.
- EvalCrafter — Environment boundary: Unified env for the three official CLIP-derived metrics and result import; official legacy environment for preparing the external full stack. Verified scope: The bounded metric path is verified. Environment preparation is documented, but complete 17-metric recomputation and leaderboard output remain unsupported.
- iWorld-Bench — Environment boundary: Unified env, including the pinned in-tree ViPE native sources and CUDA build.
WORLDFOUNDRY_IWORLD_BENCH_PYTHONis only an optional override for a non-standard ABI-compatible Python. Verified scope: One real 16-frame video passed the officialmemory_abilitycomponent path. The bundled three-prompt coverage and all nine component metrics were not run, so full-suite and leaderboard validity remain false. - VBench / VBench-2.0 / VBench++ — Environment boundary: Unified env plus dimension-specific checkpoints and optional perception packages. Verified scope: Bounded official dimensions only; each selected dimension must pass preflight.
- T2V-SafetyBench — Environment boundary: Unified client environment plus a user-provided judge API endpoint/key. Verified scope: API failure is fail-closed; no judge API result is claimed from local fixture tests.
A separate environment is never inferred from a successful import. The catalog and runtime profile must name its Python executable or setup command, assets, and verified scope. If no reproducible environment recipe exists, the affected metric remains blocked rather than being reported through a fallback score.
The in-tree code keeps reusable foundation models under worldfoundry/base_models.
These components are available for future full-suite work; their presence does
not mean the bounded EvalCrafter runner currently executes them:
- DeepFace runtime (
worldfoundry/base_models/perception_core/face/deepface) — Available for future celebrity identity integration; the supported CLIP subset does not execute it. - DOVER runtime (
worldfoundry/base_models/perception_core/video_quality/dover) — Available for future video-quality integration; the supported CLIP subset does not execute it. - VideoMAE action recipe (
worldfoundry/base_models/perception_core/action_recognition/videomae_mmaction) — Only the EvalCrafter-used VideoMAEv2 config and Kinetics-400 label map are vendored. MMAction2 is installed as an environment dependency, not copied as framework source.
For the supported subset, stage CLIP ViT-B/32 under the standard checkpoint root or set its explicit override:
export WORLDFOUNDRY_CLIP_VIT_B32_MODEL_DIR="${WORLDFOUNDRY_HFD_ROOT}/openai--clip-vit-base-patch32"See the EvalCrafter benchmark page
for the no-proxy download and smoke command. The complete upstream suite also
expects direct 0000.mp4 through 0699.mp4 files, but WorldFoundry does not
currently claim that full path.
WorldFoundry container
The supported image is the repository-root docker/Dockerfile, built through docker/build_with_docker.sh. Dockerfiles under worldfoundry/base_models/perception_core/detection/yolo_world/mmyolo/docker/ (and the MMYOLO CircleCI Dockerfile) are upstream snapshot artifacts. WorldFoundry does not build or publish those images; they keep PyTorch 1.8/1.9-era CUDA bases and are not maintained against the current runtime. If you need an MMYOLO-specific image, build it from current upstream rather than this vendored copy.
Embodied simulator environments
The host setup is Docker plus the unified WorldFoundry environment:
bash scripts/setup/bootstrap_worldfoundry.sh
source tmp/worldfoundry_unified_env.sh
docker pull ghcr.io/allenai/vla-evaluation-harness/robotwin:latest
worldfoundry-eval embodied run \
--config worldfoundry/data/benchmarks/eval_configs/embodied/robotwin/eval.yaml \
--pull-dockerBenchmark configs that have a docker.image entry, either directly or through
worldfoundry/data/benchmarks/runtime_profiles/official/<benchmark>.yaml, are
launched through worldfoundry/evaluation/tasks/embodied/docker_runner.py.
Inside the container, simulator dependencies are provided by the image. On the
host, only mount datasets/checkpoints/results and provide the policy server
endpoint.
Troubleshooting
When working with complex setups involving CUDA, simulators, or external evaluators, you may encounter environment issues:
1. CUDA mismatch
Symptom: CUDA error: no kernel image is available for execution on the device or PyTorch cannot find a GPU.
Solution: Ensure your NVIDIA driver is up-to-date and supports the CUDA version installed by Conda. If you are on an older driver, you can force the installation script to use an older CUDA wheel bucket:
bash scripts/setup/unified_install.sh --cuda cu1212. Missing system dependencies (e.g., for OpenGL/3D)
Symptom: Errors relating to libGL.so.1, EGL, or headless rendering failures when running embodied simulators or 3D benchmarks.
Solution: Install the required system-level graphics libraries. On Ubuntu/Debian:
sudo apt-get update && sudo apt-get install -y libgl1-mesa-glx libglib2.0-03. Flash Attention installation fails
Symptom: The unified install script hangs or fails during flash-attn compilation.
Solution: flash-attn is extremely sensitive to mismatched CUDA compiler (nvcc) versions. Ensure your nvcc --version matches the PyTorch CUDA version. Alternatively, you can disable flash-attn installation by passing --skip-flash-attn (if supported by your environment) or installing a pre-compiled wheel from the official release page.





