InSpatio World 1.5

Official source · Weights · Examples and input format

Use model ID inspatio-world-1p5. The original inspatio-world entry keeps its existing checkpoint and video pipeline.

The native implementation reuses WorldFoundry's causal Wan2.1 backbone, UMT5 encoder, Wan VAE, block sampler and DA3 base model. It does not need a separately installed InSpatio checkout. The released inference geometry is 832×480, BF16, three latent frames per block and four denoising calls. This adapter produces completed video artifacts; it does not yet expose an interactive streaming session.

Stage inspatio/world-1.5 and Wan-AI/Wan2.1-T2V-1.3B locally. Direct inputs also need depth-anything/DA3NESTED-GIANT-LARGE-1.1. Prepared scenes already contain depth and cameras and do not load DA3. Inference uses local checkpoints and leaves checkpoint directories unchanged.

from worldfoundry.pipelines.inspatio_world.pipeline_inspatio_world_v15 import InspatioWorldV15Pipeline

pipeline = InspatioWorldV15Pipeline.from_pretrained(
    model_path="/models/inspatio--world-1.5",
    wan_model_path="/models/Wan-AI--Wan2.1-T2V-1.3B",
    da3_model_path="/models/depth-anything--DA3NESTED-GIANT-LARGE-1.1",
    device="cuda:0",
)
result = pipeline(
    scene_dir="/data/prepared_scene",
    output_path="/outputs/pred.mp4",
    seed=0,
)
print(result["artifact_path"])

Prepared scenes use the official scene.json layout: RGB images or video, a prompt file (an empty prepared prompt is supported), uint16 depth with a shared min/max range, source intrinsics and poses, and a target trajectory. Camera matrices are OpenCV world-to-camera matrices: 16 row-major numbers per target frame. A four-image scene requires four views and complete groups of three latent frames. Valid output frames may exclude padded scene frames.

For direct inputs, replace scene_dir with videos="/data/input.mp4" or images=["/data/view_00.png"] (one or four images), and provide prompt and traj_txt_path. Videos are resized to 832×480; direct images must already have that resolution. The target trajectory must use the first-source-frame-normalized coordinate system and the same displacement scale as the source reconstruction. Use prepared scenes when supplying calibrated depth and cameras.

return_frames=True additionally loads decoded frames into host memory. The default returns artifact paths. compile_dit=True is an optional loading setting; compilation is not needed for standard inference.

Fast CPU contracts are in tests/synthesis/test_inspatio_v15_catalog.py and tests/synthesis/test_inspatio_v15_tensors.py. tests/manual/inspatio_v15_regression.py runs the same local checkpoints through this adapter and the pinned upstream implementation, then compares intermediate latents and decoded source, render, mask and prediction videos.

Six short GPU cases passed exact BF16/SDPA comparison: three prepared source types and three direct RGB inputs. Direct replay compares generation using the same DA3-estimated geometry; it does not independently assess DA3 reconstruction quality. Each case covers two causal blocks, including previous-block context.

To reproduce, check out the upstream revision shown in the catalog and stage the component weights locally. Use a new output directory; the video example ships without depth, which prepare-depth computes with the shared DA3 model.

# Set INSPATIO_GPU_ID to an available GPU index or UUID from nvidia-smi.
INSPATIO_UPSTREAM=/path/to/inspatio-world-v1.5
INSPATIO_CHECKPOINTS=/models
INSPATIO_VALIDATION=/tmp/inspatio-v15-validation
for mode in prepare prepare-depth native upstream compare; do
  CUDA_VISIBLE_DEVICES="${INSPATIO_GPU_ID}" WAN_DISABLE_FA3=1 python tests/manual/inspatio_v15_regression.py "$mode" \
    --upstream "$INSPATIO_UPSTREAM" --checkpoints "$INSPATIO_CHECKPOINTS" \
    --output "$INSPATIO_VALIDATION"
done

Repeat native, upstream and compare with --direct to exercise RGB staging and DA3. --cases selects individual cases for parallel runs on separate GPUs. The standard replay matrix includes inspatio-world-1p5-{image,multiview,video}-short; set WORLDFOUNDRY_REGRESSION_INSPATIO_SCENE_ROOT to the validation directory's fixtures subdirectory. Comparisons record checkpoint and numerical-output hashes and refuse different release weights.

Code attribution and the upstream Apache-2.0 text are centralized in THIRD-PARTY-NOTICES. Component weights keep their upstream terms.