WorldFoundry
English中文

DualCamCtrl

Method overview: DualCamCtrl’s dual-branch framework jointly generates aligned RGB and depth video latents.

HKUST-GZ EnVision: RGB + depth + camera .torch. Official demo verified: 320×480, 61 frames, 50 steps. License unknown. Expected seaside.mp4.

Edit this pageGitHubMarkdown

DualCamCtrl steers video with a camera trajectory plus a depth image: RGB and depth together pin geometry so the camera move stays grounded. WorldFoundry vendors infer under worldfoundry/synthesis/visual_generation/dualcamctrl as DualCamCtrlPipeline. Runner and demo are verified against the official RGB / depth / camera contract.

Method overview: DualCamCtrl’s dual-branch framework jointly generates aligned RGB and depth video latents.
Method overview: DualCamCtrl’s dual-branch framework jointly generates aligned RGB and depth video latents.

Paper: arXiv 2511.23127. Project: soyouthinkyoucantell.github.io/dualcamctrl-page. Code: EnVision-Research/DualCamCtrl. Publisher: HKUST (Guangzhou) EnVision Lab. Catalog license: unknown.

Build your run

Pipeline
DualCamCtrlPipelineworldfoundry.pipelines.component_pipelines
TaskInteractive Video interactive-video
Environmentworldfoundry-unified-cu128
DeviceCUDA 12.8
Runnerworldfoundry.pipeline
Loadingfrom_pretrained
Invocationnative_or_standard_invocation
worldfoundry-eval run \
  dualcamctrl \
  --pipeline.task-profile interactive-video \
  --json

What it is for

Dual-camera control for a stereo-style pair.

Launch worldfoundry-eval run dualcamctrl.

Score on CameraBench — camera-trajectory adherence of controlled generation.

Runtime contract

Required: prompt and image; optional: video; actions: camera_trajectory, depth_image; artifacts: dualcamctrl.mp4 and seaside.mp4.

KnobOfficial demo
Size320×480
Frames61
Steps50
Trajectory.torch camera file

Run in WorldFoundry

Status: Verified. Unified environment worldfoundry-unified-cu128 (Python 3.11, CUDA 12.8). Stage FayeHongfeiZhang/DualCamCtrl, alibaba-pai/Wan2.1-Fun-V1.1-1.3B-Control-Camera, and Wan-AI/Wan2.1-T2V-1.3B among others locally — missing files fail at load, nothing is downloaded. Run worldfoundry-eval run dualcamctrl. Catalog VRAM is unset.