WorldFoundry
English中文

I2VGen-XL

Method overview: The overall framework of I2VGen-XL. In the base stage , two hierarchical encoders are employed to simultaneously capture high-level semantics and low-level details of input images, ensuring more realistic dynamics.

I2VGen-XL from Alibaba (Tongyi Lab): image-to-video and text-to-video; Runner parity pending.

Edit this pageGitHubMarkdown

I2VGen-XL is Alibaba ali-vilab's cascaded image-to-video model, one of the earlier high-resolution open I2V releases. It turns a still image (with optional text) into a short video through a two-stage generate-then-refine cascade. The WorldFoundry route is complete and reports missing checkpoint assets at runtime.

Method overview: The overall framework of I2VGen-XL. In the base stage , two hierarchical encoders are employed to simultaneously capture high-level semantics and low-level details of input images, ensuring more realistic dynamics.
Method overview: The overall framework of I2VGen-XL. In the base stage , two hierarchical encoders are employed to simultaneously capture high-level semantics and low-level details of input images, ensuring more realistic dynamics.

The recorded paper is arXiv 2311.04145.

Project page: i2vgen-xl.github.io. Code: ali-vilab/i2vgen-xl.

Build your run

Pipeline
I2VGenXLPipelineworldfoundry.pipelines.video_official.pipeline_official_video
TaskInteractive Video interactive-video
Environmentworldfoundry-unified-cu128
DeviceCUDA 12.8
Runnerworldfoundry.pipeline
Loadingfrom_pretrained
Invocationnative_or_standard_invocation
worldfoundry-eval run \
  i2vgen-xl \
  --pipeline.task-profile interactive-video \
  --json

What it is for

Use i2vgen-xl for the family's joint T2V/I2V inference path. The catalog records both task profiles on this page.

Per the upstream publication, I2VGen-XL uses a base stage for semantic coherence and a refinement stage for detail/resolution, both latent diffusion models. This repository records the prompt+image contract and environment/checkpoint assumptions in the runtime profile; run readiness is kept in release run manifests, not here. It is published by Alibaba (Tongyi Lab).

Catalog tasks include image-to-video and text-to-video. Documented modalities: image and text in, video out.

Recorded evaluation entry points include AIGCBench (I2V-focused benchmark suite).

Runtime contract

Optional prompt, negative_prompt, input_path, frames, and fps; artifacts i2vgen-xl.mp4.

CLI task profiles: interactive-video. WorldFoundry binds I2VGenXLPipeline.

Family status: Runner parity pending.

Recorded interactive-video defaults: frames=16, height=704, width=1280, seed=8888.

Integration is complete; no runner parity record exists yet in this manifest. Both the official GitHub and ali-vilab HF repos are confirmed reachable. Treat it as a legacy-quality baseline next to newer DiT-based I2V models in this catalog.

Running in WorldFoundry

Launch worldfoundry-eval run i2vgen-xl --pipeline.task-profile interactive-video to sample the clip. The command builder above switches --pipeline.task-profile among recorded tasks and still emits worldfoundry-eval run.

It runs in the unified environment worldfoundry-unified-cu128 (Python 3.11, CUDA 12.8).

Recorded weight repositories include ali-vilab/i2vgen-xl. Prepare the environment and check local assets with bash scripts/inference/prepare_model_infer.sh i2vgen-xl. Add --download to fetch missing public checkpoints.

Recorded licenses include mit; status: Runner parity pending.