WorldFoundry
English中文

UniAnimate-DiT

Method overview: Wan-DiT with tunable pose encoders and QKV-LoRA generates pose-driven video from a reference image.

UniAnimate-DiT from Alibaba (Tongyi Lab): human-image-animation and image-to-video; Runner parity pending.

Edit this pageGitHubMarkdown

UniAnimate-DiT is Alibaba ali-vilab's human image animation model built on Wan2.1-I2V: it animates a person photo to follow a driving pose sequence. The in-tree route is wired with local checkpoint assets and exposes max_frames, resolution, step count, and seed controls.

Official paper teaser: UniAnimate-DiT animates stylized and photorealistic characters from driving poses.
Official paper teaser: UniAnimate-DiT animates stylized and photorealistic characters from driving poses.
Method overview: Wan-DiT with tunable pose encoders and QKV-LoRA generates pose-driven video from a reference image.
Method overview: Wan-DiT with tunable pose encoders and QKV-LoRA generates pose-driven video from a reference image.

The recorded paper is arXiv 2504.11289.

Code: ali-vilab/unianimate-dit.

Build your run

Pipeline
UniAnimateDiTPipelineworldfoundry.pipelines.video_official.pipeline_official_video
TaskInteractive Video interactive-video
Environmentworldfoundry-unified-cu128
DeviceCUDA 12.8
Runnerworldfoundry.pipeline
Loadingfrom_pretrained
Invocationnative_or_standard_invocation
worldfoundry-eval run \
  unianimate-dit \
  --pipeline.task-profile interactive-video \
  --json

What it is for

Use unianimate-dit for the family's joint T2V/I2V inference path. The catalog records both task profiles on this page.

Built on the Wan2.1 image-to-video diffusion transformer with pose-sequence conditioning for human animation (checkpoint: ZheWang123/UniAnimate-DiT). Deeper conditioning internals are not recorded in this repository. It is published by Alibaba (Tongyi Lab).

Catalog tasks include human-image-animation, image-to-video, and video-generation. Documented modalities: image in, video out.

Runtime contract

Optional prompt, negative_prompt, input_path, frames, and fps; artifacts unianimate-dit.mp4.

CLI task profiles: interactive-video. WorldFoundry binds UniAnimateDiTPipeline.

Family status: Runner parity pending.

Recorded interactive-video defaults: height=832, width=480, seed=0.

Prepare a clean subject photo and a driving pose sequence for best results.

Running in WorldFoundry

The catalog launch is worldfoundry-eval run unianimate-dit --pipeline.task-profile interactive-video. The command builder above switches --pipeline.task-profile among recorded tasks and still emits worldfoundry-eval run.

It runs in the unified environment worldfoundry-unified-cu128 (Python 3.11, CUDA 12.8).

Recorded weight repositories include ZheWang123/UniAnimate-DiT. Prepare the environment and check local assets with bash scripts/inference/prepare_model_infer.sh unianimate-dit. Add --download to fetch missing public checkpoints.

Status: Runner parity pending.