WorldFoundry
English中文

VideoCrafter2 T2V

VideoCrafter2 T2V

VideoCrafter2 T2V from Tencent AI Lab: text-to-video and video-generation; Runner parity pending.

Edit this pageGitHubMarkdown

VideoCrafter2 T2V is the improved text-to-video model of the VideoCrafter family, trained to overcome low-quality video data by combining it with high-quality image data. It is Apache-2.0 on its HF card, making it the license-cleanest member of the family. The in-tree infer-only runtime exposes local checkpoint, fps, frame count, and seed controls.

No official teaser or method figure is checked into this docs tree.

The recorded paper is arXiv 2401.09047.

Project page: ailab-cvc.github.io/videocrafter2. Code: VideoCrafter/VideoCrafter.

Build your run

Pipeline
VideoCrafter2T2VPipelineworldfoundry.pipelines.videocrafter.pipeline_videocrafter2_t2v
TaskInteractive Video interactive-video
Environmentworldfoundry-unified-cu128
DeviceCUDA 12.8
Runnerworldfoundry.pipeline
Loadingfrom_pretrained
Invocationnative_or_standard_invocation
worldfoundry-eval run \
  videocrafter2-t2v \
  --pipeline.task-profile interactive-video \
  --json

What it is for

videocrafter2-t2v is the recorded prompt-to-video route — not image editing and not a policy. Start here when the job is a clip from text.

A U-Net latent video diffusion model whose training decouples motion (from video) and appearance (from images), per the upstream publication. Checkpoint provenance is tracked on the family entry. It is published by Tencent AI Lab.

Catalog tasks include text-to-video and video-generation. Catalog aliases include videocrafter2_t2v. Documented modalities: text in, video out.

Recorded evaluation entry points include EvalCrafter (Companion benchmark of the VideoCrafter effort), VBench (Standard T2V suite).

Not VideoCrafter and VideoCrafter1 T2V. Score on EvalCrafter and VBench.

Runtime contract

Optional prompt, frames, fps, height, and width; artifacts generated_video and videocrafter2-t2v.mp4.

CLI task profiles: interactive-video. WorldFoundry binds VideoCrafter2T2VPipeline.

Family status: Runner parity pending.

Runner parity is pending. This is the recommended VideoCrafter variant: better quality than V1 and a clean Apache-2.0 license.

Checkpoint provenance is tracked on the top-level videocrafter catalog entry to avoid duplicate HF repo aliases.

Running in WorldFoundry

The catalog launch is worldfoundry-eval run videocrafter2-t2v --pipeline.task-profile interactive-video. The command builder above switches --pipeline.task-profile among recorded tasks and still emits worldfoundry-eval run.

It runs in the unified environment worldfoundry-unified-cu128 (Python 3.11, CUDA 12.8).

Recorded weight repositories include VideoCrafter/VideoCrafter2. Prepare the environment and check local assets with bash scripts/inference/prepare_model_infer.sh videocrafter2-t2v. Add --download to fetch missing public checkpoints.

Status: Runner parity pending.