WorldFoundry
English中文

Open-MAGVIT2

Method overview: Overview of Open-MAGVIT2. There are two crucial stages in Open-MAGVIT2. In Stage I {I} : the image is first encoded by MAGVIT-v2 Encoder and subsequently transformed into bits format by Lookup-Free Quantizer (LFQ)..

Open-MAGVIT2 from Tencent (ARC Lab): video-tokenization and class-conditional-image-generation; Verified.

Edit this pageGitHubMarkdown

Open-MAGVIT2 is Tencent ARC's open reproduction of the MAGVIT-v2 lookup-free visual tokenizer, plus autoregressive generation on top of it. In this catalog the runnable route is class-conditional ImageNet image generation — full 256-token generation passed checkpoint-backed GPU and semantic visual QA (verified at 256x256, ImageNet class 207). The artifact is a generated PNG image, not video.

Method overview: Overview of Open-MAGVIT2. There are two crucial stages in Open-MAGVIT2. In Stage I {I} : the image is first encoded by MAGVIT-v2 Encoder and subsequently transformed into bits format by Lookup-Free Quantizer (LFQ)..
Method overview: Overview of Open-MAGVIT2. There are two crucial stages in Open-MAGVIT2. In Stage I {I} : the image is first encoded by MAGVIT-v2 Encoder and subsequently transformed into bits format by Lookup-Free Quantizer (LFQ)..

The recorded paper is arXiv 2409.04410.

Code: TencentARC/Open-MAGVIT2.

Build your run

Pipeline
OpenMAGVIT2Pipelineworldfoundry.pipelines.component_pipelines
TaskInteractive Video interactive-video
Environmentworldfoundry-unified-cu128
DeviceCUDA 11.3
Runnerworldfoundry.pipeline
Loadingfrom_pretrained
Invocationnative_or_standard_invocation
worldfoundry-eval run \
  open-magvit2 \
  --pipeline.task-profile interactive-video \
  --json

What it is for

open-magvit2 is the recorded image-generation line. Start here for prompt-to-image inference.

Two parts recorded in-tree: the lookup-free quantization (LFQ) tokenizer with a very large codebook, and an autoregressive transformer that predicts visual tokens for a requested ImageNet class. The class_id input selects the category; generation runs the vendored CUDA wrapper. It is published by Tencent (ARC Lab).

Catalog tasks include video-tokenization, class-conditional-image-generation, and autoregressive-generation. Catalog aliases include open_magvit2 and openmagvit2.

Runtime contract

Optional prompt, input_path, interactions, fps, and steps; artifacts open_magvit2.png.

CLI task profiles: interactive-video. WorldFoundry binds OpenMAGVIT2Pipeline.

Family status: Verified.

Runner status is Verified — this route actually ran on GPU with the TencentARC/Open-MAGVIT2 weights (apache-2.0). Keep expectations scoped: it is class-conditional image generation, not prompt-to-video; video tokenization is what the tokenizer supports upstream.

Runtime source is vendored in-tree and the CUDA generation wrapper is wired.

Running in WorldFoundry

The catalog launch is worldfoundry-eval run open-magvit2 --pipeline.task-profile interactive-video. The command builder above switches --pipeline.task-profile among recorded tasks and still emits worldfoundry-eval run.

It runs in the unified environment worldfoundry-unified-cu128 (Python 3.11, CUDA 11.3).

Recorded weight repositories include TencentARC/Open-MAGVIT2. Prepare the environment and check local assets with bash scripts/inference/prepare_model_infer.sh open-magvit2. Add --download to fetch missing public checkpoints.

Recorded licenses include apache-2.0; status: Verified.