WorldFoundry
English中文

Lyra

Lyra from NVIDIA: open generative 3D and 4D world models; Verified; 2 variants.

Edit this pageGitHubMarkdown

Lyra is NVIDIA's generative world model family for exploration: from an image or video (Lyra-1) or from actions (Lyra-2) it produces an explorable world video. In this catalog both variants run through in-tree pipelines backed by vendored runtime adapters, the default runner targets Lyra-2, and runner status is Verified.

Official paper teaser: Lyra 2.0 turns one image into a long-horizon 3D-consistent world via video generation and 3D Gaussian Splatting.
Official paper teaser: Lyra 2.0 turns one image into a long-horizon 3D-consistent world via video generation and 3D Gaussian Splatting.
Method overview: spatial memory and dense 3D correspondence condition a DiT that generates camera-guided video, then reconstructs 3DGS.
Method overview: spatial memory and dense 3D correspondence condition a DiT that generates camera-guided video, then reconstructs 3DGS.

Project page: research.nvidia.com/labs/sil/projects/lyra2. Code: nv-tlabs/lyra.

The paper is Paper. Project page: research.nvidia.com/labs/sil/projects/lyra2. Code: nv-tlabs/lyra.

Build your run

Pipeline
Lyra1Pipelineworldfoundry.pipelines.lyra.pipeline_lyra1
TaskOfficial Static Sdg official-static-sdg
Environmentlyra
DeviceCUDA 12.4
Runnerworldfoundry.pipeline
Loadingfrom_pretrained
Invocationnative_or_standard_invocation
worldfoundry-eval run \
  lyra-1 \
  --pipeline.task-profile official-static-sdg \
  --json

Variants

  • lyra-1
    StatusVerified

    image-or-video-to-world-video

    Runtime profilelyra-1

    Pipeline bindinglyra-1

  • lyra-2
    StatusVerified

    action-conditioned-world-video

    Runtime profilelyra-2

    Pipeline bindinglyra-2

What it is for

Use lyra-1 for catalog world-model inference. Choose it when you need action-conditioned future frames rather than a standalone clip generator.

The catalog records two generations: Lyra-1 for image-or-video-to-world-video and Lyra-2 for action-conditioned world video (paper on OpenReview; project page at NVIDIA Research). Runtime execution requires local Lyra checkpoints, text-encoder assets, DA3 (Depth Anything 3) assets, and CUDA dependencies; internals beyond the recorded structure are not stored in this repository.

Catalog tasks include open generative 3D and 4D world models. Catalog aliases include Project Lyra, Lyra 1.0, and Lyra 2.0. Also recorded: lyra-2.

Recorded evaluation entry points include WorldScore (Standard scoring for explorable generated worlds), CameraBench (Evaluates camera/viewpoint control quality during exploration).

Runtime contract

Optional image, prompt, mode, trajectory, and camera_rotation; artifacts generated_world, generated_3d_asset, and lyra-1.mp4.

CLI task profiles: official-static-sdg and official-zoomgs. WorldFoundry binds LyraPipeline.

Start on lyra-1. The family registers 2 catalog variants (cards above): lyra-1 (image-or-video-to-world-video) — Runner parity pending; lyra-2 (action-conditioned-world-video) — Runner parity pending. Family status: Verified.

Recorded official-zoomgs defaults: fps=16, seed=1.

Pick the variant by input: lyra-1 when you start from an image or video, lyra-2 when you drive the world with actions. The default catalog runner resolves to Lyra-2.

Running in WorldFoundry

Launch worldfoundry-eval run lyra-1 --pipeline.task-profile official-static-sdg to step the world. The command builder above switches --pipeline.task-profile among recorded tasks and still emits worldfoundry-eval run.

Recorded weight repositories include depth-anything/DA3NESTED-GIANT-LARGE-1.1. Prepare the environment and check local assets with bash scripts/inference/prepare_model_infer.sh lyra-1. Add --download to fetch missing public checkpoints.

Status: Verified.