SANA-WM 80-Scene Benchmark

Last updated: Oct 7, 2026

On this page

Dimensions Data composition Metrics Leaderboard How to run

The SANA-WM 80-Scene Benchmark accompanies NVIDIA's SANA world-model work. Each of two splits (simple_60s, hard_60s) holds 80 scenes: a conditioning image plus a 961-frame, 16 FPS camera trajectory. The world model must roll out one minute of video per scene. Four families:

VBench-style quality — visual / temporal dimensions; primary key vbench_overall.
Revisit consistency — PSNR / SSIM / LPIPS on frame pairs where the trajectory returns to a prior viewpoint.
Camera pose — rotation error (degrees), relative translation error, relative motion consistency vs the target trajectory.
delta_iq — imaging-quality drop across temporal windows (long-horizon drift; lower is better).

Code: NVlabs/Sana. Data: Efficient-Large-Model/SANA-WM-Bench (Apache-2.0).

WorldFoundry implements the evaluator in tree. It reuses resident VBench and Pi3 rather than executing an NVlabs/Sana checkout at score time. Pose inference is a separately schedulable GPU stage; only eval_poses.json is consumed.

Official paper figure: the 80 initial scenes used in the one-minute SANA-WM benchmark.

Dimensions

simple_60s
KindSplit
80 scenes, 961-frame / 16 FPS camera trajectory, one-minute rollout.
hard_60s
KindSplit
80 harder scenes with the same 961-frame / 16 FPS protocol.

Data composition

Size
NVIDIA SANA-WM 80-scene world-model bench. Two splits × 80 scenes, 961-frame 16 FPS trajectories. Families: VBench quality, revisit consistency, camera pose, delta_iq. Apache-2.0.
Sources
Efficient-Large-Model/SANA-WM-Bench · github.com
Splits
simple_60s · hard_60s
Modalities
text · image · video · camera-trajectory
License
Apache-2.0
Notes
WorldFoundry owns the evaluator and reuses resident VBench/Pi3. Full 80×2 numerical parity has not been run. Pose inference is a separate GPU stage.

Metrics

Metric

vbench_overall
DirectionHigher is better
Mean over the resident VBench visual/temporal quality dimensions.
PrimaryVBench Overall
vbench_total_score
DirectionHigher is better
VBench total_score diagnostic aggregate.
revisit_psnr
DirectionHigher is better
PSNR between frame pairs where the trajectory revisits a previous viewpoint.
revisit_ssim
DirectionHigher is better
SSIM over the same revisit frame pairs.
revisit_lpips
DirectionLower is better
LPIPS over the same revisit frame pairs.
rot_err_deg
DirectionLower is better
Camera rotation error in degrees against the target trajectory.
trans_err_rel
DirectionLower is better
Relative camera translation error against the target trajectory.
cam_mc_rel
DirectionLower is better
Relative camera motion-consistency error.
delta_iq
DirectionLower is better
Imaging-quality degradation across temporal windows (long-horizon drift).

Leaderboard

Results from the paper (not a live board; not WorldFoundry-run scores). Table 2 simple-trajectory split. Resolutions differ (LingBot-World / Infinite-World / HY-WorldPlay at 480p; others 720p). SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformer (2026).

Model	VBench Overall (simple-trajectory)
LingBot-World480p	81.82
SANA-WM + refiner	80.62
SANA-WM	79.29
Infinite-World480p	79.18
Matrix-Game 3.0	78.53
HY-WorldPlay480p	68.82

How to run

hf download Efficient-Large-Model/SANA-WM-Bench --repo-type dataset \
  --local-dir "${WORLDFOUNDRY_SANA_WM_BENCH_DATASET_ROOT}"

${WORLDFOUNDRY_UNIFIED_PYTHON} \
  worldfoundry/evaluation/tasks/execution/runners/sana_wm_bench/run_sana_wm_bench_official_runner.py \
  --run-official \
  --dataset-root "${WORLDFOUNDRY_SANA_WM_BENCH_DATASET_ROOT}" \
  --generated-video-dir "${WORLDFOUNDRY_GENERATED_ARTIFACT_DIR}" \
  --split simple_60s \
  --metrics vbench,revisit,temporal \
  --output-dir tmp/sana_wm_bench/official \
  --json

Name videos <scene_id>_generated.mp4. --metrics selects vbench,revisit,temporal,camera. --max-pairs bounds revisit pairs; --revisit-lpips adds LPIPS. Camera metrics need --pose-results-path / eval_poses.json (radians converted to degrees on import). Dataset layout: images/<scene_id>.png, <split>/sanawm_export_v2/<scene_id>.npz, run_manifest.jsonl.

In-tree pointer: worldfoundry/data/benchmarks/tasks/external/sana-wm-bench.yaml.

Outputs include scorecard.json (primary vbench_overall) and raw_metric_table.jsonl.

← Benchmark Hub