---
title: WorldReasonBench
description: "Tsinghua world-state prediction bench: 436 cases, process-aware QA plus quality. WorldFoundry hosts the protocol; restricted assets and no complete official judge run."
---

WorldReasonBench (Tsinghua, with NTU and Waterloo) reframes video-generation eval as **world-state prediction**: given an initial state and an action, can the model generate a future whose evolution stays physically, socially, logically, and informationally consistent? Official size: **436 curated cases** with structured QA across **four reasoning dimensions / 22 subcategories**. Companion **WorldRewardBench** adds ~6K expert preference pairs.

Two protocols:

- **Process-aware Reasoning Verification** — structured QA plus reasoning-phase diagnostics. Primary `score_pr` geometrically combines QA accuracy with dynamic reasoning. `reasoning_gap` (static outcome minus dynamic reasoning; **lower is better**) flags outcome hacking.
- **Multi-dimensional Quality Assessment** — reasoning quality, temporal consistency, visual aesthetics. Reward-model path also reports point-wise / pairwise correlations.

Paper: [WorldReasonBench](https://arxiv.org/abs/2605.10434). Project: [unix-ai-lab.github.io/WorldReasonBench](https://unix-ai-lab.github.io/WorldReasonBench/). Code: [UniX-AI-Lab/WorldReasonBench](https://github.com/UniX-AI-Lab/WorldReasonBench). Upstream README **prohibits redistribution, modification, and commercial use** without approval despite also referring to MIT.

WorldFoundry **hosts the protocol runtime** with an OpenAI-compatible multimodal judge. Result normalization / QA component code exists; **no complete official judge run is claimed**.

<BenchmarkPageSections id="worldreasonbench" locale="en" />

## How to run

```bash
worldfoundry-eval zoo benchmark-run \
  --benchmark-id worldreasonbench \
  --mode official-validation \
  --official-results-path <official_results_dir> \
  --output-dir tmp/worldreasonbench/official-validation \
  --json
```

`--protocol` selects the protocol. QA / pair files: `--qa-json`, `--pairs-json`. Judge: `--base-url`, `--api-key`, `--model`, `--judge-model`. Default import: `WORLDFOUNDRY_WORLDREASONBENCH_RESULTS_PATH`.

In-tree pointers: `worldfoundry/data/benchmarks/tasks/external/worldreasonbench.yaml`, `worldfoundry/evaluation/tasks/execution/runners/worldreasonbench`.

Outputs include `scorecard.json` (primary `score_pr`) and `protocol_results.json`.

[← Benchmark Hub](/docs/evaluation/benchmark-hub)