---
title: PhysVidBench
description: "Caption-mediated T2V QA: 383 prompts, physics yes/no questions about AuroraCap captions. WorldFoundry imports output.csv or runs Gemini on captions you supply."
---

PhysVidBench (Koç University / Cyberiada) is **caption-mediated QA**, not pixel matching and not PhyGenEval. A T2V model generates a video; upstream **AuroraCap** captions it; an LLM answers physics **yes/no** questions about that caption (affordance, tool use, materials, temporal dynamics). Catalog records **383 prompts** and **4123 questions** (`cc-by-4.0`).

[PhyGenBench](/docs/evaluation/benchmark-hub/phygenbench) judges the video with VQAScore/VLM stages. [PhyGround](/docs/evaluation/benchmark-hub/phyground) applies 1–5 law criteria. PhysVidBench never looks at pixels in tree — only captions + questions.

Paper: [PhysVidBench](https://arxiv.org/abs/2507.15824). Project: [cyberiada.github.io/PhysVidBench](https://cyberiada.github.io/PhysVidBench/). Code: [ensanli/PhysVidBenchCode](https://github.com/ensanli/PhysVidBenchCode). Data: [Anonymousny/PhysVidBench](https://huggingface.co/datasets/Anonymousny/PhysVidBench).

WorldFoundry bundles `prompts_questions.csv` and caption tracks. It **imports** upstream `output.csv`, or runs **bounded Gemini QA** on captions you supply. It does **not** run AuroraCap on your videos and cannot prove those captions came from them.

<BenchmarkPageSections id="physvidbench" locale="en" />

## How to run

Import an upstream QA export:

```bash
worldfoundry-eval zoo benchmark-run \
  --benchmark-id physvidbench \
  --mode official-validation \
  --official-results-path <output.csv> \
  --generated-artifact-dir <generated_videos> \
  --output-dir tmp/physvidbench/official-validation \
  --json
```

Bounded Gemini QA over supplied (or bundled) captions — not official pipeline evidence:

```bash
${WORLDFOUNDRY_UNIFIED_PYTHON} \
  worldfoundry/evaluation/tasks/execution/runners/physvidbench/run_physvidbench_official_runner.py \
  --run-official \
  --generated-artifact-dir "${WORLDFOUNDRY_GENERATED_ARTIFACT_DIR}" \
  --captions-dir worldfoundry/data/benchmarks/assets/physvidbench/captions \
  --output-dir tmp/physvidbench/official \
  --json
```

Select the judge with `WORLDFOUNDRY_PHYSVIDBENCH_JUDGE_BACKEND`. Outputs include `scorecard.json` (primary `physvidbench_average`) and `output.csv`.

[← Benchmark Hub](/docs/evaluation/benchmark-hub)