---
title: IPV-Bench
description: "NUS Show Lab impossible-video bench: generate and recognize physically impossible events. WorldFoundry normalizes official CSV/JSON; it does not invent a board."
---

IPV-Bench (Impossible Videos; Show Lab, NUS) flips the usual physics question. It asks whether models can **depict** counterfactual, physically impossible events — and whether video-understanding models can **recognize** them. The taxonomy covers physical, biological, geographical, and social impossibilities. Two tracks:

- **Generation (IPV-Txt)** — visual quality and prompt following; `impossible_video_score` is the share of clips that are both high-quality and prompt-faithful.
- **Understanding (IPV-Vid)** — judgement, multi-choice QA, and open QA against official labels.

`ipv_bench_average` aggregates whatever official families you supply.

Paper: [Impossible Videos](https://arxiv.org/abs/2503.14378). Code: [showlab/Impossible-Videos](https://github.com/showlab/Impossible-Videos).

WorldFoundry **normalizes official CSV/JSON exports** for both tracks. It does not run the official judges or understanding models in tree. `--run-fixture` is wiring only.

<BenchmarkPageSections id="ipv-bench" locale="en" />

## How to run

```bash
worldfoundry-eval zoo benchmark-run \
  --benchmark-id ipv-bench \
  --mode official-validation \
  --official-results-path <official_results.csv> \
  --generated-artifact-dir <generated_videos> \
  --output-dir tmp/ipv-bench/official-validation \
  --json
```

Default import: `WORLDFOUNDRY_IPV_BENCH_RESULTS_PATH`. Optional root / backend: `WORLDFOUNDRY_IPV_BENCH_ROOT`, `WORLDFOUNDRY_IPV_BENCH_RUNTIME_BACKEND`. No Hugging Face dataset is required by the contract.

In-tree pointers: `worldfoundry/data/benchmarks/tasks/external/ipv-bench.yaml`, `worldfoundry/evaluation/tasks/execution/runners/ipv_bench`.

Outputs include `scorecard.json` (primary `ipv_bench_average`) and `raw_metric_table.jsonl`.

[← Benchmark Hub](/docs/evaluation/benchmark-hub)