---
title: EWMBench
description: "AgiBot embodied world-model video bench: scene, motion, semantics, diversity. WorldFoundry imports official rows; it does not invent a board."
---

EWMBench (AgiBot / AgibotTech) evaluates **action-conditioned** robot-manipulation videos from the AgiBot-World ecosystem. It is a world-model quality bench, not a tabletop success-rate suite. Four families roll into `ewmbench_average`:

- **Scene consistency** — does the workspace stay stable?
- **Motion correctness** — does the arm / object trajectory match the task?
- **Semantic alignment** — does the clip match the language instruction?
- **Diversity** — diversity of generated rollouts.

Paper: [EWMBench](https://arxiv.org/abs/2505.09694). Data / project: [`agibot-world/EWMBench`](https://huggingface.co/datasets/agibot-world/EWMBench) (CC-BY-NC-SA-4.0). Code: [AgibotTech/EWMBench](https://github.com/AgibotTech/EWMBench).

WorldFoundry **vendors runtime assets** and normalizes official result files through `benchmark-run`. `--run-official` can dispatch the upstream `evaluate.py` scorer when those dependencies are staged. Detector / VLM judge assets stay upstream.

<BenchmarkPageSections id="ewmbench" locale="en" />

## How to run

```bash
worldfoundry-eval zoo benchmark-run \
  --benchmark-id ewmbench \
  --mode official-validation \
  --official-results-path <official_results.json> \
  --generated-artifact-dir <generated_videos> \
  --output-dir tmp/ewmbench/official-validation \
  --json
```

`--task-manifest` and `--config-path` bind the task set and scorer config; `--limit` bounds wiring runs. Default import path: `WORLDFOUNDRY_EWMBENCH_RESULTS_PATH`. Optional root: `WORLDFOUNDRY_EWMBENCH_ROOT`. Shared asset: `ewmbench_dataset_assets`.

In-tree pointers:

```text
worldfoundry/data/benchmarks/tasks/external/ewmbench.yaml
worldfoundry/evaluation/tasks/execution/runners/ewmbench/runtime/ewmbench
```

Outputs include `scorecard.json` (primary `ewmbench_average`), `benchmark_contract.json`, and `raw_metric_table.jsonl`.

[← Benchmark Hub](/docs/evaluation/benchmark-hub)