RoboCerebra

Last updated: Oct 7, 2026

On this page

Dimensions Data composition Metrics Leaderboard How to run

RoboCerebra (Beihang CoLa Lab, NeurIPS 2025) tests System-2 planning on long household sequences, not reactive System-1 control. The release is 1,000 human-annotated trajectories over 100 task variants (mean 9.1 atomic steps; horizons on the order of 3,000 simulation steps) on a LIBERO-based stack. Published eval is 600 rollouts (60 tasks × 10 trials) with anchor-aligned planner / VLA handoffs. Six official conditions: Ideal, Memory Exploration, Memory Execution, Random Disturbance, Observation Mismatching, and Mix.

Paper: RoboCerebra. Project: robocerebra.github.io. Code: buaa-colalab/RoboCerebra.

WorldFoundry imports official result exports (worldfoundry-eval zoo benchmark-run --mode official-validation). It does not run the bundled LIBERO simulator, collect rollouts, or host the hierarchical VLM planner + VLA controller. Declare the six-way condition split on the export if you want comparable numbers.

Official paper teaser: long-horizon household tasks that need memory and plan updates.

Dimensions

Ideal
KindTrack
Static, fully observable household sequences. Control condition for long-horizon planning.
Memory Exploration
KindTrack
Actively probe the scene (cabinet contents, hidden objects) to build an internal map.
Memory Execution
KindTrack
Perceptual cues are removed; the policy must retrieve earlier observations to finish the goal.
Random Disturbance
KindTrack
Unexpected object shifts or drops mid-episode. Part of the Dynamic Scene group.
Observation Mismatching
KindTrack
Observed state disagrees with the instruction or prior belief; the planner must update.
Mix
KindTrack
Memory demands plus dynamic scene variation in one episode.

Data composition

Size
NeurIPS 2025 long-horizon household manipulation on a LIBERO-based stack. Official release: 1,000 human-annotated trajectories across 100 task variants (mean 9.1 atomic steps; trajectories on the order of 3,000 simulation steps). Published eval is 600 rollouts (60 tasks × 10 trials) with anchor-aligned System-1 / System-2 handoffs.
Sources
github.com
Splits
eval
Modalities
language · vision · action · state
License
upstream_terms
Notes
WorldFoundry imports official rollout exports. It does not run the bundled LIBERO simulator or the hierarchical VLM+VLA stack.

Metrics

Metric

success_rate
DirectionHigher is better
Episode success aggregated over the imported rollouts. Official tables also break out the six sub-task types.
PrimarySuccess Rate
task_success
DirectionHigher is better
Per-task success aggregate when the export names tasks.
episode_success
DirectionHigher is better
Per-episode success indicator aggregate.
reward
DirectionHigher is better
Mean episode reward / return when the export includes it.

Leaderboard

Results from the paper (not a live board; not WorldFoundry-run scores). Planner / VLM baselines from the paper. RoboCerebra: A Large-scale Benchmark for Long-horizon Robotic Manipulation Evaluation (2025).

Model	Success rate (SR)
GPT-4o	16.04
GPT-4o-Blind	15.10
LLaVA-Next-Video-7B	11.37
Qwen2.5-VL-7B	11.19
Qwen2.5-VL-7B-SFT	9.33

How to run

worldfoundry-eval zoo benchmark-run \
  --benchmark-id robocerebra \
  --mode official-validation \
  --official-results-path <official_results.json> \
  --output-dir tmp/robocerebra/official-validation \
  --json

Optional WORLDFOUNDRY_ROBOCEREBRA_ROOT points at a local bench/runtime root. Default import path: WORLDFOUNDRY_ROBOCEREBRA_RESULTS_PATH. Manifest: worldfoundry/data/benchmarks/tasks/external/robocerebra.yaml. Outputs: scorecard.json (primary success_rate), raw_results.jsonl, evaluation/summary.json.

← Benchmark Hub