MiKASA-Robo

Last updated: Oct 7, 2026

On this page

Dimensions Data composition Metrics Leaderboard How to run

MiKASA-Robo is the tabletop half of MIKASA (Memory-Intensive Skills Assessment Suite for Agents) from AIRI, MIPT, and HSE. The ICLR 2026 paper is 32 ManiSkill RL tasks in 12 groups across four memory types: object, spatial, sequential, and capacity (occlusion recall, prior layouts, long action orders). The later MIKASA-Robo-VLA release grows to 90 language-conditioned tasks (10 memory types), Short / Medium / Long horizons (25–2160 steps), and 22,500 oracle trajectories. Those two releases are not interchangeable.

Paper: Memory, Benchmark & Robots (OpenReview). VLA docs: mikasarobo.github.io. Code: CognitiveAISystems/MIKASA-Robo.

WorldFoundry imports official result exports. It does not run ManiSkill, collect PPO/oracle trajectories, or pick RL vs VLA for you — declare that on the export.

Official paper teaser: a taxonomy of memory problems for robot RL.

Dimensions

Object memory
KindCategory
ICLR 2026 RL suite: remember object identity under partial observability.
Spatial memory
KindCategory
Locate occluded objects and recall prior layouts.
Sequential memory
KindCategory
Reproduce a required action order over a long horizon.
Memory capacity
KindCategory
How much earlier information the agent must hold to succeed.
VLA Short / Medium / Long
KindSplit
MIKASA-Robo-VLA groups 90 language-conditioned tasks into three horizon splits (25–2160 steps).

Data composition

Size
AIRI / MIPT / HSE tabletop memory bench on ManiSkill. ICLR 2026 paper: 32 RL tasks in 12 groups across four memory types. The later VLA release grows to 90 language-conditioned tasks (10 memory types) with 22,500 oracle trajectories. WorldFoundry does not pick a release for you — declare RL vs VLA on the export.
Sources
github.com
Splits
eval
Modalities
language · vision · action · state
License
upstream_terms
Notes
Paper: arXiv:2502.10550 (OpenReview 9cLPurIZMj). WorldFoundry imports official result files only.

Metrics

Metric

success_rate
DirectionHigher is better
Primary episode success over imported rollouts.
PrimarySuccess Rate
task_success
DirectionHigher is better
Per-task success aggregate across the named memory group.
episode_success
DirectionHigher is better
Per-episode success indicator aggregate.
reward
DirectionHigher is better
Mean episode reward / return (dense rewards exist in the VLA release).

Leaderboard

The MIKASA paper catalogs 32 memory tasks (Table 1) and reports PPO-MLP / PPO-LSTM as learning curves (Figures 4–9), including a 100% oracle-state ceiling. There is no method×score ranking table to ingest, and no author-maintained live board. Memory, Benchmark & Robots (2025).

Official paper spider chart: PPO and related agents across MiKASA memory axes.

How to run

worldfoundry-eval zoo benchmark-run \
  --benchmark-id mikasa \
  --mode official-validation \
  --official-results-path <official_results.json> \
  --output-dir tmp/mikasa/official-validation \
  --json

Optional WORLDFOUNDRY_MIKASA_ROOT points at a local runtime/repo. Default import path: WORLDFOUNDRY_MIKASA_RESULTS_PATH. Manifest: worldfoundry/data/benchmarks/tasks/external/mikasa.yaml. Outputs: scorecard.json (primary success_rate), raw_results.jsonl, evaluation/summary.json.

← Benchmark Hub