VideoPhy

VideoPhy (UCLA and Google Research) tests whether text-to-video models obey physical commonsense in material interactions (solid-solid, solid-fluid, fluid-fluid). Each prompt describes a physical interaction. The official judge is VideoCon-Physics, a fine-tuned video-language model. Three scores:

• Semantic Adherence — does the clip match the prompt?
• Physical Commonsense — does the motion obey physics?
• Joint Score — both together.

videophy_average is the WorldFoundry mean over available families. The official scoring path is in tree; full 688-prompt reproduction is pending.

Paper: VideoPhy. Project: videophy.github.io. Code: Hritikbansal/videophy.

WorldFoundry bundles the prompt list and can run the in-tree VideoCon-Physics AutoEvaluator (--videocon-checkpoint, --sa-threshold / --pc-threshold) or import official JSON. The checkpoint is caller-supplied, not vendored.

Dimensions

• Semantic Adherence

  KindAxis

  Does the clip match the physical-interaction prompt?

• Physical Commonsense

  KindAxis

  Does the motion obey physics for solid/fluid interactions?

• Joint Score

  KindAxis

  SA and PC together.

Data composition

• Size

  UCLA / Google T2V physical-commonsense bench. Official judge: VideoCon-Physics. Full official suite is 688 prompts; in-tree prompt list is bundled. Scoring path present; full reproduction pending.

• Sources

  github.com

• Modalities

  text · video

• Notes

  VideoCon-Physics checkpoint is caller-supplied, not vendored.

Metrics

Metric

• semantic_adherence

  DirectionHigher is better

  Semantic Adherence normalized benchmark score.

• physical_commonsense

  DirectionHigher is better

  Physical Commonsense normalized benchmark score.

• joint_score

  DirectionHigher is better

  Joint Score normalized benchmark score.

• videophy_average

  DirectionHigher is better

  VideoPhy Average normalized benchmark score.

  PrimaryVideoPhy Average

Leaderboard

Results from the paper (not a live board; not WorldFoundry-run scores). Paper overall semantic-adherence and physical-commonsense joint rate. Later models are on the VideoPhy-2 live board. VideoPhy: Evaluating Physical Commonsense for Video Generation (2024).

Model	Overall SA, PC (%)
CogVideoX-5B	39.6
Pika	19.7
VideoCrafter2	19.0
CogVideoX-2B	18.6
LaVIE	15.7
Dream Machine	13.6
Lumiere-T2I2V	12.5
ZeroScope	11.9

How to run

worldfoundry-eval zoo benchmark-run \
  --benchmark-id videophy \
  --mode official-validation \
  --official-results-path <official_results.json> \
  --generated-artifact-dir <generated_videos> \
  --output-dir tmp/videophy/official-validation \
  --json

${WORLDFOUNDRY_UNIFIED_PYTHON} \
  worldfoundry/evaluation/tasks/execution/runners/videophy/run_videophy_official_runner.py \
  --run-official \
  --generated-artifact-dir "${WORLDFOUNDRY_GENERATED_ARTIFACT_DIR}" \
  --videocon-checkpoint /path/to/videocon_physics \
  --output-dir tmp/videophy/official \
  --json

Env: WORLDFOUNDRY_VIDEOPHY_RESULTS_PATH, WORLDFOUNDRY_VIDEOPHY_JUDGE_BACKEND. Prompt list: worldfoundry/data/benchmarks/assets/videophy/prompts.json.

In-tree pointers: worldfoundry/data/benchmarks/tasks/external/videophy.yaml, worldfoundry/evaluation/tasks/execution/runners/videophy.

Outputs include scorecard.json (primary videophy_average).

← Benchmark Hub