PhyFPS-Bench-Gen

Last updated: Oct 7, 2026

On this page

Dimensions Data composition Metrics Leaderboard How to run

PhyFPS-Bench-Gen (TACO Group, Texas A&M) audits chronometric hallucination: whether a generator's declared container frame rate (Meta FPS) matches the physical speed implied by the motion (PhyFPS, from Visual Chronometer). It is not a law, grounding, education, or continuation bench.

The suite is 100 prompts. Name outputs 0001.mp4 … 0100.mp4. The predictor runs on sliding windows (--stride, --clip-length); the scorecard reports Meta-vs-PhyFPS error plus intra- and inter-video coefficients of variation. Every metric is lower-is-better.

Code: taco-group/Visual_Chronometer. Checkpoint: xiangbog/Visual_Chronometer (vc_common_10_60fps.ckpt).

WorldFoundry runs Visual Chronometer predict.py in tree and/or normalizes a caller results.csv. Alignment metrics need WORLDFOUNDRY_PHYFPS_META_FPS (or a manifest).

Official paper teaser: chronometric hallucination — generated motion that does not match Meta FPS.

Dimensions

Chronometric FPS audit
KindTask
Does the generator's declared Meta FPS match the PhyFPS implied by motion (Visual Chronometer)? Not a physics-law or commonsense generation bench.
Meta vs PhyFPS
KindAxis
avg_error_fps and pct_error need WORLDFOUNDRY_PHYFPS_META_FPS (or a manifest).
FPS stability
KindAxis
Intra-video and inter-video coefficients of variation over sliding-window PhyFPS.

Data composition

Size
100 prompts; one generated video per prompt named 0001.mp4 … 0100.mp4. WorldFoundry runs Visual Chronometer predict.py in tree (or imports results.csv). VC-Common checkpoint: xiangbog/Visual_Chronometer.
Sources
github.com · xiangbog/Visual_Chronometer
Modalities
text · video
Notes
All metrics are lower-is-better. Alignment metrics are undefined without Meta FPS.

Metrics

Metric

avg_error_fps
DirectionLower is better
Mean absolute gap between Meta FPS and average PhyFPS.
pct_error
DirectionLower is better
Mean relative Meta-vs-PhyFPS error percentage.
inter_video_cv
DirectionLower is better
Cross-video coefficient of variation over average PhyFPS.
intra_video_cv
DirectionLower is better
Mean within-video coefficient of variation over sliding-window clips.
phyfps_bench_gen_average
DirectionLower is better
Mean over available PhyFPS-Bench-Gen families (lower is better).
PrimaryPhyFPS-Bench-Gen Average

Leaderboard

Results from the paper (not a live board; not WorldFoundry-run scores). Table 1 Avg. Error column. Authors also host a submission board that had no public rows at ingest time. The Pulse of Motion: Measuring Physical Frame Rate from Visual Dynamics (2026).

Model	Avg. error vs Meta FPS (lower is better)
Wan2.1-T2V-1.3B	7.54
Seedance-1.0-Liteclosed	8.31
Sora-2closed	8.40
Kling-o3closed	9.10
Seedance-1.5-Proclosed	10.67
Wan2.2-T2V-A14B	10.74
Wan2.1-T2V-14B	10.87
Wan2.2-TI2V-5B	11.63

How to run

Import an existing results.csv:

worldfoundry-eval zoo benchmark-run \
  --benchmark-id phyfps-bench-gen \
  --mode official-validation \
  --official-results-path <results.csv> \
  --generated-artifact-dir <generated_videos> \
  --output-dir tmp/phyfps-bench-gen/official-validation \
  --json

Predict PhyFPS in tree (needs the VC-Common checkpoint and Meta FPS for alignment):

export WORLDFOUNDRY_PHYFPS_META_FPS=24
${WORLDFOUNDRY_UNIFIED_PYTHON} \
  worldfoundry/evaluation/tasks/execution/runners/phyfps_bench_gen/run_phyfps_bench_gen_official_runner.py \
  --run-official \
  --generated-artifact-dir "${WORLDFOUNDRY_GENERATED_ARTIFACT_DIR}" \
  --stride 4 \
  --output-dir tmp/phyfps_bench_gen/official \
  --json

Outputs include scorecard.json (primary phyfps_bench_gen_average) and results.csv.

← Benchmark Hub