AIGCBench

Last updated: Oct 7, 2026

On this page

Dimensions Data composition Metrics Leaderboard How to run

AIGCBench (BenchCouncil, ICT / CAS) evaluates image-to-video generation: given a condition image and a prompt, does the clip stay faithful, move, and stay temporally coherent? Metric families:

First-frame fidelity — mse_first / ssim_first vs the condition image.
Alignment — image–video and text–video CLIP.
Reference-video match — keyframe / corresponding-frame CLIP and SSIM (needs official references).
Motion — optical-flow square mean.
Temporality — adjacent-frame CLIP.
Perceptual quality — DOVER. frame_count is also recorded.

Paper: AIGCBench. Project: benchcouncil.org/AIGCBench. Code: BenchCouncil/AIGCBench (Apache-2.0). Dataset: stevenfan/AIGCBench_v1.0.

WorldFoundry imports official metric CSV / TSV (or pipeline score dumps) through the in-tree runner. Bundled prompt_suite.json under worldfoundry/data/benchmarks/assets/aigcbench can materialize requests without the full Hugging Face download. CLIP, DOVER, and flow scorers stay caller-supplied. Primary key: aigcbench_average when the official dump provides it.

Official paper teaser: AIGCBench’s three modules — dataset, metrics, and image-to-video models.

Dimensions

First-frame fidelity
KindAxis
MSE / SSIM between the conditioning image and the first generated frame.
Image / text alignment
KindAxis
CLIP similarity of the generated video to the condition image and the text prompt.
Reference-video match
KindAxis
CLIP / SSIM against official reference videos (keyframes and corresponding frames).
Motion strength
KindAxis
Optical-flow square mean as a motion-effect signal.
Temporal consistency
KindAxis
Adjacent-frame CLIP similarity.
Perceptual quality
KindAxis
DOVER video-quality score.

Data composition

Size
BenchCouncil image-to-video suite: condition images, text prompts, and reference videos. Bundled prompt_suite.json can materialize requests without the full Hugging Face download.
Sources
stevenfan/AIGCBench_v1.0 · github.com
Modalities
image · text · video
License
apache-2.0
Notes
WorldFoundry imports official metric CSV / TSV exports. CLIP, DOVER, and optical-flow scorers stay caller-supplied.

Metrics

Metric

mse_first
DirectionLower is better
AIGCBench MSE between the condition image and first generated frame; lower is better.
ssim_first
DirectionHigher is better
AIGCBench SSIM between the condition image and first generated frame.
image_genvideo_clip
DirectionHigher is better
AIGCBench CLIP similarity between condition image and generated video.
genvideo_text_clip
DirectionHigher is better
AIGCBench CLIP similarity between generated video and text prompt.
genvideo_refvideo_clip_keyframes
DirectionHigher is better
AIGCBench CLIP similarity between generated and reference video keyframes.
flow_square_mean
DirectionHigher is better
AIGCBench optical-flow square mean for generated video motion effects.
genvideo_refvideo_clip_corresponding_frames
DirectionHigher is better
AIGCBench CLIP similarity for generated and reference corresponding frames.
genvideo_clip_adjacent_frames
DirectionHigher is better
AIGCBench adjacent-frame CLIP similarity for temporal consistency.
frame_count
DirectionHigher is better
AIGCBench generated video frame count.
dover
DirectionHigher is better
AIGCBench DOVER video quality score.
genvideo_refvideo_ssim
DirectionHigher is better
AIGCBench SSIM between generated and reference videos.
aigcbench_average
DirectionHigher is better
AIGCBench official aggregate when supplied by the official scorer or normalized output.
PrimaryAIGCBench average

Leaderboard

Results from the paper (not a live board; not WorldFoundry-run scores). AIGCBench: Comprehensive Evaluation of Image-to-Video Content Generated by AI (2024).

Model	DOVER (video quality)
Gen2	0.775
Pika	0.715
SVD	0.623
VideoCrafter	0.518
I2VGen-XL	0.510

How to run

worldfoundry-eval zoo benchmark-run \
  --benchmark-id aigcbench \
  --mode official-validation \
  --official-results-path <official_results.csv> \
  --generated-artifact-dir <generated_videos> \
  --output-dir tmp/aigcbench/official-validation \
  --json

--strict fails on coverage gaps; --prompt-manifest restricts to a prompt subset. Reference-image / reference-video families need those official assets staged. Default import path: WORLDFOUNDRY_AIGCBENCH_RESULTS_PATH. Shared stack: video_quality_motion_stack.

In-tree pointers:

worldfoundry/data/benchmarks/tasks/external/aigcbench.yaml
worldfoundry/data/benchmarks/assets/aigcbench

Outputs include scorecard.json (primary aigcbench_average), benchmark_contract.json, and raw_metric_table.jsonl.

← Benchmark Hub