T2VSafetyBench
T2VSafetyBench (CAS and Tsinghua, with RealAI) asks whether a text-to-video model refuses unsafe prompts, including risks that only appear across frames. Every official metric is an NSFW rate (lower is safer). The prompt suite has 14 aspects; WorldFoundry reports 12 rates: eleven content aspects plus a collapsed temporal-risk rate (sequential action, dynamic variation, coherent context). Prompts come from real-world text, LLM generation, and jailbreak attacks. Videos are judged NSFW / safe per aspect by GPT, with published correlation to manual review.
Paper: T2VSafetyBench. Code / prompts: yibo-miao/T2VSafetyBench. There is no required Hugging Face dataset.
WorldFoundry vendors the judge runtime and an importer. Import mode only normalizes official result files. Judge mode stages generated videos against a selected prompt class and calls the checked-in GPT evaluation prompt. A mocked API is never treated as metric evidence. Primary key: nsfw_average.
Dimensions
- Sexual contentKindCategory
Pornography and borderline-pornography NSFW rates on malicious sexual prompts.
- Violence and goreKindCategory
Violence, gore, and disturbing-content NSFW rates.
- Identity and rightsKindCategory
Public-figure likeness misuse and copyright / trademark violation rates.
- Social and legalKindCategory
Discrimination, political sensitivity, and illegal-activity NSFW rates.
- MisinformationKindCategory
Falsehood and fabricated-news NSFW rate.
- Temporal riskKindAxis
Video-only risk: frames look safe but the sequence is unsafe (sequential action, dynamic variation, coherent context).
Data composition
- Size
Official 14-aspect malicious prompt suite from the T2VSafetyBench repo (real-world, LLM-generated, and jailbreak prompts). WorldFoundry reports 12 NSFW rates: 11 content aspects plus a collapsed temporal-risk rate.
- Sources
github.com
- Modalities
text · video
- License
upstream_terms
- Notes
No Hugging Face dataset is required. Judge mode needs a GPT key; import mode only normalizes official result files.
Metrics
Metric
- DirectionLower is better
pornography_nsfw_rateShare of generated videos judged NSFW for explicit sexual content.
- DirectionLower is better
borderline_pornography_nsfw_rateShare judged NSFW for sexualized but not fully explicit content.
- DirectionLower is better
violence_nsfw_rateShare judged NSFW for physical aggression or cruelty.
- DirectionLower is better
gore_nsfw_rateShare judged NSFW for blood or severe injury.
- DirectionLower is better
public_figures_nsfw_rateShare judged NSFW for unauthorized public-figure likeness.
- DirectionLower is better
discrimination_nsfw_rateShare judged NSFW for group-based demeaning content.
- DirectionLower is better
political_sensitivity_nsfw_rateShare judged NSFW for politically sensitive or hate-symbol content.
- DirectionLower is better
illegal_activities_nsfw_rateShare judged NSFW for depicted criminal activity.
- DirectionLower is better
disturbing_content_nsfw_rateShare judged NSFW for grotesque or horror content.
- DirectionLower is better
misinformation_falsehoods_nsfw_rateShare judged NSFW for fabricated news or falsehoods.
- DirectionLower is better
copyright_trademark_nsfw_rateShare judged NSFW for unauthorized brand or copyrighted material.
- DirectionLower is better
temporal_risk_nsfw_rateShare judged NSFW only when the sequence, not a single frame, is unsafe.
- DirectionLower is better
nsfw_averageMean of the twelve aspect NSFW rates. Lower is safer.
PrimaryNSFW average
Leaderboard
Results from the paper (not a live board; not WorldFoundry-run scores). Paper NSFW Average, human column. Lower unsafe-generation rate is better. T2VSafetyBench: Evaluating the Safety of Text-to-Video Generative Models (2024).
| Model | Human NSFW average (lower is safer) |
|---|---|
| Gen2 | 40.9% |
| SVD | 46.8% |
| Open-Sora | 52.1% |
| Pika | 59.0% |
How to run
Import official aspect rates:
worldfoundry-eval zoo benchmark-run \
--benchmark-id t2v-safety-bench \
--mode official-validation \
--official-results-path <official_results.json> \
--generated-artifact-dir <generated_videos> \
--output-dir tmp/t2v-safety-bench/official-validation \
--jsonJudge mode needs OPENAI_API_KEY or WORLDFOUNDRY_T2V_SAFETY_BENCH_GPT_API_KEY, plus generated videos under WORLDFOUNDRY_GENERATED_ARTIFACT_DIR. Optional: WORLDFOUNDRY_T2V_SAFETY_BENCH_ROOT, WORLDFOUNDRY_T2V_SAFETY_BENCH_GPT_MODEL, --classes / --prompt-path, --limit for bounded wiring runs. Default import path: WORLDFOUNDRY_T2V_SAFETY_BENCH_RESULTS_PATH.
In-tree pointers:
worldfoundry/data/benchmarks/tasks/external/t2v-safety-bench.yaml
worldfoundry/evaluation/tasks/execution/runners/t2v_safety_bench/runtime/t2v_safety_bench
worldfoundry/data/benchmarks/assets/t2v-safety-bench/Outputs include scorecard.json (primary nsfw_average), benchmark_contract.json, and raw_metric_table.jsonl.