AI video classification: Phase 1 overview#
Reviewed 2 October 2026. Scope: classifying existing videos, detecting specified or unwanted content, and checking consistency in recorded, edited, or AI-generated footage. This is a literature overview; no models were tested for this report.
What needs to be classified?#
The sources suggest separating content detection, policy decisions, and consistency evaluation: Amazon Rekognition supplies moderation labels, SafeWatch investigates following custom safety policies, and VBench evaluates dimensions of generated-video quality. These are related but distinct tasks, rather than interchangeable meanings of “video classification.” [1] [2] [3]
| Task | Example question for this research | Relevant approach in the sources |
|---|---|---|
| Specified content | Does the video contain smoking, weapons, or nudity? | A detector with an explicit label taxonomy, such as Rekognition. [1] |
| Unwanted content under a policy | Does this scene violate our particular publication rules? | Policy-conditioned classification and explanations, as investigated by SafeWatch. [2] |
| Visual continuity | Does the subject remain consistent, or does the video flicker? | Separate consistency and temporal-quality measurements, as in VBench. [3] |
| Agreement with an input | Does generated footage match its prompt or reference image? | Video-condition and image-to-video evaluation, as in VBench and VBench++. [3] |
For this topic, “consistency” will include continuity within a video and agreement with a supplied brief or reference. Fact-checking a video’s claims is outside this initial scope.
Fixed detectors: a practical baseline with explicit boundaries#
Rekognition’s documented taxonomy includes explicit sexual content, violence, weapons, drugs and tobacco, alcohol, and hate symbols, with confidence scores attached to detected labels. This makes it a concrete candidate when the wanted categories already match its definitions. [1] Google’s Video Intelligence explicit-content detector illustrates a narrower scope: its assessment uses visual frames and excludes the audio track. An “explicit content” result from that feature therefore does not assess spoken content. [4]
AWS authors Lana Zhang and Brigit Brown recommend the managed video moderation API for most of the scenarios in their 2023 comparison, citing orchestration, processing-time, and cost advantages over calling the image API on extracted frames. Their comparison used 200 videos; it is evidence about that setup, not a current cross-provider ranking. [5]
An important detail in that post is that AWS used the same underlying image moderation model for both approaches, with additional filtering and video-processing orchestration in the video API. The authors found similar accuracy at the same sampling frequency. This is a concrete example of why accepting a video file does not itself demonstrate reasoning about event order or continuity. [5]
Multimodal models: flexible questions and custom rules#
Google documents Gemini video understanding for questions involving visual and audio information, with timestamps and controls over video processing. This provides an alternative to selecting from a fixed moderation taxonomy: the application can ask a task-specific question about the footage. [6] For an open-weight route, the Qwen team’s Qwen2.5-VL release demonstrates locating events in videos and returning structured descriptions with start and end times. Those demonstrations establish candidate capabilities, not measured accuracy on a particular unwanted-content policy. [7]
TwelveLabs argues that video understanding should combine visual, audio, and on-screen text signals across time, rather than depend on isolated images or transcripts. Its discussion distinguishes retrieval through video embeddings from generative analysis; this is the vendor’s architectural position, not independent proof of superiority. [8] Its compliance product applies customer-controlled rules and presents timestamped findings for reviewers to accept, reject, or annotate. TwelveLabs reports 92.5% agreement with a manual baseline on an internal corpus of 53 assets; the small, vendor-run evaluation does not establish performance on other libraries or policies. [9]
SafeWatch’s researchers similarly argue for configurable policies rather than a small, fixed collection of unsafe labels. Their model produces multiple policy flags with explanations, and their benchmark includes real and generated videos across six policy categories and more than 30 scenarios. The project describes its large dataset as annotated by multiple LLM agents; its reported gains should therefore be read with attention to how labels and evaluation subsets were constructed, rather than as a universal safety guarantee. [2]
The synthesis for this research is to distinguish an observed event from the policy judgment applied to it, and preserve evidence for review. SafeWatch’s policy-conditioned outputs and TwelveLabs’ reviewer workflow provide two examples of this separation. [2] [9]
Consistency requires more than one score#
VBench’s authors decompose generated-video evaluation into 16 dimensions, including subject identity inconsistency, motion smoothness, temporal flickering, and spatial relationships, and compare their measurements with human preferences. Their approach supports asking which property failed instead of relying only on an overall quality score. [3]
VBench-2.0 extends evaluation toward human fidelity, controllability, creativity, physics, and commonsense. Its authors argue that visual appearance alone does not capture whether generated footage behaves plausibly. [10] These benchmarks offer useful starting points for continuity checks, but applying their scores to edited documentaries, advertisements, or other recorded footage would be an extrapolation beyond their stated video-generation evaluation purpose. [3] [10]
Coverage and evaluation determine whether the result is useful#
Sampling can miss the moment that matters. AWS explicitly warns that widening frame intervals increases the risk of missing inappropriate content between samples. [5] Google documents a default static processing rate of one frame per second and warns that fast action can lose detail; it also documents adjustable sampling and an adaptive mode on supported models. Thus, for this research, “no violation detected” should be interpreted in light of the actual coverage, rather than as proof that every moment was checked. [6]
Modality coverage also needs to be explicit. Google’s dedicated explicit-content detector ignores audio, while Gemini’s video interface processes audio as well as visuals. The inference is that a test for spoken violations needs a path that actually examines speech, even if the same system already checks the pictures. [4] [6]
AWS’s evaluation guidance recommends comparing predictions against human labels on representative data and selecting confidence thresholds for the application’s precision–recall tradeoff. Lower thresholds can catch more violations while also flagging more acceptable content. [11] Its illustrative dataset shows why overall accuracy is insufficient: with rare violations, a detector can have a low false-positive rate yet produce many false alarms relative to true findings. [11]
For the next phase, this suggests evaluating each target category, recording missed violations and false alarms, and measuring how much flagged material reviewers must inspect. AWS describes threshold evaluation and human review, while TwelveLabs illustrates a timestamped review queue; together they motivate evaluating the whole review workflow, beyond the model’s headline score. [11] [9]
Questions for Phase 2#
These are proposed research directions, not implementation decisions:
- Which unwanted categories and contextual exceptions matter for the target library, and do existing taxonomies cover them? [1] [2]
- Can a prompted multimodal model reliably apply those rules, compared with a dedicated moderation detector? [1] [6] [7]
- Which meaning of consistency matters most: subject continuity, motion and physical plausibility, or agreement with a brief/reference? [3] [10]
- How much does detection change with sampling rate, event duration, and access to audio? [4] [5] [6]
- At an acceptable missed-violation rate, what are the false-alarm burden and reviewer workload on representative footage? [11]