didi-lot1-ai/ai_platform/modules/video-analysis

Video Analysis

Video analysis service for deepfake detection using semantic analysis via vLLM backends.

Prerequisites

vLLM Server Requirement: This service does NOT start or manage a vLLM instance. You must have a vLLM-compatible server already running and accessible from the machine where this service is deployed.

• By default, the service expects vLLM at the URL set in VIDEO_ANALYSIS_VLLM_BASE_URL
• This can be changed via: VIDEO_ANALYSIS_VLLM_BASE_URL=http://:

Important:

• If you run video-analysis in Docker with network_mode: host, the vLLM server must be reachable from the host network.
• vLLM may run:
• locally on the same machine, or
• on another machine, as long as the URL is reachable.

The service will fail to start if the vLLM endpoint is not reachable.

Required:

• All global prerequisites (see main README.md)
• A running vLLM server with a vision-language model (e.g., BusterX)

vLLM Server Requirements:

• GPU required; VRAM depends on model (recommend ≥16GB, may require more).
• NVIDIA Driver 535+
• NVIDIA Container Toolkit

Note: video-analysis itself runs on CPU. The GPU is only needed for the vLLM server.

Features

• Deepfake Detection: Analyzes videos for signs of manipulation
• Uniform Frame Sampling: Extracts representative frames from videos
• vLLM Integration: Uses vision-language models for semantic analysis
• Reproducibility Artifacts (optional): Saves request/response artifacts to a configurable runs directory for debugging and reproducibility
  (this directory is created/used at runtime; it is not meant to be committed to the repo)

Installation

cd modules/video-analysis

# Install dependencies
uv sync

# Install with dev dependencies
uv sync --extra dev

Quick Start

As an API Server

cd deploy/

# Create deployment env file
# NOTE: deploy.sh loads env ONLY from deploy/.env
cp ../.env.example .env
# Edit deploy/.env with your vLLM server URL and model

# Start the server
./deploy.sh --profile api --detach

# Or with nginx reverse proxy
./deploy.sh --profile api-nginx --detach

API Endpoints

Endpoint	Method	Description
/health	GET	Health check
/analyze/video	POST	Deepfake detection (fast, 16 frames)
/analyze/video/semantic	POST	Semantic analysis (detailed, 144+ frames)
/v1/info	GET	Service catalog metadata (used by catalog-api)

Example API Request

# Health check
curl http://localhost:54600/health

# Analyze video
VIDEO="/path/to/video.mp4"
curl -sS -X POST "http://localhost:54600/analyze/video" -F "file=@${VIDEO}"

Example Response

{
  "request_id": "550e8400-e29b-41d4-a716-446655440000",
  "run_dir": "runs/550e8400-e29b-41d4-a716-446655440000",
  "verdict": "FAKE",
  "explanation": "The video shows clear signs of manipulation...",
  "usage": {
    "prompt_tokens": 1250,
    "completion_tokens": 150,
    "total_tokens": 1400
  },
  "latency_s": {
    "sampling_time_s": 0.234,
    "encode_time_s": 0.567,
    "model_inference_time_s": 12.345
  },
  "meta": {
    "fps": 30.0,
    "total_frames": 450,
    "duration_s": 15.0,
    "sampled": 16
  }
}

Configuration

Required Environment Variables

Configured via environment variables (prefix: VIDEO_ANALYSIS_). These are typically set in deploy/.env:

Variable	Description
VIDEO_ANALYSIS_VLLM_BASE_URL	vLLM server URL (e.g., http://didiAI-video-vllm-buster:54500)
VIDEO_ANALYSIS_VLLM_MODEL	Served model name (e.g., busterx)
VIDEO_ANALYSIS_RUNS_DIR	Directory for storing analysis artifacts (created/used at runtime)

| VIDEO_ANALYSIS_EXTERNAL_URL | External URL for OpenAPI spec (e.g., http://localhost:54600) |

Optional Nginx Environment Variables (api-nginx profile)

If you use the api-nginx profile, the nginx container can read these optional variables from deploy/.env:

Variable	Example	Description
NGINX_CONNECT_TIMEOUT	60s	Upstream connect timeout
NGINX_SEND_TIMEOUT	120s	Upstream send timeout
NGINX_READ_TIMEOUT	600s	Upstream read timeout

Optional Tuning Parameters

These default to the values below in settings.py and are overridden via the matching VIDEO_ANALYSIS_* environment variables (a deploy/config.yaml may optionally be supplied to override defaults, but none ships with the module):

Parameter	Default	Description
frames	16	Number of frames to sample
max_side	960	Max image dimension (pixels)
jpeg_quality	85	JPEG encoding quality (1-100)
max_tokens	750	Max tokens for model response
temperature	0.000001	Sampling temperature
repetition_penalty	1.05	Repetition penalty

Semantic Analysis Pipeline

Both endpoints use the same BusterX vLLM (busterx @ port 54500):

• Deepfake endpoint — BusterX returns the REAL / FAKE / UNCERTAIN verdict + explanation.
• Semantic endpoint — BusterX produces a per-chunk description for each temporal chunk. When enable_aggregation=true, those chunk descriptions are merged into a single narrative final_summary by the DIDI text LLM (Qwen3.5) via http://didiAI-llm-api:14011 (set through VIDEO_ANALYSIS_SEMANTIC_LLM_BASE_URL).

BusterX is self-contained — Qwen2.5-VL is bundled inside the l8cv/BusterX_plusplus fine-tune, so no separate vision base model is loaded. There is no separate Qwen3-VL vision backend in this deployment; the only vision model the service talks to is BusterX.

# Vision backend (deepfake + semantic chunk descriptions)
VIDEO_ANALYSIS_VLLM_BASE_URL=http://didiAI-video-vllm-buster:54500
VIDEO_ANALYSIS_VLLM_MODEL=busterx

# Text LLM used only to aggregate semantic chunks into a narrative summary
VIDEO_ANALYSIS_SEMANTIC_LLM_BASE_URL=http://didiAI-llm-api:14011

Deployment

cd deploy/

# Create deployment env file (REQUIRED)
cp ../.env.example .env
# Edit deploy/.env with your settings

# API only
./deploy.sh --profile api --detach

# API with nginx reverse proxy
./deploy.sh --profile api-nginx --detach

# View logs
./deploy.sh --profile api --logs

# Stop services
./deploy.sh --profile api --down

Common Docker Commands

cd deploy/

# Restart the API service (compose service name)
docker compose restart video-analysis-api

# (Optional) restart by container name
docker restart video_analysis

Port Allocation

Port	Service	Environment
54600	Video Analysis API	Development
54500	BusterX vLLM Server	Development

Development

# Install dev dependencies
uv sync --extra dev

# Run tests
uv run pytest

# Run tests with coverage
uv run pytest --cov=src/video_analysis --cov-report=term-missing

# Lint and format
uv run ruff check .
uv run ruff format .

Architecture

modules/video-analysis/
├── deploy/
│   ├── deploy.sh             # Deployment script
│   ├── docker-compose.yml    # Docker services
│   ├── Dockerfile            # Container image
│   ├── nginx.conf            # Nginx reverse proxy config (optional)
│   └── nginx.conf.template   # Template-based nginx config (optional)
├── src/video_analysis/
│   ├── __init__.py
│   ├── app.py                # FastAPI application
│   ├── buster_client.py      # vLLM client
│   ├── schemas.py            # Response schemas
│   ├── settings.py           # Configuration (env + yaml)
│   └── video_sampling.py     # Frame extraction
├── tests/
├── .env.example              # Environment template
├── API.md                    # API documentation
├── pyproject.toml            # Dependencies
└── README.md                 # This file