didi-lot1-ai/ai_platform/modules/video-analysis

Video Analysis

Video analysis service for deepfake detection using semantic analysis via vLLM backends.

Prerequisites

vLLM Server Requirement: This service does NOT start or manage a vLLM instance. You must have a vLLM-compatible server already running and accessible from the machine where this service is deployed.

• By default, the service expects vLLM at the URL set in VIDEO_ANALYSIS_VLLM_BASE_URL
• This can be changed via: VIDEO_ANALYSIS_VLLM_BASE_URL=http://:

Important:

• If you run video-analysis in Docker with network_mode: host, the vLLM server must be reachable from the host network.
• vLLM may run:
• locally on the same machine, or
• on another machine, as long as the URL is reachable.

The service will fail to start if the vLLM endpoint is not reachable.

Required:

• All global prerequisites (see main README.md)
• A running vLLM server with a vision-language model (e.g., BusterX)

vLLM Server Requirements:

• GPU required; VRAM depends on model (recommend ≥16GB, may require more).
• NVIDIA Driver 535+
• NVIDIA Container Toolkit

Note: video-analysis itself runs on CPU. The GPU is only needed for the vLLM server.

Features

• Deepfake Detection: Analyzes videos for signs of manipulation
• Uniform Frame Sampling: Extracts representative frames from videos
• vLLM Integration: Uses vision-language models for semantic analysis
• Reproducibility Artifacts (optional): Saves request/response artifacts to a configurable runs directory for debugging and reproducibility
  (this directory is created/used at runtime; it is not meant to be committed to the repo)

Installation

cd modules/video-analysis

# Install dependencies
uv sync

# Install with dev dependencies
uv sync --extra dev

Quick Start

As an API Server

cd deploy/

# Create deployment env file
# NOTE: deploy.sh loads env ONLY from deploy/.env
cp ../.env.example .env
# Edit deploy/.env with your vLLM server URL and model

# Start the server
./deploy.sh --profile api --detach

# Or with nginx reverse proxy
./deploy.sh --profile api-nginx --detach

API Endpoints

Endpoint	Method	Description
/health	GET	Health check
/analyze/video	POST	Deepfake detection (fast, 16 frames)
/analyze/video/semantic	POST	Semantic analysis (detailed, 144+ frames)

Example API Request

# Health check
curl http://localhost:54600/health

# Analyze video
VIDEO="/path/to/video.mp4"
curl -sS -X POST "http://localhost:54600/analyze/video" -F "file=@${VIDEO}"

Example Response

{
  "request_id": "550e8400-e29b-41d4-a716-446655440000",
  "run_dir": "runs/550e8400-e29b-41d4-a716-446655440000",
  "verdict": "FAKE",
  "explanation": "The video shows clear signs of manipulation...",
  "usage": {
    "prompt_tokens": 1250,
    "completion_tokens": 150,
    "total_tokens": 1400
  },
  "latency_s": {
    "sampling_time_s": 0.234,
    "encode_time_s": 0.567,
    "model_inference_time_s": 12.345
  },
  "meta": {
    "fps": 30.0,
    "total_frames": 450,
    "duration_s": 15.0,
    "sampled": 16
  }
}

Configuration

Required Environment Variables

Configured via environment variables (prefix: VIDEO_ANALYSIS_). These are typically set in deploy/.env:

Variable	Description
VIDEO_ANALYSIS_VLLM_BASE_URL	vLLM server URL (e.g., http://didiAI-video-vllm-buster:8000)
VIDEO_ANALYSIS_VLLM_MODEL	Model name (e.g., l8cv/BusterX_plusplus)
VIDEO_ANALYSIS_RUNS_DIR	Directory for storing analysis artifacts (created/used at runtime)

| VIDEO_ANALYSIS_EXTERNAL_URL | External URL for OpenAPI spec (e.g., http://localhost:54600) |

Optional Nginx Environment Variables (api-nginx profile)

If you use the api-nginx profile, the nginx container can read these optional variables from deploy/.env:

Variable	Example	Description
NGINX_CONNECT_TIMEOUT	60s	Upstream connect timeout
NGINX_SEND_TIMEOUT	120s	Upstream send timeout
NGINX_READ_TIMEOUT	600s	Upstream read timeout

Optional Tuning Parameters

Configured via deploy/config.yaml (env vars override YAML):

Parameter	Default	Description
frames	16	Number of frames to sample
max_side	960	Max image dimension (pixels)
jpeg_quality	85	JPEG encoding quality (1-100)
max_tokens	750	Max tokens for model response
temperature	0.000001	Sampling temperature
repetition_penalty	1.05	Repetition penalty

⚠️ Testing Recommendations

Model Selection for Semantic Analysis

Current Setup:

• Both deepfake and semantic analysis use BusterX (7B parameters)
• BusterX is optimized for deepfake detection

TODO: Test Semantic Analysis with Qwen3-VL-30B

For better semantic understanding, consider testing with the larger Qwen3-VL-30B model (already running @ port 8102):

# Current (BusterX 7B)
VIDEO_ANALYSIS_VLLM_BASE_URL=http://didiAI-video-vllm-buster:54500
VIDEO_ANALYSIS_VLLM_MODEL=busterx

# Alternative (Qwen3-VL 30B) - Better for semantic analysis
VIDEO_ANALYSIS_VLLM_BASE_URL=http://didiAI-llm-vllm-vision:14002
VIDEO_ANALYSIS_VLLM_MODEL=qwen3-vl

Expected Improvements:

• ✅ More detailed scene descriptions (30B vs 7B)
• ✅ Better understanding of complex actions
• ✅ More coherent narrative synthesis
• ✅ Higher quality semantic annotations

Trade-offs:

• ⏱️ Slightly higher latency (~15-20s per chunk vs ~12s)
• 📊 Better for semantic analysis, but keep BusterX for deepfake detection

Recommendation:

• Deepfake endpoint: Keep using BusterX (specialized for forgery detection)
• Semantic endpoint: Test with Qwen3-VL-30B for better results

Deployment

cd deploy/

# Create deployment env file (REQUIRED)
cp ../.env.example .env
# Edit deploy/.env with your settings

# API only
./deploy.sh --profile api --detach

# API with nginx reverse proxy
./deploy.sh --profile api-nginx --detach

# View logs
./deploy.sh --profile api --logs

# Stop services
./deploy.sh --profile api --down

Common Docker Commands

cd deploy/

# Restart the API service (compose service name)
docker compose restart video-analysis-api

# (Optional) restart by container name
docker restart video_analysis

Port Allocation

Port	Service	Environment
54600	Video Analysis API	Development
54500	BusterX vLLM Server	Development

Development

# Install dev dependencies
uv sync --extra dev

# Run tests
uv run pytest

# Run tests with coverage
uv run pytest --cov=src/video_analysis --cov-report=term-missing

# Lint and format
uv run ruff check .
uv run ruff format .

Architecture

modules/video-analysis/
├── deploy/
│   ├── deploy.sh             # Deployment script
│   ├── docker-compose.yml    # Docker services
│   ├── Dockerfile            # Container image
│   ├── nginx.conf            # Nginx reverse proxy config (optional)
│   ├── nginx.conf.template   # Template-based nginx config (optional)
│   └── config.yaml           # Tuning parameters
├── src/video_analysis/
│   ├── __init__.py
│   ├── app.py                # FastAPI application
│   ├── buster_client.py      # vLLM client
│   ├── schemas.py            # Response schemas
│   ├── settings.py           # Configuration (env + yaml)
│   └── video_sampling.py     # Frame extraction
├── tests/
├── .env.example              # Environment template
├── API.md                    # API documentation
├── pyproject.toml            # Dependencies
└── README.md                 # This file