didi-lot1-ai/ai_platform/modules/video-analysis/README.md

# Video Analysis

Video analysis service for deepfake detection using semantic analysis via vLLM backends.

## Prerequisites

**vLLM Server Requirement:**
This service **does NOT start or manage a vLLM instance**.
You must have a **vLLM-compatible server already running and accessible**
from the machine where this service is deployed.

- By default, the service expects vLLM at the URL set in VIDEO_ANALYSIS_VLLM_BASE_URL
- This can be changed via: VIDEO_ANALYSIS_VLLM_BASE_URL=http://<host>:<port>

**Important:**
- If you run `video-analysis` in Docker with `network_mode: host`,
the vLLM server must be reachable from the host network.
- vLLM may run:
- locally on the same machine, **or**
- on another machine, as long as the URL is reachable.

The service will **fail to start** if the vLLM endpoint is not reachable.

**Required:**
- All global prerequisites (see main [README.md](../../README.md))
- A running vLLM server with a vision-language model (e.g., BusterX)

**vLLM Server Requirements:**
- GPU required; VRAM depends on model (recommend ≥16GB, may require more).
- NVIDIA Driver 535+
- NVIDIA Container Toolkit

> **Note:** `video-analysis` itself runs on CPU. The GPU is only needed for the vLLM server.

## Features

- **Deepfake Detection**: Analyzes videos for signs of manipulation
- **Uniform Frame Sampling**: Extracts representative frames from videos
- **vLLM Integration**: Uses vision-language models for semantic analysis
- **Reproducibility Artifacts (optional)**: Saves request/response artifacts to a configurable runs directory for debugging and reproducibility  
  (this directory is created/used at runtime; it is not meant to be committed to the repo)

## Installation

```bash
cd modules/video-analysis

# Install dependencies
uv sync

# Install with dev dependencies
uv sync --extra dev
```

## Quick Start

### As an API Server

```bash
cd deploy/

# Create deployment env file
# NOTE: deploy.sh loads env ONLY from deploy/.env
cp ../.env.example .env
# Edit deploy/.env with your vLLM server URL and model

# Start the server
./deploy.sh --profile api --detach

# Or with nginx reverse proxy
./deploy.sh --profile api-nginx --detach
```

### API Endpoints

| Endpoint | Method | Description |
|----------|--------|-------------|
| `/health` | GET | Health check |
| `/analyze/video` | POST | Deepfake detection (fast, 16 frames) |
| `/analyze/video/semantic` | POST | Semantic analysis (detailed, 144+ frames) |
| `/v1/info` | GET | Service catalog metadata (used by catalog-api) |

### Example API Request

```bash
# Health check
curl http://localhost:54600/health

# Analyze video
VIDEO="/path/to/video.mp4"
curl -sS -X POST "http://localhost:54600/analyze/video" -F "file=@${VIDEO}"
```

### Example Response

```json
{
  "request_id": "550e8400-e29b-41d4-a716-446655440000",
  "run_dir": "runs/550e8400-e29b-41d4-a716-446655440000",
  "verdict": "FAKE",
  "explanation": "The video shows clear signs of manipulation...",
  "usage": {
    "prompt_tokens": 1250,
    "completion_tokens": 150,
    "total_tokens": 1400
  },
  "latency_s": {
    "sampling_time_s": 0.234,
    "encode_time_s": 0.567,
    "model_inference_time_s": 12.345
  },
  "meta": {
    "fps": 30.0,
    "total_frames": 450,
    "duration_s": 15.0,
    "sampled": 16
  }
}
```

## Configuration

### Required Environment Variables

Configured via environment variables (prefix: `VIDEO_ANALYSIS_`). These are typically set in `deploy/.env`:

| Variable | Description |
|----------|-------------|
| `VIDEO_ANALYSIS_VLLM_BASE_URL` | vLLM server URL (e.g., `http://didiAI-video-vllm-buster:54500`) |
| `VIDEO_ANALYSIS_VLLM_MODEL` | Served model name (e.g., `busterx`) |
| `VIDEO_ANALYSIS_RUNS_DIR` | Directory for storing analysis artifacts (created/used at runtime) |

| `VIDEO_ANALYSIS_EXTERNAL_URL` | External URL for OpenAPI spec (e.g., `http://localhost:54600`) |


### Optional Nginx Environment Variables (api-nginx profile)

If you use the `api-nginx` profile, the nginx container can read these optional variables from `deploy/.env`:

| Variable | Example | Description |
|----------|---------|-------------|
| `NGINX_CONNECT_TIMEOUT` | `60s` | Upstream connect timeout |
| `NGINX_SEND_TIMEOUT` | `120s` | Upstream send timeout |
| `NGINX_READ_TIMEOUT` | `600s` | Upstream read timeout |

### Optional Tuning Parameters

These default to the values below in `settings.py` and are overridden via the matching `VIDEO_ANALYSIS_*` environment variables (a `deploy/config.yaml` may optionally be supplied to override defaults, but none ships with the module):

| Parameter | Default | Description |
|-----------|---------|-------------|
| `frames` | `16` | Number of frames to sample |
| `max_side` | `960` | Max image dimension (pixels) |
| `jpeg_quality` | `85` | JPEG encoding quality (1-100) |
| `max_tokens` | `750` | Max tokens for model response |
| `temperature` | `0.000001` | Sampling temperature |
| `repetition_penalty` | `1.05` | Repetition penalty |

## Semantic Analysis Pipeline

Both endpoints use the **same** BusterX vLLM (`busterx` @ port `54500`):

- **Deepfake endpoint** — BusterX returns the `REAL` / `FAKE` / `UNCERTAIN` verdict + explanation.
- **Semantic endpoint** — BusterX produces a per-chunk `description` for each temporal chunk. When `enable_aggregation=true`, those chunk descriptions are merged into a single narrative `final_summary` by the DIDI text LLM (Qwen3.5) via `http://didiAI-llm-api:14011` (set through `VIDEO_ANALYSIS_SEMANTIC_LLM_BASE_URL`).

BusterX is self-contained — Qwen2.5-VL is bundled inside the `l8cv/BusterX_plusplus` fine-tune, so no separate vision base model is loaded. There is no separate Qwen3-VL vision backend in this deployment; the only vision model the service talks to is BusterX.

```bash
# Vision backend (deepfake + semantic chunk descriptions)
VIDEO_ANALYSIS_VLLM_BASE_URL=http://didiAI-video-vllm-buster:54500
VIDEO_ANALYSIS_VLLM_MODEL=busterx

# Text LLM used only to aggregate semantic chunks into a narrative summary
VIDEO_ANALYSIS_SEMANTIC_LLM_BASE_URL=http://didiAI-llm-api:14011
```

## Deployment

```bash
cd deploy/

# Create deployment env file (REQUIRED)
cp ../.env.example .env
# Edit deploy/.env with your settings

# API only
./deploy.sh --profile api --detach

# API with nginx reverse proxy
./deploy.sh --profile api-nginx --detach

# View logs
./deploy.sh --profile api --logs

# Stop services
./deploy.sh --profile api --down
```

### Common Docker Commands

```bash
cd deploy/

# Restart the API service (compose service name)
docker compose restart video-analysis-api

# (Optional) restart by container name
docker restart video_analysis
```

### Port Allocation

| Port | Service | Environment |
|------|---------|-------------|
| `54600` | Video Analysis API | Development |
| `54500` | BusterX vLLM Server | Development |

## Development

```bash
# Install dev dependencies
uv sync --extra dev

# Run tests
uv run pytest

# Run tests with coverage
uv run pytest --cov=src/video_analysis --cov-report=term-missing

# Lint and format
uv run ruff check .
uv run ruff format .
```

## Architecture

```
modules/video-analysis/
├── deploy/
│   ├── deploy.sh             # Deployment script
│   ├── docker-compose.yml    # Docker services
│   ├── Dockerfile            # Container image
│   ├── nginx.conf            # Nginx reverse proxy config (optional)
│   └── nginx.conf.template   # Template-based nginx config (optional)
├── src/video_analysis/
│   ├── __init__.py
│   ├── app.py                # FastAPI application
│   ├── buster_client.py      # vLLM client
│   ├── schemas.py            # Response schemas
│   ├── settings.py           # Configuration (env + yaml)
│   └── video_sampling.py     # Frame extraction
├── tests/
├── .env.example              # Environment template
├── API.md                    # API documentation
├── pyproject.toml            # Dependencies
└── README.md                 # This file
```