didi-lot1-ai/ai_platform/modules/video-analysis/README.md

# Video Analysis

Video analysis service for deepfake detection using semantic analysis via vLLM backends.

## Prerequisites

**vLLM Server Requirement:**
This service **does NOT start or manage a vLLM instance**.
You must have a **vLLM-compatible server already running and accessible**
from the machine where this service is deployed.

- By default, the service expects vLLM at the URL set in VIDEO_ANALYSIS_VLLM_BASE_URL
- This can be changed via: VIDEO_ANALYSIS_VLLM_BASE_URL=http://<host>:<port>

**Important:**
- If you run `video-analysis` in Docker with `network_mode: host`,
the vLLM server must be reachable from the host network.
- vLLM may run:
- locally on the same machine, **or**
- on another machine, as long as the URL is reachable.

The service will **fail to start** if the vLLM endpoint is not reachable.

**Required:**
- All global prerequisites (see main [README.md](../../README.md))
- A running vLLM server with a vision-language model (e.g., BusterX)

**vLLM Server Requirements:**
- GPU required; VRAM depends on model (recommend ≥16GB, may require more).
- NVIDIA Driver 535+
- NVIDIA Container Toolkit

> **Note:** `video-analysis` itself runs on CPU. The GPU is only needed for the vLLM server.

## Features

- **Deepfake Detection**: Analyzes videos for signs of manipulation
- **Uniform Frame Sampling**: Extracts representative frames from videos
- **vLLM Integration**: Uses vision-language models for semantic analysis
- **Reproducibility Artifacts (optional)**: Saves request/response artifacts to a configurable runs directory for debugging and reproducibility  
  (this directory is created/used at runtime; it is not meant to be committed to the repo)

## Installation

```bash
cd modules/video-analysis

# Install dependencies
uv sync

# Install with dev dependencies
uv sync --extra dev
```

## Quick Start

### As an API Server

```bash
cd deploy/

# Create deployment env file
# NOTE: deploy.sh loads env ONLY from deploy/.env
cp ../.env.example .env
# Edit deploy/.env with your vLLM server URL and model

# Start the server
./deploy.sh --profile api --detach

# Or with nginx reverse proxy
./deploy.sh --profile api-nginx --detach
```

### API Endpoints

| Endpoint | Method | Description |
|----------|--------|-------------|
| `/health` | GET | Health check |
| `/analyze/video` | POST | Deepfake detection (fast, 16 frames) |
| `/analyze/video/semantic` | POST | Semantic analysis (detailed, 144+ frames) |

### Example API Request

```bash
# Health check
curl http://localhost:54600/health

# Analyze video
VIDEO="/path/to/video.mp4"
curl -sS -X POST "http://localhost:54600/analyze/video" -F "file=@${VIDEO}"
```

### Example Response

```json
{
  "request_id": "550e8400-e29b-41d4-a716-446655440000",
  "run_dir": "runs/550e8400-e29b-41d4-a716-446655440000",
  "verdict": "FAKE",
  "explanation": "The video shows clear signs of manipulation...",
  "usage": {
    "prompt_tokens": 1250,
    "completion_tokens": 150,
    "total_tokens": 1400
  },
  "latency_s": {
    "sampling_time_s": 0.234,
    "encode_time_s": 0.567,
    "model_inference_time_s": 12.345
  },
  "meta": {
    "fps": 30.0,
    "total_frames": 450,
    "duration_s": 15.0,
    "sampled": 16
  }
}
```

## Configuration

### Required Environment Variables

Configured via environment variables (prefix: `VIDEO_ANALYSIS_`). These are typically set in `deploy/.env`:

| Variable | Description |
|----------|-------------|
| `VIDEO_ANALYSIS_VLLM_BASE_URL` | vLLM server URL (e.g., `http://didiAI-video-vllm-buster:8000`) |
| `VIDEO_ANALYSIS_VLLM_MODEL` | Model name (e.g., `l8cv/BusterX_plusplus`) |
| `VIDEO_ANALYSIS_RUNS_DIR` | Directory for storing analysis artifacts (created/used at runtime) |

| `VIDEO_ANALYSIS_EXTERNAL_URL` | External URL for OpenAPI spec (e.g., `http://localhost:54600`) |


### Optional Nginx Environment Variables (api-nginx profile)

If you use the `api-nginx` profile, the nginx container can read these optional variables from `deploy/.env`:

| Variable | Example | Description |
|----------|---------|-------------|
| `NGINX_CONNECT_TIMEOUT` | `60s` | Upstream connect timeout |
| `NGINX_SEND_TIMEOUT` | `120s` | Upstream send timeout |
| `NGINX_READ_TIMEOUT` | `600s` | Upstream read timeout |

### Optional Tuning Parameters

Configured via `deploy/config.yaml` (env vars override YAML):

| Parameter | Default | Description |
|-----------|---------|-------------|
| `frames` | `16` | Number of frames to sample |
| `max_side` | `960` | Max image dimension (pixels) |
| `jpeg_quality` | `85` | JPEG encoding quality (1-100) |
| `max_tokens` | `750` | Max tokens for model response |
| `temperature` | `0.000001` | Sampling temperature |
| `repetition_penalty` | `1.05` | Repetition penalty |

## ⚠️ Testing Recommendations

### Model Selection for Semantic Analysis

**Current Setup:**
- Both deepfake and semantic analysis use **BusterX** (7B parameters)
- BusterX is optimized for deepfake detection

**TODO: Test Semantic Analysis with Qwen3-VL-30B**

For better semantic understanding, consider testing with the larger **Qwen3-VL-30B** model (already running @ port 8102):

```bash
# Current (BusterX 7B)
VIDEO_ANALYSIS_VLLM_BASE_URL=http://didiAI-video-vllm-buster:54500
VIDEO_ANALYSIS_VLLM_MODEL=busterx

# Alternative (Qwen3-VL 30B) - Better for semantic analysis
VIDEO_ANALYSIS_VLLM_BASE_URL=http://didiAI-llm-vllm-vision:14002
VIDEO_ANALYSIS_VLLM_MODEL=qwen3-vl
```

**Expected Improvements:**
- ✅ More detailed scene descriptions (30B vs 7B)
- ✅ Better understanding of complex actions
- ✅ More coherent narrative synthesis
- ✅ Higher quality semantic annotations

**Trade-offs:**
- ⏱️ Slightly higher latency (~15-20s per chunk vs ~12s)
- 📊 Better for semantic analysis, but keep BusterX for deepfake detection

**Recommendation:**
- **Deepfake endpoint:** Keep using BusterX (specialized for forgery detection)
- **Semantic endpoint:** Test with Qwen3-VL-30B for better results

## Deployment

```bash
cd deploy/

# Create deployment env file (REQUIRED)
cp ../.env.example .env
# Edit deploy/.env with your settings

# API only
./deploy.sh --profile api --detach

# API with nginx reverse proxy
./deploy.sh --profile api-nginx --detach

# View logs
./deploy.sh --profile api --logs

# Stop services
./deploy.sh --profile api --down
```

### Common Docker Commands

```bash
cd deploy/

# Restart the API service (compose service name)
docker compose restart video-analysis-api

# (Optional) restart by container name
docker restart video_analysis
```

### Port Allocation

| Port | Service | Environment |
|------|---------|-------------|
| `54600` | Video Analysis API | Development |
| `54500` | BusterX vLLM Server | Development |

## Development

```bash
# Install dev dependencies
uv sync --extra dev

# Run tests
uv run pytest

# Run tests with coverage
uv run pytest --cov=src/video_analysis --cov-report=term-missing

# Lint and format
uv run ruff check .
uv run ruff format .
```

## Architecture

```
modules/video-analysis/
├── deploy/
│   ├── deploy.sh             # Deployment script
│   ├── docker-compose.yml    # Docker services
│   ├── Dockerfile            # Container image
│   ├── nginx.conf            # Nginx reverse proxy config (optional)
│   ├── nginx.conf.template   # Template-based nginx config (optional)
│   └── config.yaml           # Tuning parameters
├── src/video_analysis/
│   ├── __init__.py
│   ├── app.py                # FastAPI application
│   ├── buster_client.py      # vLLM client
│   ├── schemas.py            # Response schemas
│   ├── settings.py           # Configuration (env + yaml)
│   └── video_sampling.py     # Frame extraction
├── tests/
├── .env.example              # Environment template
├── API.md                    # API documentation
├── pyproject.toml            # Dependencies
└── README.md                 # This file
```