LOT 1 - Optimizare script build -Instalare mono comanda
This commit is contained in:
parent
5380c3fc63
commit
42ff22bf85
127 changed files with 16163 additions and 532 deletions
|
|
@ -1,7 +1,7 @@
|
|||
# video-analysis configuration
|
||||
# Copy to deploy/.env and fill ALL required values.
|
||||
# Service must FAIL to start if required vars are missing.
|
||||
# Tuning parameters are in deploy/config.yaml (not here)
|
||||
# Tuning parameters default in settings.py; override via the VIDEO_ANALYSIS_* vars below
|
||||
# =============================================================================
|
||||
# REQUIRED (no defaults)
|
||||
# =============================================================================
|
||||
|
|
@ -18,7 +18,7 @@ VIDEO_ANALYSIS_RUNS_DIR=/app/runs
|
|||
# vLLM Connection (auto-configured for docker-compose profiles)
|
||||
# =============================================================================
|
||||
# For profile 'api-vllm' or 'full': uses internal vllm-buster container
|
||||
VIDEO_ANALYSIS_VLLM_BASE_URL=http://vllm-buster:8000
|
||||
VIDEO_ANALYSIS_VLLM_BASE_URL=http://didiAI-video-vllm-buster:54500
|
||||
VIDEO_ANALYSIS_VLLM_MODEL=busterx
|
||||
|
||||
# For profile 'api' with external vLLM: point to your external vLLM server
|
||||
|
|
@ -26,7 +26,7 @@ VIDEO_ANALYSIS_VLLM_MODEL=busterx
|
|||
# VIDEO_ANALYSIS_VLLM_MODEL=l8cv/BusterX_plusplus
|
||||
|
||||
# =============================================================================
|
||||
# Optional Tuning (override config.yaml defaults)
|
||||
# Optional Tuning (override settings.py defaults)
|
||||
# =============================================================================
|
||||
# VIDEO_ANALYSIS_FRAMES=16
|
||||
# VIDEO_ANALYSIS_MAX_SIDE=960
|
||||
|
|
|
|||
|
|
@ -22,49 +22,17 @@ No authentication required.
|
|||
|
||||
---
|
||||
|
||||
## ⚠️ TESTING REMINDER: Alternative Vision Models
|
||||
## Vision & Aggregation Models
|
||||
|
||||
**Current Configuration:**
|
||||
- **Deepfake Detection:** Uses BusterX (Qwen2.5-VL-7B fine-tuned) @ port 54500
|
||||
- **Semantic Analysis:** Uses BusterX (7B parameters)
|
||||
Both endpoints are backed by a single vision model — **BusterX** (`l8cv/BusterX_plusplus`, served as `busterx`) @ port `54500`:
|
||||
|
||||
**TODO - Test with Qwen3-VL-30B for Better Semantic Analysis:**
|
||||
| Stage | Model | Endpoint | Role |
|
||||
|-------|-------|----------|------|
|
||||
| Deepfake verdict | BusterX (Qwen2.5-VL-7B fine-tune) | `http://didiAI-video-vllm-buster:54500` | `REAL` / `FAKE` / `UNCERTAIN` + explanation |
|
||||
| Semantic chunk descriptions | BusterX (same endpoint) | `http://didiAI-video-vllm-buster:54500` | per-chunk `description` |
|
||||
| Semantic aggregation | DIDI text LLM (Qwen3.5) | `http://didiAI-llm-api:14011` | merges chunk descriptions into `final_summary` |
|
||||
|
||||
The semantic analysis endpoint can be configured to use **Qwen3-VL-30B** (already running @ port 14002) instead of BusterX for potentially better results:
|
||||
|
||||
| Model | Size | Port | Best For |
|
||||
|-------|------|------|----------|
|
||||
| **BusterX** | 7B | 54500 | Deepfake detection (specialized) |
|
||||
| **Qwen3-VL-30B** | 30B | 14002 | General semantic understanding |
|
||||
|
||||
**To test with Qwen3-VL-30B:**
|
||||
|
||||
1. Update `.env`:
|
||||
```bash
|
||||
VIDEO_ANALYSIS_VLLM_BASE_URL=http://didiAI-llm-vllm-vision:14002 # Use Qwen3-VL instead of BusterX
|
||||
VIDEO_ANALYSIS_VLLM_MODEL=qwen3-vl # Change from busterx
|
||||
```
|
||||
|
||||
2. Rebuild container:
|
||||
```bash
|
||||
cd deploy/
|
||||
docker compose build video-analysis-api
|
||||
docker compose up -d video-analysis-api
|
||||
```
|
||||
|
||||
3. Test semantic analysis:
|
||||
```bash
|
||||
curl -X POST http://localhost:54600/analyze/video/semantic \
|
||||
-F "file=@test_video.mp4"
|
||||
```
|
||||
|
||||
**Expected Benefits:**
|
||||
- More detailed scene descriptions (30B vs 7B parameters)
|
||||
- Better context understanding
|
||||
- More coherent narrative flow
|
||||
- Higher accuracy for complex scenes
|
||||
|
||||
**Note:** Deepfake detection should continue using BusterX (specialized model).
|
||||
BusterX is self-contained — Qwen2.5-VL is bundled inside the fine-tune, so no separate vision base model is loaded. There is no separate Qwen3-VL vision backend in this deployment.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -359,6 +327,20 @@ curl -X POST http://localhost:54600/analyze/video/semantic \
|
|||
|
||||
---
|
||||
|
||||
### Service Info
|
||||
|
||||
Return service catalog metadata (resources, models, functions). Consumed by the DIDI `catalog-api`.
|
||||
|
||||
**GET** `/v1/info`
|
||||
|
||||
**Example**
|
||||
|
||||
```bash
|
||||
curl http://localhost:54600/v1/info
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Error Responses
|
||||
|
||||
| Status | Description |
|
||||
|
|
|
|||
|
|
@ -4,8 +4,8 @@ Video analysis service for DIDI. Performs deepfake detection and semantic tempor
|
|||
|
||||
- **Stack:** Python 3.11+, FastAPI, uvicorn, OpenCV (headless), Pillow, NumPy, httpx/requests, ffmpeg toolchain (via OpenCV), pydantic-settings + YAML
|
||||
- **URL (Dev):** `http://10.11.10.12:54600`
|
||||
- **Container:** runs on GPU host (`network_mode: host` in `deploy/docker-compose.yml`); the service itself is CPU-only — GPU is consumed by the upstream vLLM server
|
||||
- **Vision backend:** external vLLM server (default: BusterX 7B @ port `54500`); same endpoint also drives semantic analysis. Optional alternative: Qwen3-VL-30B @ port `14002` for richer semantic narratives. The wider DIDI vision cascade (Qwen Vision local → Gemini Flash → GPT-4o) lives in `agent-v3`; this service only talks to one vLLM at a time.
|
||||
- **Container:** runs on GPU host, attached to the external bridge network `didi-network` (see `deploy/docker-compose.yml`); the service itself is CPU-only — GPU is consumed by the upstream vLLM server
|
||||
- **Vision backend:** external vLLM server (BusterX 7B @ port `54500`); the **same** BusterX endpoint drives both deepfake detection and semantic per-chunk analysis. Semantic narratives are produced by aggregating the per-chunk descriptions with the DIDI text LLM (Qwen3.5) via `http://didiAI-llm-api:14011`. The wider DIDI vision cascade (Qwen Vision local → Gemini Flash → GPT-4o) lives in `agent-v3`; this service talks only to the BusterX vLLM.
|
||||
|
||||
## Ce face
|
||||
|
||||
|
|
@ -50,8 +50,8 @@ Auth: none (called over private network / through Kong upstream by agent-v3).
|
|||
|
||||
This service does **not** implement a multi-provider cascade. It is a thin client over a single vLLM endpoint configured at startup:
|
||||
|
||||
- **Primary (deepfake):** BusterX (Qwen2.5-VL-7B fine-tune, `l8cv/BusterX_plusplus`) at `VIDEO_ANALYSIS_VLLM_BASE_URL` — typically `http://didiAI-video-vllm-buster:54500` on the GPU host
|
||||
- **Optional (semantic):** Qwen3-VL-30B at `http://didiAI-llm-vllm-vision:14002` — swap by editing `deploy/.env` and rebuilding
|
||||
- **Deepfake + semantic (vision):** BusterX (Qwen2.5-VL-7B fine-tune, `l8cv/BusterX_plusplus`, served as `busterx`) at `VIDEO_ANALYSIS_VLLM_BASE_URL` — typically `http://didiAI-video-vllm-buster:54500` on the GPU host. The same endpoint handles both the deepfake verdict and the per-chunk semantic descriptions. BusterX is self-contained (Qwen2.5-VL is bundled inside the fine-tune) — it does not load a separate Qwen base model.
|
||||
- **Semantic aggregation (text):** per-chunk descriptions are merged into a narrative `final_summary` by the DIDI text LLM (Qwen3.5) via `http://didiAI-llm-api:14011` — set through `VIDEO_ANALYSIS_SEMANTIC_LLM_BASE_URL`.
|
||||
- **DIDI-wide cascade** (Qwen Vision local → OpenRouter Gemini Flash → GPT-4o) is implemented in agent-v3, NOT here. This service is a leaf node in that chain — agent-v3 calls it as one of several vision options.
|
||||
- Service refuses to start if `VIDEO_ANALYSIS_VLLM_BASE_URL` is not set or the vLLM endpoint is unreachable (see `buster_client.py`, `settings.py`).
|
||||
|
||||
|
|
@ -62,7 +62,7 @@ Env vars (prefix `VIDEO_ANALYSIS_`), loaded from `deploy/.env`:
|
|||
| Variable | Required | Description |
|
||||
|---|---|---|
|
||||
| `VIDEO_ANALYSIS_VLLM_BASE_URL` | yes | Upstream vLLM server URL |
|
||||
| `VIDEO_ANALYSIS_VLLM_MODEL` | yes | Model name passed to vLLM (`busterx`, `qwen3-vl`, `l8cv/BusterX_plusplus`, …) |
|
||||
| `VIDEO_ANALYSIS_VLLM_MODEL` | yes | Served model name passed to vLLM (`busterx`; underlying weights `l8cv/BusterX_plusplus`) |
|
||||
| `VIDEO_ANALYSIS_RUNS_DIR` | yes | Where to drop per-request artifacts (default `/app/runs` in container) |
|
||||
| `VIDEO_ANALYSIS_EXTERNAL_URL` | yes | External URL embedded in the OpenAPI spec |
|
||||
| `HF_TOKEN`, `HF_CACHE_DIR` | yes (when running bundled vLLM) | HuggingFace creds + shared cache for the vLLM container |
|
||||
|
|
@ -74,7 +74,7 @@ Env vars (prefix `VIDEO_ANALYSIS_`), loaded from `deploy/.env`:
|
|||
| `VIDEO_ANALYSIS_REPETITION_PENALTY` | no (default 1.05) | Repetition penalty |
|
||||
| `NGINX_CONNECT_TIMEOUT` / `_SEND_TIMEOUT` / `_READ_TIMEOUT` | no | nginx upstream timeouts (only `api-nginx` profile) |
|
||||
|
||||
Tuning defaults live in `deploy/config.yaml`; env vars override YAML.
|
||||
Tuning defaults are baked into `settings.py` and overridden via the `VIDEO_ANALYSIS_*` env vars above (a `deploy/config.yaml` may optionally be supplied to override defaults, but none ships with the module).
|
||||
|
||||
## Deployment
|
||||
|
||||
|
|
@ -93,8 +93,8 @@ Tuning defaults live in `deploy/config.yaml`; env vars override YAML.
|
|||
## Related
|
||||
|
||||
- **agent-v3 video pipeline** — `/home/admin365/didi_mono/backend/services/orchestration-layer/agent-v3` is the consumer; orchestrates async video sessions and merges this service's verdict into `ai_tampered` + `techniques` results
|
||||
- **BusterX vLLM** (port `54500`) — sibling service in the AI platform; the actual GPU-backed model that this service queries (referenced in main `CLAUDE.md` ports section)
|
||||
- **Qwen3-VL vision vLLM** (port `14002`) — alternative semantic backend (`didiAI-llm-vllm-vision`)
|
||||
- **BusterX vLLM** (port `54500`) — sibling service in the AI platform; the actual GPU-backed model that this service queries for both deepfake and semantic chunks (referenced in main `CLAUDE.md` ports section)
|
||||
- **DIDI text LLM API** (`http://didiAI-llm-api:14011`) — Qwen3.5 endpoint used to aggregate semantic chunk descriptions into the final narrative
|
||||
- **AI platform shared assets** — `../../README.md`, `../../ruff.toml`
|
||||
- **Internal package layout:** `src/video_analysis/{app.py, buster_client.py, schemas.py, settings.py, video_sampling.py}`
|
||||
- **Sibling docs:** `README.md`, `API.md`, `TESTING.md` in this folder
|
||||
|
|
|
|||
|
|
@ -78,6 +78,7 @@ cp ../.env.example .env
|
|||
| `/health` | GET | Health check |
|
||||
| `/analyze/video` | POST | Deepfake detection (fast, 16 frames) |
|
||||
| `/analyze/video/semantic` | POST | Semantic analysis (detailed, 144+ frames) |
|
||||
| `/v1/info` | GET | Service catalog metadata (used by catalog-api) |
|
||||
|
||||
### Example API Request
|
||||
|
||||
|
|
@ -125,8 +126,8 @@ Configured via environment variables (prefix: `VIDEO_ANALYSIS_`). These are typi
|
|||
|
||||
| Variable | Description |
|
||||
|----------|-------------|
|
||||
| `VIDEO_ANALYSIS_VLLM_BASE_URL` | vLLM server URL (e.g., `http://didiAI-video-vllm-buster:8000`) |
|
||||
| `VIDEO_ANALYSIS_VLLM_MODEL` | Model name (e.g., `l8cv/BusterX_plusplus`) |
|
||||
| `VIDEO_ANALYSIS_VLLM_BASE_URL` | vLLM server URL (e.g., `http://didiAI-video-vllm-buster:54500`) |
|
||||
| `VIDEO_ANALYSIS_VLLM_MODEL` | Served model name (e.g., `busterx`) |
|
||||
| `VIDEO_ANALYSIS_RUNS_DIR` | Directory for storing analysis artifacts (created/used at runtime) |
|
||||
|
||||
| `VIDEO_ANALYSIS_EXTERNAL_URL` | External URL for OpenAPI spec (e.g., `http://localhost:54600`) |
|
||||
|
|
@ -144,7 +145,7 @@ If you use the `api-nginx` profile, the nginx container can read these optional
|
|||
|
||||
### Optional Tuning Parameters
|
||||
|
||||
Configured via `deploy/config.yaml` (env vars override YAML):
|
||||
These default to the values below in `settings.py` and are overridden via the matching `VIDEO_ANALYSIS_*` environment variables (a `deploy/config.yaml` may optionally be supplied to override defaults, but none ships with the module):
|
||||
|
||||
| Parameter | Default | Description |
|
||||
|-----------|---------|-------------|
|
||||
|
|
@ -155,42 +156,24 @@ Configured via `deploy/config.yaml` (env vars override YAML):
|
|||
| `temperature` | `0.000001` | Sampling temperature |
|
||||
| `repetition_penalty` | `1.05` | Repetition penalty |
|
||||
|
||||
## ⚠️ Testing Recommendations
|
||||
## Semantic Analysis Pipeline
|
||||
|
||||
### Model Selection for Semantic Analysis
|
||||
Both endpoints use the **same** BusterX vLLM (`busterx` @ port `54500`):
|
||||
|
||||
**Current Setup:**
|
||||
- Both deepfake and semantic analysis use **BusterX** (7B parameters)
|
||||
- BusterX is optimized for deepfake detection
|
||||
- **Deepfake endpoint** — BusterX returns the `REAL` / `FAKE` / `UNCERTAIN` verdict + explanation.
|
||||
- **Semantic endpoint** — BusterX produces a per-chunk `description` for each temporal chunk. When `enable_aggregation=true`, those chunk descriptions are merged into a single narrative `final_summary` by the DIDI text LLM (Qwen3.5) via `http://didiAI-llm-api:14011` (set through `VIDEO_ANALYSIS_SEMANTIC_LLM_BASE_URL`).
|
||||
|
||||
**TODO: Test Semantic Analysis with Qwen3-VL-30B**
|
||||
|
||||
For better semantic understanding, consider testing with the larger **Qwen3-VL-30B** model (already running @ port 8102):
|
||||
BusterX is self-contained — Qwen2.5-VL is bundled inside the `l8cv/BusterX_plusplus` fine-tune, so no separate vision base model is loaded. There is no separate Qwen3-VL vision backend in this deployment; the only vision model the service talks to is BusterX.
|
||||
|
||||
```bash
|
||||
# Current (BusterX 7B)
|
||||
# Vision backend (deepfake + semantic chunk descriptions)
|
||||
VIDEO_ANALYSIS_VLLM_BASE_URL=http://didiAI-video-vllm-buster:54500
|
||||
VIDEO_ANALYSIS_VLLM_MODEL=busterx
|
||||
|
||||
# Alternative (Qwen3-VL 30B) - Better for semantic analysis
|
||||
VIDEO_ANALYSIS_VLLM_BASE_URL=http://didiAI-llm-vllm-vision:14002
|
||||
VIDEO_ANALYSIS_VLLM_MODEL=qwen3-vl
|
||||
# Text LLM used only to aggregate semantic chunks into a narrative summary
|
||||
VIDEO_ANALYSIS_SEMANTIC_LLM_BASE_URL=http://didiAI-llm-api:14011
|
||||
```
|
||||
|
||||
**Expected Improvements:**
|
||||
- ✅ More detailed scene descriptions (30B vs 7B)
|
||||
- ✅ Better understanding of complex actions
|
||||
- ✅ More coherent narrative synthesis
|
||||
- ✅ Higher quality semantic annotations
|
||||
|
||||
**Trade-offs:**
|
||||
- ⏱️ Slightly higher latency (~15-20s per chunk vs ~12s)
|
||||
- 📊 Better for semantic analysis, but keep BusterX for deepfake detection
|
||||
|
||||
**Recommendation:**
|
||||
- **Deepfake endpoint:** Keep using BusterX (specialized for forgery detection)
|
||||
- **Semantic endpoint:** Test with Qwen3-VL-30B for better results
|
||||
|
||||
## Deployment
|
||||
|
||||
```bash
|
||||
|
|
@ -258,8 +241,7 @@ modules/video-analysis/
|
|||
│ ├── docker-compose.yml # Docker services
|
||||
│ ├── Dockerfile # Container image
|
||||
│ ├── nginx.conf # Nginx reverse proxy config (optional)
|
||||
│ ├── nginx.conf.template # Template-based nginx config (optional)
|
||||
│ └── config.yaml # Tuning parameters
|
||||
│ └── nginx.conf.template # Template-based nginx config (optional)
|
||||
├── src/video_analysis/
|
||||
│ ├── __init__.py
|
||||
│ ├── app.py # FastAPI application
|
||||
|
|
|
|||
|
|
@ -1,14 +1,23 @@
|
|||
# Video Analysis - Testing Checklist
|
||||
|
||||
## ⚠️ HIGH PRIORITY: Model Comparison for Semantic Analysis
|
||||
> ⚠️ **NOTĂ:** Acest fișier este un plan **EXPLORATORIU / R&D**, NU configurația livrată.
|
||||
> Modelul `Qwen3-VL-30B` (port 14002 / `gpt-oss-120b` / `deploy-llm-api-1`) **nu există** în
|
||||
> deployment-ul livrat. Pipeline-ul REAL: deepfake + semantic folosesc **BusterX** (`busterx`,
|
||||
> vLLM `didiAI-video-vllm-buster:54500`), iar agregarea semantică pe text folosește **Qwen3.5**
|
||||
> (`didiAI-llm-api:14011`). Pentru testarea sistemului livrat vezi `ai_platform/local_gpu_stack/TESTING.md`.
|
||||
|
||||
## R&D (opțional): Model Comparison for Semantic Analysis
|
||||
|
||||
### Background
|
||||
|
||||
Currently both deepfake detection and semantic analysis use **BusterX** (Qwen2.5-VL-7B, 7B parameters). However, we have access to a much larger model **Qwen3-VL-30B** (30B parameters) that could provide significantly better semantic understanding.
|
||||
Sistemul livrat folosește **BusterX** (Qwen2.5-VL-7B) atât pentru deepfake cât și pentru cadrele
|
||||
din analiza semantică, cu agregare text pe **Qwen3.5**. Ca direcție de cercetare, s-ar putea
|
||||
evalua un model vision mai mare pentru partea semantică (dacă va fi disponibil în viitor).
|
||||
|
||||
### Hypothesis
|
||||
|
||||
Semantic analysis (content understanding, scene description, narrative) would benefit from the larger Qwen3-VL-30B model, while deepfake detection should continue using the specialized BusterX model.
|
||||
Analiza semantică (descriere scenă, narativ) ar putea beneficia de un model vision mai mare,
|
||||
în timp ce detecția deepfake rămâne pe modelul specializat BusterX.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -41,7 +50,7 @@ VIDEO_ANALYSIS_VLLM_MODEL=qwen3-vl # 30B parameters
|
|||
cd /home/vasi/ml-projects/modules/video-analysis/deploy
|
||||
|
||||
# Test semantic analysis with BusterX
|
||||
curl -X POST http://localhost:8007/analyze/video/semantic \
|
||||
curl -X POST http://localhost:54600/analyze/video/semantic \
|
||||
-F "file=@test_video_60s.mp4" \
|
||||
-F "chunk_duration_s=10.0" \
|
||||
-F "frames_per_chunk=24" \
|
||||
|
|
@ -91,7 +100,7 @@ docker compose up -d video-analysis-api
|
|||
sleep 10
|
||||
|
||||
# Test with same video
|
||||
curl -X POST http://localhost:8007/analyze/video/semantic \
|
||||
curl -X POST http://localhost:54600/analyze/video/semantic \
|
||||
-F "file=@test_video_60s.mp4" \
|
||||
-F "chunk_duration_s=10.0" \
|
||||
-F "frames_per_chunk=24" \
|
||||
|
|
|
|||
|
|
@ -8,8 +8,8 @@ COPY --from=ghcr.io/astral-sh/uv:0.10 /uv /usr/local/bin/uv
|
|||
|
||||
WORKDIR /app
|
||||
|
||||
# Copy project files
|
||||
COPY pyproject.toml uv.lock README.md ./
|
||||
# Copy project files (uv.lock not committed; resolved at build time)
|
||||
COPY pyproject.toml README.md ./
|
||||
COPY src/ ./src/
|
||||
|
||||
# Install dependencies (allow resolving to pick up new deps)
|
||||
|
|
|
|||
|
|
@ -16,10 +16,10 @@
|
|||
# Naming Convention: didiAI-{module}-{service}
|
||||
#
|
||||
# Network:
|
||||
# Uses deploy_default network (shared with other modules)
|
||||
# Uses the external didi-network bridge (shared with other AI modules)
|
||||
|
||||
networks:
|
||||
deploy_default:
|
||||
didi-network:
|
||||
external: true
|
||||
|
||||
services:
|
||||
|
|
@ -31,11 +31,11 @@ services:
|
|||
# BusterX is based on Qwen2.5-VL-7B (~17GB VRAM with optimizations)
|
||||
vllm-buster:
|
||||
container_name: didiAI-video-vllm-buster
|
||||
image: vllm/vllm-openai:v0.8.5
|
||||
image: vllm/vllm-openai:qwen3_5
|
||||
ports:
|
||||
- "54500:54500"
|
||||
networks:
|
||||
- deploy_default
|
||||
- didi-network
|
||||
volumes:
|
||||
- ${HF_CACHE_DIR:-/cai2_ds_storage/hf_cache}:/root/.cache/huggingface
|
||||
environment:
|
||||
|
|
@ -83,7 +83,7 @@ services:
|
|||
ports:
|
||||
- "54600:54600"
|
||||
networks:
|
||||
- deploy_default
|
||||
- didi-network
|
||||
|
||||
environment:
|
||||
# Server settings
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue