clossers/didi-lot1-ai
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

LOT 1 - Optimizare script build -Instalare mono comanda

Browse source
This commit is contained in:
Dezvoltari Evotech 2026-06-27 06:42:02 -07:00
parent 5380c3fc63
commit 42ff22bf85
127 changed files with 16163 additions and 532 deletions
Download patch file Download diff file Expand all files Collapse all files

45
ai_platform/INDEX.md
View file

@ -25,7 +25,7 @@ Acest fisier descrie fiecare modul, fiecare fisier, fiecare clasa, fiecare funct
## 1. Arhitectura generala
Monorepo cu 8 module independente. Fiecare modul este un pachet Python instalabil cu FastAPI, containerizat in Docker, conectat pe reteaua comuna `didi-network`. Toate comunica prin HTTP intern. Singurul port expus extern este 11000 (gateway nginx).
Monorepo cu 13 module independente (audio, catalog-api, cloak, dashboard, didi_brain, embeddings, extractors, forensic_features, gateway, llm-inference, rerank, video-analysis, web). Fiecare modul este un pachet Python instalabil cu FastAPI, containerizat in Docker, conectat pe reteaua comuna `didi-network`. Toate comunica prin HTTP intern. Singurul port expus extern este 11000 (gateway nginx).
Flux tipic de request extern:
```
@ -39,12 +39,14 @@ Video API -> vLLM (BusterX)
Catalog API -> (interogheaza toate celelalte servicii pe /v1/info)
```
Modele ML servite:
- Qwen3.5-35B-A3B (text + vision, MoE) - vLLM pe GPU 0
- BAAI/bge-m3 (embeddings) - vLLM sau llama.cpp
- BAAI/bge-reranker-v2-m3 (reranking) - vLLM sau llama.cpp
Modele ML servite (7 modele, 13 module):
- Qwen3.5-35B-A3B (text + vision + OCR, MoE) - vLLM pe GPU 0, servit ca `qwen3.5`. Nu exista model vision separat: OCR/vision se face prin acest gateway LLM.
- BAAI/bge-m3 (embeddings) - vLLM sau llama.cpp, GPU 1
- BAAI/bge-reranker-v2-m3 (reranking) - vLLM sau llama.cpp, GPU 1
- Whisper large-v3-turbo (speech-to-text) - faster-whisper pe GPU 0
- BusterX / Qwen2.5-VL-7B (deepfake detection) - vLLM pe GPU 1
- BusterX (Qwen2.5-VL-7B fine-tuned, deepfake detection) - vLLM pe GPU 1
- GLiNER (backbone mdeberta, NER) + YOLOv8n (object detection) - extractors
- MediaPipe (rPPG / lip-sync / landmarks) - forensic_features
---
@ -63,7 +65,7 @@ Gateway-ul este un reverse proxy nginx care ruteaza toate request-urile catre se
Defineste 4 upstream-uri:
- `llm` -> `didiAI-llm-api:14011`
- `audio` -> `didiAI-audio-api:54300`
- `audio` -> `didiAI-audio:54300`
- `web` -> `didiAI-web-api:51100`
- `catalog` -> `didiAI-catalog-api:11000`
@ -547,7 +549,7 @@ Structura identica cu embeddings. Diferente specifice:
## 7. Audio - transcriere audio
**Locatie:** `modules/audio/`
**Container:** `didiAI-audio-api` (port 54300)
**Container:** `didiAI-audio` (port 54300)
### Ce face
@ -599,7 +601,7 @@ Rute:
**deploy/Dockerfile** - Bazat pe `nvidia/cuda:12.1.0-runtime-ubuntu22.04`. Instaleaza Python 3.10, ffmpeg. Nu foloseste uv, ci pip direct. Port 54300.
**deploy/docker-compose.yml** - Serviciu `audio-api`, container `didiAI-audio-api`. GPU 0 (CUDA_VISIBLE_DEVICES=0). Volum pentru cache modele. Profile: api. Start period 60s.
**deploy/docker-compose.yml** - Serviciu `audio-api`, container `didiAI-audio`. GPU 0 (CUDA_VISIBLE_DEVICES=0). Volum pentru cache modele. Profile: api. Start period 60s.
**deploy/deploy.sh** - Valideaza AUDIO_MODEL, AUDIO_DEVICE, AUDIO_CACHE_DIR. Suporta profile `api` si `api-nginx`.
@ -613,7 +615,7 @@ Rute:
### Ce face
Doua functionalitati:
1. Detectie deepfake - extrage 16 frame-uri uniforme, le trimite la BusterX (model fine-tuned pe Qwen2.5-VL-7B), obtine verdict REAL/FAKE/INCONCLUSIVE
1. Detectie deepfake - extrage 16 frame-uri uniforme, le trimite la BusterX (model fine-tuned pe Qwen2.5-VL-7B), obtine verdict REAL/FAKE/UNCERTAIN
2. Analiza semantica - divide video-ul in chunk-uri temporale (default 10s), extrage 24 frame-uri/chunk, descrie fiecare chunk cu LLM vision, optional agrega intr-un summary final
### Fisiere sursa
@ -653,10 +655,10 @@ Suporta configurare din `deploy/config.yaml` (YAML), cu override din variabile d
- `frame_to_data_url_b64jpeg(frame_bgr, max_side, jpeg_quality)` - converteste frame BGR la RGB PIL Image, scaleaza la max_side, encodeaza JPEG, returneaza data URI base64
- `call_vllm_chat(base_url, model, data_urls, prompt, max_tokens, temperature, repetition_penalty, timeout_s=180)` - construieste payload cu imagini + text, POST la `/v1/chat/completions`, masoara timpul. Returneaza (response JSON, elapsed_seconds)
- `parse_verdict_and_explanation(model_text)` - verifica primele 20 caractere (uppercase) pt prefix verdict. REAL/FAKE/altceva=INCONCLUSIVE.
- `parse_verdict_and_explanation(model_text)` - verifica primele 20 caractere (uppercase) pt prefix verdict. REAL/FAKE/altceva=UNCERTAIN.
**src/video_analysis/schemas.py**
- `Verdict = Literal["REAL", "FAKE", "INCONCLUSIVE"]`
- `Verdict = Literal["REAL", "FAKE", "UNCERTAIN"]`
- `Usage` - prompt_tokens, completion_tokens, total_tokens
- `LatencyS` - sampling_time_s, encode_time_s, model_inference_time_s
- `Meta` - fps, total_frames, duration_s, sampled, indices, timestamps_s
@ -696,7 +698,7 @@ Functii helper: `safe_mkdir()`, `write_json()`, `sha256_file()`.
**deploy/docker-compose.yml** - 2 servicii:
1. `vllm-buster` (didiAI-video-vllm-buster, port 54500) - Image: `vllm/vllm-openai:latest`. Model: `l8cv/BusterX_plusplus` (served as "busterx"). GPU 1. max-model-len 32768, gpu-memory-utilization 0.25, prefix caching activat. Profile: api-vllm. 600s start_period.
1. `vllm-buster` (didiAI-video-vllm-buster, port 54500) - Image: `vllm/vllm-openai:qwen3_5`. Model: `l8cv/BusterX_plusplus` (served as "busterx"). GPU 1. max-model-len 32768, gpu-memory-utilization 0.25, prefix caching activat. Profile: api-vllm. 600s start_period.
2. `video-analysis-api` (didiAI-video-api, port 54600) - FastAPI. Volum `../runs` montat la `/app/runs`. Profile: api, api-vllm.
**deploy/Dockerfile** - Multi-stage python:3.11-slim cu uv. Port 54600.
@ -717,7 +719,7 @@ Modul complex de fact-checking cu pipeline complet: detectie context -> cautare
**src/web/config.py** - `WebSettings(BaseSettings)` cu prefix `WEB_`:
Required:
- `searxng_base_url` - URL SearXNG (ex: http://localhost:55100)
- `searxng_base_url` - URL SearXNG (ex: http://didiAI-web-searxng:8080)
- `llm_base_url` - URL LLM inference server
- `external_url` - URL extern OpenAPI
@ -911,6 +913,7 @@ PRODUCTION (1xxxx):
DEVELOPMENT (5xxxx):
51100 Web API - fact-checking + cautare web
51300 Dashboard - admin UI + API (monitorizare AI)
54100 Embeddings API Dev
54101 vLLM Embed Server Dev
54110 llama.cpp Embed Server Dev
@ -918,8 +921,14 @@ DEVELOPMENT (5xxxx):
54201 vLLM Rerank Server Dev
54210 llama.cpp Rerank Server Dev
54300 Audio API - transcriere Whisper
54400 Extractors API - metadata/sentiment/OCR/NER/detect
54500 BusterX vLLM - server vision deepfake
54600 Video Analysis API - analiza video
8080 SearXNG (didiAI-web-searxng, intern; nepublicat pe host) - metasearch free tier web
ALTE PORTURI (in afara schemei 5-cifre):
8085 Forensic Features API - detectoare forensice (rPPG/lip-sync/forgery)
8090 didi_brain API - cache rezultate + RAG + fact-checking
```
Schema porturi: 5 cifre. Prima cifra: 1=prod, 5=dev. A doua cifra: 1=API/Gateway, 4=LLM/AI.
@ -931,7 +940,7 @@ Schema porturi: 5 cifre. Prima cifra: 1=prod, 5=dev. A doua cifra: 1=API/Gateway
| GPU | Ce ruleaza | VRAM folosit | VRAM total |
|-----|-----------|-------------|------------|
| GPU 0 | Qwen3.5-35B-A3B (~57GB) + Whisper large-v3-turbo (~2GB) | ~59GB | 143GB |
| GPU 1 | BusterX / Qwen2.5-VL-7B (~22GB) | ~22GB | 143GB |
| GPU 1 | BusterX / Qwen2.5-VL-7B (~22GB) + BAAI/bge-m3 + BAAI/bge-reranker-v2-m3 | ~30GB | 143GB |
---
@ -940,9 +949,9 @@ Schema porturi: 5 cifre. Prima cifra: 1=prod, 5=dev. A doua cifra: 1=API/Gateway
Toate containerele sunt pe reteaua externa `didi-network`. Comunicarea interna se face prin DNS Docker (nume containere):
```
didiAI-gateway -> didiAI-llm-api, didiAI-audio-api, didiAI-web-api, didiAI-catalog-api
didiAI-gateway -> didiAI-llm-api, didiAI-audio, didiAI-web-api, didiAI-catalog-api
didiAI-llm-api -> didiAI-vllm-qwen3.5
didiAI-catalog-api -> didiAI-llm-api, didiAI-audio-api, didiAI-video-api, didiAI-web-api
didiAI-catalog-api -> didiAI-llm-api, didiAI-audio, didiAI-video-api, didiAI-web-api
didiAI-web-api -> SearXNG, didiAI-llm-api
didiAI-video-api -> didiAI-video-vllm-buster, didiAI-llm-api (pt agregare semantica)
didiAI-embeddings-api -> didiAI-embeddings-vllm, didiAI-embeddings-llamacpp
@ -953,7 +962,7 @@ Naming convention containere: `didiAI-{modul}-{serviciu}`.
## Recent Changes (2026-05-05)
- **Login Keycloak SSO functional la `/admin-ai/`**: realm `didi-admins` (mutat din `didi-clients`), client `ai-platform-dashboard` (creat in didi-admins ca clona), required role `admin`. SSO comun cu admin-dashboard backend (1 login = ambele dashboard-uri).
- **Dashboard ruleaza in STAGING MODE (`DASHBOARD_STAGING_MODE=true`) — Keycloak este bypassed**: autentificarea SSO este configurata dar dezactivata in starea livrata. Config-ul pregatit: realm `didi-admins` (mutat din `didi-clients`), client `ai-platform-dashboard` (creat in didi-admins ca clona), required role `admin`, SSO comun cu admin-dashboard backend. **Inainte de productie: scoate `DASHBOARD_STAGING_MODE` pentru a activa auth-ul.**
- **AI dashboard env**: `VITE_KEYCLOAK_URL=https://sso.clossers.com`, `VITE_KEYCLOAK_REALM=didi-admins`, `VITE_KEYCLOAK_CLIENT_ID=ai-platform-dashboard`, `VITE_KEYCLOAK_REQUIRED_ROLE=admin`. Dual var pentru build (VITE_*) + runtime (DASHBOARD_*).
- **Schema config DB-overridable**: tabel nou `config_schema_override` (auto-creat la startup), helper `_merged_schema(session)` in `routes/config.py`, endpoint-uri admin `GET /api/config/schema/_overrides`, `PUT /api/config/schema/{key}`, `DELETE /api/config/schema/{key}`. Audit trail (action `config.schema.upsert/delete/seed`).
- **Migrare automata 98 chei -> DB**: la primul startup, `seed_schema_if_empty()` populeaza tabelul din `KNOWN_KEYS` (idempotent). Codul KNOWN_KEYS ramane fallback daca DB e sters. DB = single source of truth pentru schema acum.