LOT 1 - Optimizare script build -Instalare mono comanda

This commit is contained in:
Dezvoltari Evotech 2026-06-27 06:42:02 -07:00
parent 5380c3fc63
commit 42ff22bf85
127 changed files with 16163 additions and 532 deletions

View file

@ -0,0 +1,43 @@
# =============================================================================
# DiDi LOT 1 — Local GPU stack configuration
# Copy to `.env` (deploy.sh face asta automat daca lipseste) si ajusteaza.
# =============================================================================
# Host IP folosit in EXTERNAL_URL-uri si in comenzile de test.
# Lasa GOL -> deploy.sh detecteaza automat IP-ul principal si il scrie aici.
HOST_IP=
# --- Brain / Atomic / Postgres -----------------------------------------------
# Parola e folosita de postgres, atomic-server SI brain-api (trebuie sa fie una singura).
POSTGRES_PASSWORD=brain_local_dev
# Token API atomic-server. Lasa GOL -> deploy.sh il genereaza automat
# (atomic-server token create) si il scrie inapoi aici.
ATOMIC_TOKEN=
# --- Gateway -----------------------------------------------------------------
GATEWAY_API_TOKEN=didi-local-dev-token-123
# --- Dashboard ---------------------------------------------------------------
DASHBOARD_DB_USER=dashboard
DASHBOARD_DB_PASSWORD=devpass123
DASHBOARD_DB_NAME=dashboard
# staging=true -> bypass Keycloak (intri direct in UI). Scoate in productie.
DASHBOARD_STAGING_MODE=true
# --- Modele (nume servite; nu schimba fara motiv) ----------------------------
LLM_MODEL=Qwen/Qwen3.5-35B-A3B
LLM_SERVED_NAME=qwen3.5
EMB_MODEL=BAAI/bge-m3
RERANK_MODEL=BAAI/bge-reranker-v2-m3
WHISPER_MODEL=large-v3-turbo
WHISPER_REPO=mobiuslabsgmbh/faster-whisper-large-v3-turbo
BUSTER_MODEL=l8cv/BusterX_plusplus
# Extractoare (NER) — GLiNER + backbone-ul lui
NER_MODEL=urchade/gliner_multi-v2.1
NER_BACKBONE=microsoft/mdeberta-v3-base
# Zona LOCALA unica pentru TOATE modelele (un folder per model, format HF cache:
# models--<org>--<name>/). Toate containerele (Qwen/BusterX/bge/whisper/GLiNER)
# sunt servite OFFLINE de aici. Lasa GOL -> deploy.sh foloseste ai_platform/models.
# Ce lipseste se descarca automat din HF in acest director (offline-first).
MODELS_DIR=

View file

@ -0,0 +1,74 @@
# DiDi LOT 1 — Local GPU stack (deploy progresiv)
Deploy „cărămidă cu cărămidă" al întregii platforme pe un host cu GPU.
Pentru fiecare componentă: **build/pull + download → seed (dacă e cazul) →
TEST funcțional → doar dacă testul trece corect → următoarea**. Idempotent,
se oprește la primul test picat.
## Cerințe (verificate de preflight, nu se instalează automat)
- Docker Engine 24+ cu plugin `docker compose` v2
- NVIDIA driver ≥535 + `nvidia-container-toolkit` (runtime configurat în Docker)
- ≥1 GPU (proiectat pe 2× H200; GPU 0 = LLM+Whisper, GPU 1 = BusterX+bge×2)
- ~100 GB liberi pe `/` (toate modelele ≈ 94 GB)
## Modele — o singură zonă locală
Toate modelele stau în **`ai_platform/models/`** (format HF cache, un folder per model),
servite **offline** de acolo (fără rețea la runtime). `MODELS_DIR` în `.env` (auto = acest folder).
Ce lipsește se descarcă o singură dată în el (`deploy.sh` pasul DOWNLOAD). NU se comite în git.
| Folder | Model | Folosit de |
|---|---|---|
| `models--Qwen--Qwen3.5-35B-A3B` | LLM (67G) | llm (vLLM) |
| `models--l8cv--BusterX_plusplus` | deepfake (16G, Qwen2.5-VL intern) | video (vLLM) |
| `models--BAAI--bge-m3` | embeddings | embeddings (vLLM) |
| `models--BAAI--bge-reranker-v2-m3` | reranker | rerank (vLLM) |
| `models--mobiuslabsgmbh--faster-whisper-large-v3-turbo` | STT | audio |
| `models--urchade--gliner_multi-v2.1` + `models--microsoft--mdeberta-v3-base` | NER + backbone | extractors |
Copiezi `ai_platform/models/` pe altă mașină → zero download la deploy.
## Utilizare
```bash
cd ai_platform/local_gpu_stack
./deploy.sh # tot, de la preflight la dashboard
./deploy.sh --from web # reia de la cărămida 'web' (sare peste cele dinainte)
./deploy.sh --only llm # rulează o singură cărămidă (presupune deps deja up)
./deploy.sh --install-deps # instalează Docker + nvidia-container-toolkit (sudo), apoi continuă
```
`.env` se creează automat din `.env.example` la prima rulare.
### Mașină nouă (VM „chioară" cu GPU)
1. (o singură dată) driver NVIDIA ≥535 + reboot — `sudo ubuntu-drivers install` (nu se automatizează, cere reboot).
2. `git clone <repo> && cd .../ai_platform/local_gpu_stack`
3. `./deploy.sh --install-deps` → instalează Docker+toolkit, apoi:
- **HOST_IP** se auto-detectează și se scrie în `.env`
- **BusterX** se descarcă automat din HF dacă nu e pe disc (`BUSTER_HF_CACHE`)
- Qwen/bge se descarcă (cu progres), totul se ridică cărămidă cu cărămidă cu test la fiecare.
> Singurul pas care NU e o comandă: driverul NVIDIA (reboot). Restul e `./deploy.sh`.
Cărămizi, în ordine: `embeddings · rerank · llm · extractors · audio · video · forensic · web · catalog · gateway · brain · dashboard`.
La final: dashboard pe `http://<HOST_IP>:51300/admin-ai/` (staging mode, fără login).
## Ce automatizează (fixurile care înainte erau manuale)
- rețea unică `didi-network`; plasare GPU 0/1; nume containere `didiAI-*`
- base-URL vLLM **fără** `/v1` (codul îl adaugă) pentru llm/embeddings/video
- YOLO scrie modelul în `/tmp` (cwd e read-only la non-root)
- gateway LLM injectează `enable_thinking:false` → JSON curat din Qwen3.5 (sentiment/OCR/semantic/brain)
- SearXNG fără proxy outbound, ca serviciu compose
- brain: aliniere parolă postgres, creare **token atomic** (auto, salvat în `.env`), **seed taxonomie** (79 tag-uri)
- dashboard: `.env` cu staging + override-uri health; **seed catalog** (9 intrări)
## Fișiere
| Fișier | Rol |
|---|---|
| `deploy.sh` | orchestratorul progresiv gated |
| `lib.sh` | logging, `wait_health`, `gate`, shim docker (merge și prin `sg docker`) |
| `docker-compose.yml` | stack-ul GPU + non-GPU + searxng |
| `.env.example` | configurația (host, porturi, parole, token) |
| `seed/seed_catalog.sh` | populează catalogul DB al dashboard-ului (idempotent) |
| `seed/seed_brain.sh` | token atomic + taxonomie + env brain |
| `TESTING.md` | comenzi de testare per componentă + smoke-test |
> Brain rulează din `modules/didi_brain/infra/docker-compose.yml`; dashboard și gateway
> din `modules/*/deploy/``deploy.sh` le orchestrează pe toate sub proiectul `local_gpu_stack`.

View file

@ -0,0 +1,228 @@
# DiDi LOT 1 — Comenzi de testare per componentă
Host: **10.11.10.18**. Toate comenzile sunt `curl` copy-paste. Statusul din dreptul
fiecăreia e rezultatul verificat la ultima rulare.
> Convenție: `H=10.11.10.18`. Rulează întâi `export H=10.11.10.18`.
---
## 0. Pregătire fișiere de test (o singură dată)
```bash
export H=10.11.10.18
cd /home/topclossers/didi-lot1-ai/ai_platform/local_gpu_stack
mkdir -p test_assets
# video mic din artefacte
cp "$(find ../../artefacte_lot1/runs_deepfake -iname '*.mp4' -printf '%s\t%p\n' | sort -n | head -1 | cut -f2)" test_assets/sample.mp4
# extrage un cadru (jpg) + audio (wav) cu ffmpeg din containerul extractors
docker cp test_assets/sample.mp4 didiAI-extractors:/tmp/s.mp4
docker exec didiAI-extractors sh -c 'ffmpeg -y -i /tmp/s.mp4 -frames:v 1 /tmp/frame.jpg -vn -ar 16000 -ac 1 -t 8 /tmp/audio.wav'
docker cp didiAI-extractors:/tmp/frame.jpg test_assets/frame.jpg
docker cp didiAI-extractors:/tmp/audio.wav test_assets/audio.wav
cd test_assets # restul comenzilor presupun că ești aici pentru fișiere
```
---
## 1. LLM — Qwen3.5-35B-A3B (port 14011) ✅
```bash
# modele servite
curl -s http://$H:14011/v1/models | python3 -m json.tool
# chat completion
curl -s http://$H:14011/v1/chat/completions -H 'Content-Type: application/json' -d '{
"model":"qwen3.5",
"messages":[{"role":"user","content":"Ce este un deepfake? Raspunde scurt."}],
"max_tokens":200
}' | python3 -m json.tool
# health
curl -s http://$H:14011/health
```
> Notă: Qwen3.5 e model de **reasoning** — răspunsul conține un bloc „Thinking" înainte de
> răspunsul final. Normal.
## 2. Embeddings — bge-m3 (port 14100) ✅
```bash
curl -s http://$H:14100/v1/embeddings -H 'Content-Type: application/json' -d '{
"model":"bge-m3","input":"text de analizat"
}' | python3 -c "import sys,json;print('dim:',len(json.load(sys.stdin)['data'][0]['embedding']))"
```
> Așteptat: `dim: 1024`.
## 3. Rerank — bge-reranker-v2-m3 (port 14200) ✅
```bash
curl -s http://$H:14200/v1/rerank -H 'Content-Type: application/json' -d '{
"model":"bge-reranker-v2-m3",
"query":"detectie deepfake",
"documents":["o pisica neagra","sistem de detectie deepfake pentru video"]
}' | python3 -m json.tool
```
> Așteptat: documentul relevant primește scor mult mai mare (index 1).
## 4. Audio / Whisper — large-v3-turbo (port 54300) ✅
```bash
curl -s http://$H:54300/v1/audio/transcriptions \
-F "file=@audio.wav" -F "model=large-v3-turbo" | python3 -m json.tool
# modele / info
curl -s http://$H:54300/v1/models | python3 -m json.tool
```
> Acceptă și `.mp4` direct (ffmpeg în container). Returnează `text` + `language`.
## 5. Video / BusterX++ (deepfake) (port 54600) ✅
```bash
# verdict deepfake REAL/FAKE/UNCERTAIN
curl -s http://$H:54600/analyze/video -F "file=@sample.mp4" | python3 -m json.tool
# analiza semantica (descriere continut, foloseste si LLM-ul)
curl -s http://$H:54600/analyze/video/semantic -F "file=@sample.mp4" | python3 -m json.tool
curl -s http://$H:54600/v1/info | python3 -m json.tool
```
> Așteptat: `{"verdict":"REAL"|"FAKE"|"UNCERTAIN","frames_analyzed":16,"evidence":[...]}`.
## 6. Extractors (port 54400)
```bash
# 6a. metadata: EXIF/ELA/ffprobe/integritate/C2PA ✅
curl -s http://$H:54400/v1/metadata -F "file=@sample.mp4" | python3 -m json.tool
# 6b. NER (GLiNER, multilingv) ✅
curl -s http://$H:54400/v1/ner -H 'Content-Type: application/json' -d '{
"text":"Klaus Iohannis s-a intalnit cu Emmanuel Macron la Bucuresti."
}' | python3 -m json.tool
# 6c. detectie obiecte (YOLOv8) ✅
curl -s http://$H:54400/v1/detect -F "file=@frame.jpg" | python3 -m json.tool
# 6d. OCR (LLM vision) ✅
curl -s http://$H:54400/v1/ocr -F "file=@frame.jpg" | python3 -m json.tool
# 6e. sentiment (deleaga LLM) ✅
curl -s http://$H:54400/v1/sentiment -H 'Content-Type: application/json' -d '{
"text":"Produsul este excelent, sunt foarte multumit!"
}' | python3 -m json.tool
```
> Așteptat: `{"ok":true,"results":{"label":"pozitiv","score":0.9,"rationale":"..."}}`.
> (Gateway-ul LLM injectează `enable_thinking:false` pe vLLM → JSON curat din modelul de reasoning.)
## 7. Forensic — rPPG / lip-sync / forgery / lighting (port 8085) ✅
```bash
# module disponibile
curl -s http://$H:8085/api/forensic-modules | python3 -m json.tool
# analiza forensica (multipart; campul fisierului se numeste 'video')
curl -s http://$H:8085/api/forensic-evidence \
-F "video=@sample.mp4" -F "modules=m25,m26,m27,m28,m29" -F "every_n_frames=10" \
| python3 -m json.tool
```
> Rulează sincron implicit; returnează `modules_run`, `n_frames_extracted`, evidence per modul.
> Pentru async: adaugă `-F "async_mode=true"` → primești `job_id`, apoi
> `GET /api/status/{job_id}` și `GET /api/result/{job_id}`.
## 8. Web — căutare & fact-checking evidence (port 51100) ✅
```bash
# cautare (SearXNG + provideri); ATENTIE: campul e "queries" (lista)
curl -s http://$H:51100/v1/search -H 'Content-Type: application/json' -d '{
"queries":["deepfake detection 2024"],"max_results":5
}' | python3 -m json.tool
# fetch pagini (camp "urls" = LISTA)
curl -s http://$H:51100/v1/fetch -H 'Content-Type: application/json' -d '{
"urls":["https://en.wikipedia.org/wiki/Deepfake"]
}' | python3 -m json.tool
# gather (cautare + fetch + evidence pack); camp "claim", max_search_results >= 5
curl -s http://$H:51100/v1/gather -H 'Content-Type: application/json' -d '{
"claim":"Deepfakes can be detected by AI","max_search_results":5
}' | python3 -m json.tool
curl -s http://$H:51100/v1/info | python3 -m json.tool
```
## 9. Catalog — service discovery (port 11000, doar prin gateway/intern) ✅
`catalog-api` nu e publicat pe host; se accesează prin **gateway** (Bearer) sau din rețea.
```bash
TOK=didi-local-dev-token-123
curl -s http://$H:11000/catalog/v1/status -H "Authorization: Bearer $TOK" | python3 -m json.tool
curl -s http://$H:11000/catalog/v1/components -H "Authorization: Bearer $TOK" | python3 -m json.tool
curl -s http://$H:11000/catalog/v1/models -H "Authorization: Bearer $TOK" | python3 -m json.tool
```
## 10. Gateway — nginx reverse proxy + auth (port 11000) ✅
```bash
TOK=didi-local-dev-token-123
curl -s http://$H:11000/health # fara auth -> ok
curl -s http://$H:11000/web/v1/info -H "Authorization: Bearer $TOK" # rutare -> web
curl -s http://$H:11000/catalog/v1/status -H "Authorization: Bearer $TOK" # rutare -> catalog
curl -s http://$H:11000/web/v1/info # fara token -> 401
```
## 11. Brain — RAG + fact-checking + cache (port 8090) ✅
```bash
# cautare in baza de cunostinte (contract web; camp "queries")
curl -s http://$H:8090/v1/search -H 'Content-Type: application/json' -d '{
"queries":["deepfake"],"max_results":3
}' | python3 -m json.tool
# gather (RAG complet: search -> fetch -> evidence); camp "claim"
curl -s http://$H:8090/v1/gather -H 'Content-Type: application/json' -d '{
"claim":"Deepfakes can be detected by AI"
}' | python3 -m json.tool
# statistici atom store (folosit de dashboard "Brain — fact status")
curl -s http://$H:8090/v1/analysis_atom/stats/extended | python3 -m json.tool
curl -s http://$H:8090/health
```
> `results: []` la început e normal — baza de cunoștințe e goală până se indexează conținut.
---
## Smoke-test rapid (toate componentele, un singur script)
```bash
export H=10.11.10.18; TOK=didi-local-dev-token-123
ok(){ printf "%-26s %s\n" "$1" "$2"; }
ok "llm" "$(curl -s -m60 http://$H:14011/v1/chat/completions -H 'Content-Type: application/json' -d '{"model":"qwen3.5","messages":[{"role":"user","content":"hi"}],"max_tokens":5}' -o /dev/null -w %{http_code})"
ok "embeddings" "$(curl -s -m20 http://$H:14100/v1/embeddings -H 'Content-Type: application/json' -d '{"model":"bge-m3","input":"x"}' -o /dev/null -w %{http_code})"
ok "rerank" "$(curl -s -m20 http://$H:14200/v1/rerank -H 'Content-Type: application/json' -d '{"model":"bge-reranker-v2-m3","query":"a","documents":["a","b"]}' -o /dev/null -w %{http_code})"
ok "audio" "$(curl -s -m10 http://$H:54300/health -o /dev/null -w %{http_code})"
ok "video" "$(curl -s -m10 http://$H:54600/health -o /dev/null -w %{http_code})"
ok "extractors" "$(curl -s -m10 http://$H:54400/health -o /dev/null -w %{http_code})"
ok "forensic" "$(curl -s -m10 http://$H:8085/health -o /dev/null -w %{http_code})"
ok "web" "$(curl -s -m10 http://$H:51100/health -o /dev/null -w %{http_code})"
ok "gateway" "$(curl -s -m10 http://$H:11000/health -o /dev/null -w %{http_code})"
ok "catalog" "$(curl -s -m10 http://$H:11000/catalog/v1/status -H "Authorization: Bearer $TOK" -o /dev/null -w %{http_code})"
ok "brain" "$(curl -s -m10 http://$H:8090/health -o /dev/null -w %{http_code})"
# 200 peste tot = totul up
```
---
## Probleme rezolvate (istoric)
- **sentiment / output JSON** ✅ — gateway-ul LLM injectează acum `enable_thinking:false`
pe vLLM (`LLM_VLLM_DISABLE_THINKING`, default true), deci modelul de reasoning întoarce
JSON curat pentru sentiment/OCR/semantic/brain.
- **catalog `degraded`** ✅ — aliniat prin `CATALOG_LLM_URL/AUDIO_URL/VIDEO_URL/WEB_URL`
în compose → acum `healthy`.
- **brain taxonomy 401** ✅ — creat token atomic (`atomic-server token create`), setat
`ATOMIC_TOKEN`, seed-uită taxonomia (79 tag-uri în atomic), parolele postgres aliniate
(`infra/.env`). Scheduler-ul feed-uiește acum cu `errors=0`.
## Rămas minor (non-blocant)
- **atomic embeddings** — atomic-server logează „OpenRouter API key not configured, skipping
embedding". Atomic-server și-ar face vectori proprii prin OpenRouter; nu e nevoie — brain
folosește bge-m3 local pentru embeddings. Doar un warning în logul atomic.

View file

@ -0,0 +1,298 @@
#!/usr/bin/env bash
# =============================================================================
# DiDi LOT 1 — deploy progresiv „cărămidă cu cărămidă".
# Pentru fiecare componentă: build/pull + download → seed (dacă e cazul) →
# TEST funcțional → doar dacă testul TRECE corect → trece la următoarea.
# Idempotent. STOP la primul test picat.
#
# ./deploy.sh # tot, de la preflight la dashboard
# ./deploy.sh --from web # sare peste cărămizile dinainte de 'web'
# ./deploy.sh --only llm # rulează o singură cărămidă (presupune deps up)
# =============================================================================
set -uo pipefail
cd "$(dirname "$0")"
HERE=$(pwd)
export COMPOSE_PROJECT_NAME=local_gpu_stack
COMPOSE="${HERE}/docker-compose.yml"
ENV_FILE="${HERE}/.env"
MODULES="${HERE}/../modules"
DASH_DIR="${MODULES}/dashboard/deploy"
GW_DIR="${MODULES}/gateway/deploy"
BRAIN_DIR="${MODULES}/didi_brain"
ASSETS="${HERE}/test_assets"
source "${HERE}/lib.sh"
source "${HERE}/seed/seed_catalog.sh"
source "${HERE}/seed/seed_brain.sh"
# ---- .env -------------------------------------------------------------------
[ -f "$ENV_FILE" ] || { cp "${HERE}/.env.example" "$ENV_FILE"; info ".env creat din .env.example"; }
set -a; source "$ENV_FILE"; set +a
# auto-detect HOST_IP daca e gol (si scrie-l inapoi in .env pt interpolarea compose)
if [ -z "${HOST_IP:-}" ]; then
HOST_IP=$(hostname -I 2>/dev/null | awk '{print $1}')
[ -z "$HOST_IP" ] && HOST_IP=$(ip -4 route get 1.1.1.1 2>/dev/null | awk '{for(i=1;i<=NF;i++)if($i=="src")print $(i+1)}')
[ -n "$HOST_IP" ] && { sed -i "s#^HOST_IP=.*#HOST_IP=${HOST_IP}#" "$ENV_FILE"; info "HOST_IP detectat automat: ${HOST_IP}"; }
fi
export HOST_IP
H="${HOST_IP:-127.0.0.1}"
# MODELS_DIR: zona locala unica pentru TOATE modelele (default: ai_platform/models)
if [ -z "${MODELS_DIR:-}" ]; then
MODELS_DIR="$(cd "${HERE}/.." && pwd)/models"
sed -i "s#^MODELS_DIR=.*#MODELS_DIR=${MODELS_DIR}#" "$ENV_FILE"
info "MODELS_DIR: ${MODELS_DIR}"
fi
mkdir -p "$MODELS_DIR"; export MODELS_DIR
# ---- arg parsing ------------------------------------------------------------
FROM=""; ONLY=""; INSTALL_DEPS=0
while [ $# -gt 0 ]; do case "$1" in
--from) FROM=$2; shift 2;; --only) ONLY=$2; shift 2;;
--install-deps) INSTALL_DEPS=1; shift;;
*) die "argument necunoscut: $1";; esac; done
SKIP=0; [ -n "$FROM" ] && SKIP=1
should_run(){ # $1 = brick name
[ -n "$ONLY" ] && { [ "$ONLY" = "$1" ]; return; }
[ "$SKIP" = 1 ] && { [ "$FROM" = "$1" ] && SKIP=0; }
[ "$SKIP" = 0 ]
}
# =============================================================================
# PREFLIGHT (verifică + instruiește; nu instalează)
# =============================================================================
# =============================================================================
# INSTALL DEPS (optional, --install-deps; necesita sudo; Ubuntu/Debian)
# =============================================================================
install_deps() {
brick "INSTALL DEPS — Docker + NVIDIA Container Toolkit (sudo)"
command -v sudo >/dev/null || die "sudo necesar pentru --install-deps"
if ! command -v docker >/dev/null; then
info "instalez Docker Engine..."; curl -fsSL https://get.docker.com | sudo sh || die "instalare Docker a picat"
sudo usermod -aG docker "$USER" || true
warn "Adăugat în grupul docker — poate fi nevoie de re-login pentru ca 'docker' să meargă fără sudo."
else ok "Docker prezent"; fi
if ! docker info 2>/dev/null | grep -qiE 'nvidia|cdi' && ! sudo docker info 2>/dev/null | grep -qiE 'nvidia|cdi'; then
info "instalez nvidia-container-toolkit..."
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list \
| sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' \
| sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list >/dev/null
sudo apt-get update -y && sudo apt-get install -y nvidia-container-toolkit || die "instalare toolkit a picat"
sudo nvidia-ctk runtime configure --runtime=docker && sudo systemctl restart docker
ok "nvidia-container-toolkit instalat & Docker reconfigurat"
else ok "NVIDIA runtime prezent"; fi
command -v nvidia-smi >/dev/null || warn "Driver NVIDIA lipsește — instalează manual (ex: sudo ubuntu-drivers install) și REBOOT, apoi re-rulează deploy.sh."
}
preflight() {
brick "PREFLIGHT — unelte & GPU"
command -v docker >/dev/null || die "Docker lipsește. Instalează: https://docs.docker.com/engine/install/ubuntu/"
if ! docker ps >/dev/null 2>&1; then
[ "$USE_SG" = 1 ] && warn "rulez docker prin 'sg docker' (userul nu e în grupul docker activ)" \
|| die "Nu pot accesa Docker. Adaugă userul în grup: sudo usermod -aG docker \$USER (apoi re-login)"
fi
docker compose version >/dev/null 2>&1 || die "Docker Compose v2 lipsește (plugin 'docker compose')."
command -v nvidia-smi >/dev/null || die "nvidia-smi lipsește — instalează driverul NVIDIA (>=535) + CUDA."
local ngpu; ngpu=$(nvidia-smi --query-gpu=name --format=csv,noheader 2>/dev/null | wc -l)
[ "$ngpu" -ge 1 ] || die "Niciun GPU detectat de nvidia-smi."
if ! docker info 2>/dev/null | grep -qiE 'nvidia|cdi'; then
die "Runtime-ul NVIDIA pt Docker lipsește. Instalează nvidia-container-toolkit:
https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html
apoi: sudo nvidia-ctk runtime configure --runtime=docker && sudo systemctl restart docker"
fi
command -v curl >/dev/null || die "curl lipsește (apt-get install -y curl)."
command -v python3 >/dev/null || die "python3 lipsește."
local freeg; freeg=$(df -BG --output=avail / | tail -1 | tr -dc '0-9')
[ "${freeg:-0}" -ge 80 ] || warn "Doar ${freeg}G liberi pe / — Qwen are ~72GB. Recomandat >100G."
ok "Docker $(docker --version | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' | head -1) · ${ngpu} GPU · ${freeg}G liberi"
}
# =============================================================================
# NETWORK
# =============================================================================
ensure_network() {
brick "REȚEA — didi-network (una singură, partajată)"
if docker network inspect didi-network >/dev/null 2>&1; then ok "didi-network există"
else docker network create didi-network >/dev/null && ok "didi-network creată"; fi
}
# =============================================================================
# BUILD images (app) + PULL (vLLM/searxng)
# =============================================================================
build_images() {
brick "BUILD — imagini aplicație"
local m
declare -A MAP=(
[llm-inference]=didiai-llm-inference [embeddings]=didiai-embeddings [rerank]=didiai-rerank
[audio]=didiai-audio [video-analysis]=didiai-video-analysis [extractors]=didiai-extractors
[web]=didiai-web [catalog-api]=didiai-catalog-api )
for m in "${!MAP[@]}"; do
printf ' build %-16s ' "$m"
if docker build -q -t "${MAP[$m]}:audit" -f "${MODULES}/$m/deploy/Dockerfile" "${MODULES}/$m" >/tmp/build_$m.log 2>&1; then ok "ok"
else err "FAIL"; tail -15 /tmp/build_$m.log | sed 's/^/ /'; die "build $m a picat"; fi
done
# forensic (layout non-standard: Dockerfile la root)
printf ' build %-16s ' "forensic"
if docker build -q -t didiai-forensic:audit -f "${MODULES}/forensic_features/Dockerfile" "${MODULES}/forensic_features" >/tmp/build_forensic.log 2>&1; then ok "ok"
else err "FAIL"; tail -15 /tmp/build_forensic.log | sed 's/^/ /'; die "build forensic a picat"; fi
gate "imagini prezente" bash -c 'docker images --format "{{.Repository}}" | grep -q didiai-llm-inference'
}
pull_images() {
brick "PULL — runtime-uri vLLM + SearXNG"
local img
for img in vllm/vllm-openai:v0.8.5 vllm/vllm-openai:qwen3_5 docker.io/searxng/searxng:latest; do
printf ' pull %-34s ' "$img"
if docker image inspect "$img" >/dev/null 2>&1; then ok "deja prezent"
elif docker pull -q "$img" >/dev/null 2>&1; then ok "ok"
else die "pull $img a picat (verifică rețeaua / Docker Hub)"; fi
done
}
# =============================================================================
# PRE-DOWNLOAD modele (cu progres)
# =============================================================================
predownload_models() {
brick "DOWNLOAD — modele în MODELS_DIR (offline-first, un folder per model)"
mkdir -p "${MODELS_DIR}"
# mini-script de download (evită iadul de ghilimele prin sg docker)
local dl="${HERE}/.dl_model.py"
printf 'import os\nfrom huggingface_hub import snapshot_download\nsnapshot_download(os.environ["REPO"])\nprint("OK", os.environ["REPO"])\n' > "$dl"
# repo HF -> subfolder cache (models--<org>--<name>)
local repo cache
for repo in "$BUSTER_MODEL" "$LLM_MODEL" "$EMB_MODEL" "$RERANK_MODEL" "$WHISPER_REPO" "$NER_MODEL" "$NER_BACKBONE"; do
cache="models--$(printf '%s' "$repo" | sed 's#/#--#')"
if [ -d "${MODELS_DIR}/${cache}/snapshots" ]; then
ok "prezent local: ${repo}"
else
info "download: ${repo}"
docker run --rm -e HF_HOME=/c -e "REPO=${repo}" \
-v "${MODELS_DIR}:/c/hub" -v "${dl}:/dl.py" \
--entrypoint python3 vllm/vllm-openai:qwen3_5 /dl.py \
|| die "download ${repo} a picat."
ok "${repo} descărcat"
fi
done
rm -f "$dl"
ok "Toate modelele sunt în ${MODELS_DIR}"
}
# =============================================================================
# TEST helpers (output corect, nu doar 200)
# =============================================================================
t_embeddings(){ post_json "http://$H:14100/v1/embeddings" '{"model":"bge-m3","input":"test"}' 20 | python3 -c 'import sys,json;v=json.load(sys.stdin)["data"][0]["embedding"];assert len(v)==1024;print(f"dim={len(v)}")'; }
t_rerank(){ post_json "http://$H:14200/v1/rerank" '{"model":"bge-reranker-v2-m3","query":"deepfake detection","documents":["o pisica","sistem detectie deepfake video"]}' 20 | python3 -c 'import sys,json;r=json.load(sys.stdin)["results"];assert r[0]["index"]==1;print("top_idx=%d score=%.3f"%(r[0]["index"],r[0]["relevance_score"]))'; }
t_llm(){ post_json "http://$H:14011/v1/chat/completions" '{"model":"qwen3.5","messages":[{"role":"user","content":"Spune un singur cuvant: salut"}],"max_tokens":20}' 90 | python3 -c 'import sys,json;c=json.load(sys.stdin)["choices"][0]["message"]["content"].strip();assert c and not c.lower().startswith("thinking");print(repr(c[:40]))'; }
t_audio(){ curl -s -m120 "http://$H:54300/v1/audio/transcriptions" -F "file=@${ASSETS}/audio.wav" -F "model=${WHISPER_MODEL}" | python3 -c 'import sys,json;t=json.load(sys.stdin)["text"].strip();assert len(t)>0;print(f"text[:40]={t[:40]!r}")'; }
t_video(){ curl -s -m180 "http://$H:54600/analyze/video" -F "file=@${ASSETS}/sample.mp4" | python3 -c 'import sys,json;d=json.load(sys.stdin);assert d["verdict"] in ("REAL","FAKE","UNCERTAIN");print("verdict=%s frames=%s"%(d["verdict"],d.get("frames_analyzed")))'; }
t_ner(){ post_json "http://$H:54400/v1/ner" '{"text":"Klaus Iohannis s-a intalnit cu Emmanuel Macron la Bucuresti."}' 60 | python3 -c 'import sys,json;e=json.load(sys.stdin)["results"]["entities"];assert len(e)>=2;print(f"entities={len(e)}")'; }
t_detect(){ curl -s -m90 "http://$H:54400/v1/detect" -F "file=@${ASSETS}/frame.jpg" | python3 -c 'import sys,json;d=json.load(sys.stdin)["results"]["detections"];assert len(d)>=1;print(f"objects={len(d)}")'; }
t_metadata(){ curl -s -m40 "http://$H:54400/v1/metadata" -F "file=@${ASSETS}/sample.mp4" | python3 -c 'import sys,json;d=json.load(sys.stdin);assert d["analyses"]["integrity"]["ok"];print("integrity ok")'; }
t_sentiment(){ post_json "http://$H:54400/v1/sentiment" '{"text":"Produsul este excelent, sunt foarte multumit!"}' 60 | python3 -c 'import sys,json;d=json.load(sys.stdin);assert d["ok"];print("label=%s"%d["results"].get("label"))'; }
t_forensic(){ curl -s -m15 "http://$H:8085/api/forensic-modules" | python3 -c 'import sys,json;m=json.load(sys.stdin)["available_modules"];assert len(m)>=3;print(f"modules={len(m)}")'; }
t_web(){ post_json "http://$H:51100/v1/search" '{"queries":["deepfake detection"],"max_results":3}' 40 | python3 -c 'import sys,json;r=json.load(sys.stdin)["results"];assert len(r)>0;print(f"results={len(r)}")'; }
t_catalog(){ curl -s -m15 "http://$H:11000/catalog/v1/status" -H "Authorization: Bearer ${GATEWAY_API_TOKEN}" | python3 -c 'import sys,json;d=json.load(sys.stdin);assert d["status"]=="healthy";print("catalog healthy")'; }
t_gateway(){ local a b; a=$(curl -s -m10 -o /dev/null -w '%{http_code}' "http://$H:11000/health"); b=$(curl -s -m10 -o /dev/null -w '%{http_code}' "http://$H:11000/web/v1/info"); [ "$a" = 200 ] && [ "$b" = 401 ] && echo "health=200 noauth=401" || { echo "health=$a noauth=$b (astept 200/401)"; return 1; }; }
t_brain(){ curl -s -m15 "http://$H:8090/v1/analysis_atom/stats/extended" | python3 -c 'import sys,json;json.load(sys.stdin);print("stats ok")' && post_json "http://$H:8090/v1/search" '{"queries":["deepfake"],"max_results":2}' 30 | python3 -c 'import sys,json;assert "results" in json.load(sys.stdin);print("search ok")'; }
t_dashboard(){ curl -s -m15 "http://$H:51300/api/monitoring/services" | python3 -c 'import sys,json;d=json.load(sys.stdin);s=d if isinstance(d,list) else d.get("services",[]);h=sum(x.get("status")=="healthy" for x in s);assert h==len(s) and h>=11,f"{h}/{len(s)}";print(f"{h}/{len(s)} healthy")'; }
prepare_assets() {
mkdir -p "$ASSETS"
[ -f "${ASSETS}/sample.mp4" ] || cp "$(find ${HERE}/../../artefacte_lot1/runs_deepfake -iname '*.mp4' -printf '%s\t%p\n' | sort -n | head -1 | cut -f2)" "${ASSETS}/sample.mp4"
if [ ! -f "${ASSETS}/frame.jpg" ] || [ ! -f "${ASSETS}/audio.wav" ]; then
docker cp "${ASSETS}/sample.mp4" didiAI-extractors:/tmp/s.mp4 >/dev/null
docker exec didiAI-extractors sh -c 'ffmpeg -y -i /tmp/s.mp4 -frames:v 1 /tmp/frame.jpg -vn -ar 16000 -ac 1 -t 8 /tmp/audio.wav' >/dev/null 2>&1
docker cp didiAI-extractors:/tmp/frame.jpg "${ASSETS}/frame.jpg" >/dev/null
docker cp didiAI-extractors:/tmp/audio.wav "${ASSETS}/audio.wav" >/dev/null
fi
}
# =============================================================================
# CĂRĂMIZI (fiecare: up dependențe → health → test → gate)
# =============================================================================
brick_embeddings(){ brick "EMBEDDINGS (bge-m3)"; dc up -d vllm-embeddings embeddings-api
wait_health "http://$H:14100/health" 180 "embeddings-api" || die "embeddings-api nu pornește"; gate "embeddings dim 1024" t_embeddings; }
brick_rerank(){ brick "RERANK (bge-reranker-v2-m3)"; dc up -d vllm-rerank rerank-api
wait_health "http://$H:14200/health" 180 "rerank-api" || die "rerank-api nu pornește"; gate "rerank ranking" t_rerank; }
brick_llm(){ brick "LLM (Qwen3.5-35B-A3B)"; dc up -d vllm-qwen llm-api
wait_health "http://$H:14001/v1/models" 900 "vLLM Qwen (load+compile)" || die "vLLM Qwen nu pornește"
wait_health "http://$H:14011/health" 60 "llm-api" || die "llm-api nu pornește"; gate "LLM output curat" t_llm; }
brick_extractors(){ brick "EXTRACTORS (NER/YOLO/metadata/OCR/sentiment)"; dc up -d extractors
wait_health "http://$H:54400/health" 120 "extractors" || die "extractors nu pornește"
prepare_assets
gate "metadata" t_metadata; gate "NER (GLiNER)" t_ner; gate "detect (YOLO)" t_detect; gate "sentiment (LLM)" t_sentiment; }
brick_audio(){ brick "AUDIO (Whisper large-v3-turbo)"; dc up -d audio-api
wait_health "http://$H:54300/health" 240 "audio (download model)" || die "audio nu pornește"; gate "transcriere" t_audio; }
brick_video(){ brick "VIDEO / BusterX++ (deepfake)"; dc up -d vllm-buster video-api
wait_health "http://$H:54500/health" 300 "vLLM BusterX" || die "vLLM BusterX nu pornește"
wait_health "http://$H:54600/health" 60 "video-api" || die "video-api nu pornește"; gate "verdict deepfake" t_video; }
brick_forensic(){ brick "FORENSIC (rPPG/lip-sync/forgery/lighting)"; dc up -d forensic
wait_health "http://$H:8085/health" 120 "forensic" || die "forensic nu pornește"; gate "module forensice" t_forensic; }
brick_web(){ brick "WEB + SearXNG (căutare)"
# curăță o instanță searxng pornită manual (docker run) ca s-o preia compose
if docker inspect didiAI-web-searxng >/dev/null 2>&1 && \
[ -z "$(docker inspect didiAI-web-searxng --format '{{ index .Config.Labels "com.docker.compose.project" }}' 2>/dev/null)" ]; then
docker rm -f didiAI-web-searxng >/dev/null 2>&1 || true
fi
dc up -d searxng web-api
wait_health "http://$H:51100/health" 120 "web-api" || die "web-api nu pornește"
sleep 8; gate "căutare web" t_web; }
brick_catalog(){ brick "CATALOG (service discovery)"; dc up -d catalog-api
sleep 6; } # health-ul agregat depinde de gateway; testat după gateway
brick_gateway(){ brick "GATEWAY (nginx + auth)"
( cd "$GW_DIR"; grep -q . .env 2>/dev/null || echo "GATEWAY_API_TOKEN=${GATEWAY_API_TOKEN}" > .env; docker compose up -d )
wait_health "http://$H:11000/health" 60 "gateway" || die "gateway nu pornește"
gate "gateway auth+rutare" t_gateway; gate "catalog agregat" t_catalog; }
brick_brain(){ brick "BRAIN (RAG + fact-check + cache)"
# render env din master, up postgres+atomic, aliniază parola, token, taxonomie, up api+scheduler
brain_write_env "$POSTGRES_PASSWORD" "${ATOMIC_TOKEN:-PLACEHOLDER}"
( cd "$BRAIN_DIR"; docker compose -f infra/docker-compose.yml up -d postgres atomic-server )
wait_health "http://$H:8088/health" 120 "atomic-server" || warn "atomic health întârzie"
brain_align_pg "$POSTGRES_PASSWORD"
local tok; tok=$(brain_ensure_token "$POSTGRES_PASSWORD" "${ATOMIC_TOKEN:-}")
[ -n "$tok" ] || die "nu am putut crea tokenul atomic"
if [ "$tok" != "${ATOMIC_TOKEN:-}" ]; then
sed -i "s#^ATOMIC_TOKEN=.*#ATOMIC_TOKEN=${tok}#" "$ENV_FILE"; ATOMIC_TOKEN="$tok"; ok "token atomic creat & salvat în .env"
fi
brain_write_env "$POSTGRES_PASSWORD" "$tok"
( cd "$BRAIN_DIR"; docker compose -f infra/docker-compose.yml up -d brain-api scheduler )
wait_health "http://$H:8090/health" 120 "brain-api" || die "brain-api nu pornește"
brain_seed_taxonomy
( cd "$BRAIN_DIR"; docker compose -f infra/docker-compose.yml restart brain-api scheduler ) >/dev/null
wait_health "http://$H:8090/health" 60 "brain-api (post-seed)" || die "brain-api nu revine"
gate "brain stats+search" t_brain; }
brick_dashboard(){ brick "DASHBOARD (UI + monitorizare)"
( cd "$DASH_DIR"
cat > .env <<EOF
DASHBOARD_DB_USER=${DASHBOARD_DB_USER}
DASHBOARD_DB_PASSWORD=${DASHBOARD_DB_PASSWORD}
DASHBOARD_DB_NAME=${DASHBOARD_DB_NAME}
DASHBOARD_EXTERNAL_URL=http://${H}:51300
DASHBOARD_STAGING_MODE=${DASHBOARD_STAGING_MODE}
VITE_STAGING_MODE=${DASHBOARD_STAGING_MODE}
DASHBOARD_LLM_API_URL=http://didiAI-llm-api:14011
DASHBOARD_EMBEDDINGS_HEALTH_URL=http://didiAI-embeddings-api:14100/health
DASHBOARD_RERANK_HEALTH_URL=http://didiAI-rerank-api:14200/health
DASHBOARD_AUDIO_HEALTH_URL=http://didiAI-audio:54300/v1/info
DASHBOARD_VIDEO_HEALTH_URL=http://didiAI-video-api:54600/v1/info
DASHBOARD_EXTRACTORS_HEALTH_URL=http://didiAI-extractors:54400/health
DASHBOARD_FORENSIC_HEALTH_URL=http://didiAI-forensic:8080/health
DASHBOARD_WEB_HEALTH_URL=http://didiAI-web-api:51100/v1/info
EOF
docker compose --profile dashboard up -d --build )
wait_health "http://$H:51300/health" 120 "dashboard" || die "dashboard nu pornește"
seed_catalog
sleep 5; gate "dashboard 11/11 healthy" t_dashboard; }
# =============================================================================
# RUN
# =============================================================================
[ "$INSTALL_DEPS" = 1 ] && install_deps
if [ -z "$ONLY" ]; then preflight; ensure_network; build_images; pull_images; predownload_models; fi
for b in embeddings rerank llm extractors audio video forensic web catalog gateway brain dashboard; do
should_run "$b" && "brick_$b"
done
brick "GATA"
ok "Toate cărămizile au trecut testele. Dashboard: http://${H}:51300/admin-ai/"
echo "Smoke-test complet: vezi ${HERE}/TESTING.md"

View file

@ -0,0 +1,310 @@
# Local GPU inference stack — 10.11.10.18 (2x H200 NVL)
# GPU 0: Qwen3.5-35B-A3B (LLM) + Whisper (audio)
# GPU 1: BusterX++ (deepfake video) + bge-m3 (embeddings) + bge-reranker-v2-m3 (rerank)
# All services on the shared `didi-network` so the dashboard can probe them by name.
networks:
didi-network:
external: true
services:
# ===================== vLLM model servers =====================
# BusterX++ — served OFFLINE from the on-disk HF cache (no download)
vllm-buster:
container_name: didiAI-video-vllm-buster
# qwen3_5 image (vLLM 0.16 / transformers 4.57) — BusterX++ config was saved
# with transformers 5.x and the older 0.8.5 image can't parse its nested text_config.
image: vllm/vllm-openai:qwen3_5
networks: [didi-network]
ports: ["54500:54500"]
volumes:
- ${MODELS_DIR}:/root/.cache/huggingface/hub:ro
environment:
- HF_HOME=/root/.cache/huggingface
- HF_HUB_OFFLINE=1
- TRANSFORMERS_OFFLINE=1
command: >
--model l8cv/BusterX_plusplus
--host 0.0.0.0 --port 54500
--served-model-name busterx
--tensor-parallel-size 1
--max-model-len 32768
--gpu-memory-utilization 0.25
--trust-remote-code
--enable-prefix-caching
--disable-log-requests
deploy:
resources:
reservations:
devices:
- driver: nvidia
device_ids: ['1']
capabilities: [gpu]
restart: unless-stopped
# Qwen3.5-35B-A3B — downloads ~72GB on first start
vllm-qwen:
container_name: didiAI-vllm-qwen3.5
image: vllm/vllm-openai:qwen3_5
networks: [didi-network]
ports: ["14001:14001"]
volumes:
- ${MODELS_DIR}:/root/.cache/huggingface/hub:ro
environment:
- HF_HOME=/root/.cache/huggingface
- HF_HUB_OFFLINE=1
- TRANSFORMERS_OFFLINE=1
- VLLM_ALLOW_LONG_MAX_MODEL_LEN=1
command: >
--model Qwen/Qwen3.5-35B-A3B
--host 0.0.0.0 --port 14001
--served-model-name qwen3.5
--tensor-parallel-size 1
--max-model-len 32000
--gpu-memory-utilization 0.65
--trust-remote-code
--disable-log-requests
deploy:
resources:
reservations:
devices:
- driver: nvidia
device_ids: ['0']
capabilities: [gpu]
restart: unless-stopped
# bge-m3 embeddings — downloads ~2GB on first start
vllm-embeddings:
container_name: didiAI-vllm-embeddings
image: vllm/vllm-openai:v0.8.5
networks: [didi-network]
ports: ["14101:14101"]
volumes:
- ${MODELS_DIR}:/root/.cache/huggingface/hub:ro
environment:
- HF_HOME=/root/.cache/huggingface
- HF_HUB_OFFLINE=1
- TRANSFORMERS_OFFLINE=1
command: >
--model BAAI/bge-m3
--host 0.0.0.0 --port 14101
--served-model-name bge-m3
--task embed
--gpu-memory-utilization 0.10
--trust-remote-code
--disable-log-requests
deploy:
resources:
reservations:
devices:
- driver: nvidia
device_ids: ['1']
capabilities: [gpu]
restart: unless-stopped
# bge-reranker-v2-m3 — downloads ~2GB on first start
vllm-rerank:
container_name: didiAI-vllm-rerank
image: vllm/vllm-openai:v0.8.5
networks: [didi-network]
ports: ["14201:14201"]
volumes:
- ${MODELS_DIR}:/root/.cache/huggingface/hub:ro
environment:
- HF_HOME=/root/.cache/huggingface
- HF_HUB_OFFLINE=1
- TRANSFORMERS_OFFLINE=1
command: >
--model BAAI/bge-reranker-v2-m3
--host 0.0.0.0 --port 14201
--served-model-name bge-reranker-v2-m3
--task score
--gpu-memory-utilization 0.10
--trust-remote-code
--disable-log-requests
deploy:
resources:
reservations:
devices:
- driver: nvidia
device_ids: ['1']
capabilities: [gpu]
restart: unless-stopped
# ===================== FastAPI app wrappers =====================
llm-api:
container_name: didiAI-llm-api
image: didiai-llm-inference:audit
networks: [didi-network]
ports: ["14011:14011"]
environment:
- LLM_HOST=0.0.0.0
- LLM_PORT=14011
- LLM_DEFAULT_BACKEND=vllm
- LLM_ENABLE_VLLM=true
- LLM_ENABLE_LLAMACPP=false
- LLM_VLLM_BASE_URL=http://didiAI-vllm-qwen3.5:14001
- LLM_DEFAULT_MODEL=qwen3.5
- LLM_EXTERNAL_URL=http://${HOST_IP}:14011
restart: unless-stopped
embeddings-api:
container_name: didiAI-embeddings-api
image: didiai-embeddings:audit
networks: [didi-network]
ports: ["14100:14100"]
environment:
- EMB_HOST=0.0.0.0
- EMB_PORT=14100
- EMB_DEFAULT_BACKEND=vllm
- EMB_ENABLE_VLLM=true
- EMB_ENABLE_LLAMACPP=false
- EMB_VLLM_BASE_URL=http://didiAI-vllm-embeddings:14101
- EMB_VLLM_MODEL=bge-m3
- EMB_EXTERNAL_URL=http://${HOST_IP}:14100
restart: unless-stopped
rerank-api:
container_name: didiAI-rerank-api
image: didiai-rerank:audit
networks: [didi-network]
ports: ["14200:14200"]
environment:
- RERANK_HOST=0.0.0.0
- RERANK_PORT=14200
- RERANK_DEFAULT_BACKEND=vllm
- RERANK_ENABLE_VLLM=true
- RERANK_ENABLE_LLAMACPP=false
- RERANK_VLLM_BASE_URL=http://didiAI-vllm-rerank:14201
- RERANK_VLLM_MODEL=bge-reranker-v2-m3
- RERANK_EXTERNAL_URL=http://${HOST_IP}:14200
restart: unless-stopped
video-api:
container_name: didiAI-video-api
image: didiai-video-analysis:audit
networks: [didi-network]
ports: ["54600:54600"]
volumes:
- ./runs:/app/runs
environment:
- VIDEO_ANALYSIS_HOST=0.0.0.0
- VIDEO_ANALYSIS_PORT=54600
- VIDEO_ANALYSIS_EXTERNAL_URL=http://${HOST_IP}:54600
- VIDEO_ANALYSIS_VLLM_BASE_URL=http://didiAI-video-vllm-buster:54500
- VIDEO_ANALYSIS_VLLM_MODEL=busterx
- VIDEO_ANALYSIS_RUNS_DIR=/app/runs
- VIDEO_ANALYSIS_SEMANTIC_LLM_BASE_URL=http://didiAI-llm-api:14011
- VIDEO_ANALYSIS_SEMANTIC_AGGREGATION_MODEL=qwen3.5
- VIDEO_ANALYSIS_SEMANTIC_LLM_API_KEY=none
restart: unless-stopped
# ===================== Non-GPU services =====================
catalog-api:
container_name: didiAI-catalog-api
image: didiai-catalog-api:audit
networks: [didi-network]
environment:
- CATALOG_HOST=0.0.0.0
- CATALOG_PORT=11000
- CATALOG_EXTERNAL_URL=http://${HOST_IP}
- CATALOG_LLM_URL=http://didiAI-llm-api:14011
- CATALOG_AUDIO_URL=http://didiAI-audio:54300
- CATALOG_VIDEO_URL=http://didiAI-video-api:54600
- CATALOG_WEB_URL=http://didiAI-web-api:51100
restart: unless-stopped
extractors:
container_name: didiAI-extractors
image: didiai-extractors:audit
networks: [didi-network]
ports: ["54400:54400"]
volumes:
# GLiNER (+ backbone mdeberta) servit offline din zona locala de modele
- ${MODELS_DIR}:/models-hf/hub:ro
environment:
- EXTRACTORS_HOST=0.0.0.0
- EXTRACTORS_PORT=54400
- EXTRACTORS_LLM_GATEWAY_URL=http://didiAI-llm-api:14011
- EXTRACTORS_DETECT_MODEL=/tmp/yolov8n.pt
- YOLO_CONFIG_DIR=/tmp/ultralytics
- HF_HOME=/models-hf
- HF_HUB_OFFLINE=1
- TRANSFORMERS_OFFLINE=1
restart: unless-stopped
forensic:
container_name: didiAI-forensic
image: didiai-forensic:audit
networks: [didi-network]
ports: ["8085:8080"]
environment:
- API_PORT=8080
restart: unless-stopped
# cloak — scraper SERP stealth (tier-3 fallback pentru web)
cloak:
container_name: didiAI-cloak
image: didiai-cloak:audit
networks: [didi-network]
ports: ["8770:8770"]
environment:
- CLOAK_HOST=0.0.0.0
- CLOAK_PORT=8770
restart: unless-stopped
# SearXNG metasearch (motorul de cautare liber folosit de web-api)
searxng:
container_name: didiAI-web-searxng
image: docker.io/searxng/searxng:latest
networks: [didi-network]
volumes:
- ../../modules/web/deploy/metasearch/searxng:/etc/searxng:ro
environment:
- SEARXNG_BASE_URL=http://${HOST_IP}:8080/
restart: unless-stopped
web-api:
container_name: didiAI-web-api
image: didiai-web:audit
networks: [didi-network]
ports: ["51100:51100"]
depends_on: [searxng]
environment:
- WEB_HOST=0.0.0.0
- WEB_PORT=51100
- WEB_EXTERNAL_URL=http://${HOST_IP}:51100
- WEB_SEARXNG_BASE_URL=http://didiAI-web-searxng:8080
- WEB_LLM_BASE_URL=http://didiAI-llm-api:14011
restart: unless-stopped
# Whisper STT — faster-whisper loads in-process on GPU 0
audio-api:
container_name: didiAI-audio
image: didiai-audio:audit
networks: [didi-network]
ports: ["54300:54300"]
volumes:
# faster-whisper foloseste download_root=cache_dir -> models-- direct sub el (nu /hub)
- ${MODELS_DIR}:/root/.cache/huggingface:ro
environment:
- AUDIO_HOST=0.0.0.0
- AUDIO_PORT=54300
- AUDIO_MODEL=large-v3-turbo
- AUDIO_DEVICE=cuda
- AUDIO_COMPUTE_TYPE=float16
- AUDIO_CACHE_DIR=/root/.cache/huggingface
- HF_HUB_OFFLINE=1
- TRANSFORMERS_OFFLINE=1
- AUDIO_EXTERNAL_URL=http://${HOST_IP}:54300
deploy:
resources:
reservations:
devices:
- driver: nvidia
device_ids: ['0']
capabilities: [gpu]
restart: unless-stopped

View file

@ -0,0 +1,74 @@
#!/usr/bin/env bash
# Shared helpers for the brick-by-brick deploy. Sourced by deploy.sh.
# Logging, test gates, health waits, and a docker shim that works whether or not
# the current shell is in the `docker` group.
set -uo pipefail
# ---- colors ----------------------------------------------------------------
if [ -t 1 ]; then
C_R=$'\e[31m'; C_G=$'\e[32m'; C_Y=$'\e[33m'; C_B=$'\e[36m'; C_D=$'\e[2m'; C_0=$'\e[0m'
else
C_R=; C_G=; C_Y=; C_B=; C_D=; C_0=
fi
log() { printf '%s\n' "$*"; }
info() { printf '%s\n' "${C_B}»${C_0} $*"; }
ok() { printf '%s\n' "${C_G}${C_0} $*"; }
warn() { printf '%s\n' "${C_Y}!${C_0} $*"; }
err() { printf '%s\n' "${C_R}${C_0} $*" >&2; }
brick(){ printf '\n%s\n' "${C_B}━━━ CĂRĂMIDĂ: $* ━━━${C_0}"; }
die() { err "$*"; err "Deploy oprit (gate). Repară și re-rulează — e idempotent."; exit 1; }
# ---- docker shim (portable: plain docker, else `sg docker`) -----------------
USE_SG=0
if ! docker ps >/dev/null 2>&1; then
if command -v sg >/dev/null 2>&1 && sg docker -c 'docker ps' >/dev/null 2>&1; then
USE_SG=1
fi
fi
docker() {
if [ "$USE_SG" = 1 ]; then
local a args=""
for a in "$@"; do args+=" $(printf '%q' "$a")"; done
sg docker -c "docker$args"
else
command docker "$@"
fi
}
# ---- compose wrapper -------------------------------------------------------
# COMPOSE is exported by deploy.sh
dc() { docker compose -f "$COMPOSE" "$@"; }
# ---- health / test helpers -------------------------------------------------
# wait_health <url> <timeout_s> <label>
wait_health() {
local url=$1 timeout=${2:-120} label=${3:-$1} i=0
printf ' %s' "aștept $label "
while [ "$i" -lt "$timeout" ]; do
if [ "$(curl -s -m5 -o /dev/null -w '%{http_code}' "$url" 2>/dev/null)" = "200" ]; then
printf ' %sup%s\n' "$C_G" "$C_0"; return 0
fi
printf '.'; sleep 3; i=$((i+3))
done
printf '\n'; return 1
}
# gate <label> <test-command...>
# Runs the test; on non-zero exit, STOPS the whole deploy.
gate() {
local label=$1; shift
printf ' test: %-22s ' "$label"
local out rc
out=$("$@" 2>&1); rc=$?
if [ "$rc" -eq 0 ]; then
ok "PASS ${C_D}${out}${C_0}"
return 0
fi
err "FAIL"
printf '%s\n' "$out" | sed 's/^/ /'
die "Testul '$label' nu a trecut."
}
# json_field <url> <python-expr-on-d> -> prints value, exit 1 on error
post_json() { curl -s -m"${4:-60}" "$1" -H 'Content-Type: application/json' -d "$2"; }

View file

@ -0,0 +1,422 @@
{
"generated_at": "2026-06-27T14:46:37.623279+03:00",
"host": "10.11.10.18",
"summary": {
"total": 30,
"passed": 30,
"failed": 0
},
"tests": [
{
"id": "llm",
"name": "LLM (Qwen3.5-35B-A3B)",
"endpoint": "POST /v1/chat/completions",
"http_code": 200,
"ok": true,
"latency_ms": 342.2,
"result": {
"reply": "Un deepfake este o media manipulat, realizat prin tehnologia inteligenței artificiale, care înlocuiează cu realism înspăimântător fața sau vocea unei persoane cu cea a altcuiva."
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "llm_completions",
"name": "LLM · completions",
"endpoint": "POST /v1/completions",
"http_code": 200,
"ok": true,
"latency_ms": 111.8,
"result": {
"text": " un oras care atrage foarte multi turisti si"
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "llm_info",
"name": "LLM · info",
"endpoint": "GET /v1/info",
"http_code": 200,
"ok": true,
"latency_ms": 14.5,
"result": {
"keys": [
"resource",
"models",
"functions"
]
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "embeddings",
"name": "Embeddings (bge-m3)",
"endpoint": "POST /v1/embeddings",
"http_code": 200,
"ok": true,
"latency_ms": 43.8,
"result": {
"dim": 1024
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "rerank",
"name": "Rerank (bge-reranker-v2-m3)",
"endpoint": "POST /v1/rerank",
"http_code": 200,
"ok": true,
"latency_ms": 39.3,
"result": {
"top_index": 1,
"score": 0.626
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "audio",
"name": "Audio (Whisper large-v3-turbo)",
"endpoint": "POST /v1/audio/transcriptions",
"http_code": 200,
"ok": true,
"latency_ms": 371.3,
"result": {
"text": "Nici nu mai știi cine este de fapt agresorul și agresatul. De ce? Simplu. Pentru că în acest moment, Uniunea Europeană...",
"language": "ro"
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "video",
"name": "Video/BusterX (deepfake)",
"endpoint": "POST /analyze/video",
"http_code": 200,
"ok": true,
"latency_ms": 1389.8,
"result": {
"verdict": "REAL",
"frames_analyzed": 16
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "video_semantic",
"name": "Video · semantic",
"endpoint": "POST /analyze/video/semantic",
"http_code": 200,
"ok": true,
"latency_ms": 18390.9,
"result": {
"num_chunks": 5,
"summary": "The video presents a continuous, polemical monologue delivered by a male speaker in a studio setting, likely a podcast o"
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "extractors_metadata",
"name": "Extractors · metadata",
"endpoint": "POST /v1/metadata",
"http_code": 200,
"ok": true,
"latency_ms": 284.1,
"result": {
"media_type": "video",
"sha256": "27d6c00786e63905"
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "extractors_ner",
"name": "Extractors · NER (GLiNER)",
"endpoint": "POST /v1/ner",
"http_code": 200,
"ok": true,
"latency_ms": 262.9,
"result": {
"entities": [
"Klaus Iohannis:persoană",
"Emmanuel Macron:persoană",
"Bucuresti:locație"
]
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "extractors_detect",
"name": "Extractors · detect (YOLOv8)",
"endpoint": "POST /v1/detect",
"http_code": 200,
"ok": true,
"latency_ms": 77.1,
"result": {
"objects": 4,
"labels": [
"person",
"person",
"person",
"person"
]
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "extractors_ocr",
"name": "Extractors · OCR (LLM vision)",
"endpoint": "POST /v1/ocr",
"http_code": 200,
"ok": true,
"latency_ms": 684.6,
"result": {
"text": "Asa arată\nANTI-ROMÂNISMUL\n22:08\nVOCEA LIBERTĂȚII\n15 AUG. 25\nALASKA\nTRUMP VS PUTIN\nTikTok\n@ ovi1623\n>> Da. Deci acuma nici nu mai știi cine"
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "extractors_sentiment",
"name": "Extractors · sentiment (LLM)",
"endpoint": "POST /v1/sentiment",
"http_code": 200,
"ok": true,
"latency_ms": 551.5,
"result": {
"label": "pozitiv",
"score": 0.9
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "forensic",
"name": "Forensic · module list",
"endpoint": "GET /api/forensic-modules",
"http_code": 200,
"ok": true,
"latency_ms": 9.5,
"result": {
"modules": [
"m25",
"m26",
"m27",
"m28",
"m29"
]
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "forensic_evidence",
"name": "Forensic · analiza reala",
"endpoint": "POST /api/forensic-evidence",
"http_code": 200,
"ok": true,
"latency_ms": 9563.6,
"result": {
"modules_run": [
"m25",
"m26",
"m27",
"m28",
"m29"
],
"frames": 131
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "web",
"name": "Web · search (SearXNG)",
"endpoint": "POST /v1/search",
"http_code": 200,
"ok": true,
"latency_ms": 1303.6,
"result": {
"results": 3,
"first": "AI Deepfake Detection Online Free - Images, Videos, Voices"
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "web_fetch",
"name": "Web · fetch",
"endpoint": "POST /v1/fetch",
"http_code": 200,
"ok": true,
"latency_ms": 731.4,
"result": {
"bytes": 113707
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "web_gather",
"name": "Web · gather (evidence)",
"endpoint": "POST /v1/gather",
"http_code": 200,
"ok": true,
"latency_ms": 3115.4,
"result": {
"evidence_items": 5,
"urls_found": 5
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "catalog",
"name": "Catalog · status",
"endpoint": "GET /catalog/v1/status",
"http_code": 200,
"ok": true,
"latency_ms": 81.8,
"result": {
"status": "healthy",
"components": [
"llm-inference",
"audio-transcription",
"video-analysis",
"web-factcheck"
]
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "catalog_components",
"name": "Catalog · components",
"endpoint": "GET /catalog/v1/components",
"http_code": 200,
"ok": true,
"latency_ms": 76.0,
"result": {
"components": 4
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "catalog_models",
"name": "Catalog · models",
"endpoint": "GET /catalog/v1/models",
"http_code": 200,
"ok": true,
"latency_ms": 67.9,
"result": {
"models": 3
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "gateway",
"name": "Gateway (auth+rutare)",
"endpoint": "GET /health + 401",
"http_code": 200,
"ok": true,
"latency_ms": 8.8,
"result": {
"health": 200,
"web_no_auth": 401
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "brain_atoms",
"name": "Brain · analysis_atom",
"endpoint": "GET /v1/analysis_atom/stats/extended",
"http_code": 200,
"ok": true,
"latency_ms": 15.3,
"result": {
"total_atoms": 9,
"by_tier": {
"gold": 4,
"silver": 3,
"bronze": 2
}
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "brain_facts",
"name": "Brain · fact_status",
"endpoint": "GET /v1/fact_status/list",
"http_code": 200,
"ok": true,
"latency_ms": 14.5,
"result": {
"total": 7
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "brain_vcache",
"name": "Brain · verification_cache",
"endpoint": "GET /v1/verification_cache/list",
"http_code": 200,
"ok": true,
"latency_ms": 11.3,
"result": {
"total": 5
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "brain_search",
"name": "Brain · search (RAG)",
"endpoint": "POST /v1/search",
"http_code": 200,
"ok": true,
"latency_ms": 3044.4,
"result": {
"total_results": 0,
"cache": "MISS"
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "brain_gather",
"name": "Brain · gather (RAG)",
"endpoint": "POST /v1/gather",
"http_code": 200,
"ok": true,
"latency_ms": 3038.3,
"result": {
"evidence_items": 0,
"urls": 0
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "dashboard",
"name": "Dashboard · health",
"endpoint": "GET /health",
"http_code": 200,
"ok": true,
"latency_ms": 19.3,
"result": {
"status": "healthy",
"db": true
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "dashboard_monitoring",
"name": "Dashboard · monitoring 11/11",
"endpoint": "GET /api/monitoring/services",
"http_code": 200,
"ok": true,
"latency_ms": 107.6,
"result": {
"healthy": "11/11"
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "cloak",
"name": "Cloak · SERP scraper",
"endpoint": "POST /v1/search",
"http_code": 200,
"ok": true,
"latency_ms": 1991.0,
"result": {
"results": 3,
"first": "Deepfake - Wikipedia"
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
}
]
}

View file

@ -0,0 +1,249 @@
{
"generated_at": "2026-06-27T14:34:04.610734+03:00",
"host": "10.11.10.18",
"summary": {
"total": 17,
"passed": 17,
"failed": 0
},
"tests": [
{
"id": "llm",
"name": "LLM (Qwen3.5-35B-A3B)",
"endpoint": "POST /v1/chat/completions",
"http_code": 200,
"ok": true,
"latency_ms": 424.6,
"result": {
"reply": "Un deepfake este un conținut media manipulat prin tehnologia inteligenței artificiale, care înlocuiește fața sau vocea unei persoane cu cea a alteia pentru a crea imagini sau înregistrări video extrem de realiste, dar falsificate."
},
"checked_at": "2026-06-27T14:34:04.610734+03:00"
},
{
"id": "embeddings",
"name": "Embeddings (bge-m3)",
"endpoint": "POST /v1/embeddings",
"http_code": 200,
"ok": true,
"latency_ms": 42.5,
"result": {
"dim": 1024
},
"checked_at": "2026-06-27T14:34:04.610734+03:00"
},
{
"id": "rerank",
"name": "Rerank (bge-reranker-v2-m3)",
"endpoint": "POST /v1/rerank",
"http_code": 200,
"ok": true,
"latency_ms": 33.3,
"result": {
"top_index": 1,
"score": 0.626
},
"checked_at": "2026-06-27T14:34:04.610734+03:00"
},
{
"id": "audio",
"name": "Audio (Whisper large-v3-turbo)",
"endpoint": "POST /v1/audio/transcriptions",
"http_code": 200,
"ok": true,
"latency_ms": 251.9,
"result": {
"text": "Nici nu mai știi cine este de fapt agresorul și agresatul. De ce? Simplu. Pentru că în acest moment, Uniunea Europeană...",
"language": "ro"
},
"checked_at": "2026-06-27T14:34:04.610734+03:00"
},
{
"id": "video",
"name": "Video/BusterX (deepfake)",
"endpoint": "POST /analyze/video",
"http_code": 200,
"ok": true,
"latency_ms": 1355.9,
"result": {
"verdict": "REAL",
"frames_analyzed": 16
},
"checked_at": "2026-06-27T14:34:04.610734+03:00"
},
{
"id": "extractors_metadata",
"name": "Extractors · metadata",
"endpoint": "POST /v1/metadata",
"http_code": 200,
"ok": true,
"latency_ms": 275.1,
"result": {
"media_type": "video",
"sha256": "27d6c00786e63905"
},
"checked_at": "2026-06-27T14:34:04.610734+03:00"
},
{
"id": "extractors_ner",
"name": "Extractors · NER (GLiNER)",
"endpoint": "POST /v1/ner",
"http_code": 200,
"ok": true,
"latency_ms": 813.0,
"result": {
"entities": [
"Klaus Iohannis:persoană",
"Emmanuel Macron:persoană",
"Bucuresti:locație"
]
},
"checked_at": "2026-06-27T14:34:04.610734+03:00"
},
{
"id": "extractors_detect",
"name": "Extractors · detect (YOLOv8)",
"endpoint": "POST /v1/detect",
"http_code": 200,
"ok": true,
"latency_ms": 1464.4,
"result": {
"objects": 4,
"labels": [
"person",
"person",
"person",
"person"
]
},
"checked_at": "2026-06-27T14:34:04.610734+03:00"
},
{
"id": "extractors_ocr",
"name": "Extractors · OCR (LLM vision)",
"endpoint": "POST /v1/ocr",
"http_code": 200,
"ok": true,
"latency_ms": 5621.1,
"result": {
"text": "Asa arată\nANTI-ROMÂNISMUL\n22:08\nVOCEA LIBERTĂȚII\n15 AUG. 25\nALASKA\nTRUMP VS PUTIN\nTikTok\n@ ovi1623\n>> Da. Deci acuma nici nu mai știi cine"
},
"checked_at": "2026-06-27T14:34:04.610734+03:00"
},
{
"id": "extractors_sentiment",
"name": "Extractors · sentiment (LLM)",
"endpoint": "POST /v1/sentiment",
"http_code": 200,
"ok": true,
"latency_ms": 551.3,
"result": {
"label": "pozitiv",
"score": 0.9
},
"checked_at": "2026-06-27T14:34:04.610734+03:00"
},
{
"id": "forensic",
"name": "Forensic (rPPG/lip-sync/...)",
"endpoint": "GET /api/forensic-modules",
"http_code": 200,
"ok": true,
"latency_ms": 6.3,
"result": {
"modules": [
"m25",
"m26",
"m27",
"m28",
"m29"
]
},
"checked_at": "2026-06-27T14:34:04.610734+03:00"
},
{
"id": "web",
"name": "Web search (SearXNG)",
"endpoint": "POST /v1/search",
"http_code": 200,
"ok": true,
"latency_ms": 1504.0,
"result": {
"results": 3,
"first": "AI Deepfake Detection Online Free - Images, Videos, Voices"
},
"checked_at": "2026-06-27T14:34:04.610734+03:00"
},
{
"id": "catalog",
"name": "Catalog (via gateway)",
"endpoint": "GET /catalog/v1/status",
"http_code": 200,
"ok": true,
"latency_ms": 68.9,
"result": {
"status": "healthy",
"components": [
"llm-inference",
"audio-transcription",
"video-analysis",
"web-factcheck"
]
},
"checked_at": "2026-06-27T14:34:04.610734+03:00"
},
{
"id": "gateway",
"name": "Gateway (auth+rutare)",
"endpoint": "GET /health + 401",
"http_code": 200,
"ok": true,
"latency_ms": 7.3,
"result": {
"health": 200,
"web_no_auth": 401
},
"checked_at": "2026-06-27T14:34:04.610734+03:00"
},
{
"id": "brain_atoms",
"name": "Brain · analysis_atom",
"endpoint": "GET /v1/analysis_atom/stats/extended",
"http_code": 200,
"ok": true,
"latency_ms": 54.1,
"result": {
"total_atoms": 9,
"by_tier": {
"gold": 4,
"silver": 3,
"bronze": 2
}
},
"checked_at": "2026-06-27T14:34:04.610734+03:00"
},
{
"id": "brain_facts",
"name": "Brain · fact_status",
"endpoint": "GET /v1/fact_status/list",
"http_code": 200,
"ok": true,
"latency_ms": 16.3,
"result": {
"total": 7
},
"checked_at": "2026-06-27T14:34:04.610734+03:00"
},
{
"id": "brain_vcache",
"name": "Brain · verification_cache",
"endpoint": "GET /v1/verification_cache/list",
"http_code": 200,
"ok": true,
"latency_ms": 22.8,
"result": {
"total": 5
},
"checked_at": "2026-06-27T14:34:04.610734+03:00"
}
]
}

View file

@ -0,0 +1,422 @@
{
"generated_at": "2026-06-27T14:43:58.666410+03:00",
"host": "10.11.10.18",
"summary": {
"total": 30,
"passed": 30,
"failed": 0
},
"tests": [
{
"id": "llm",
"name": "LLM (Qwen3.5-35B-A3B)",
"endpoint": "POST /v1/chat/completions",
"http_code": 200,
"ok": true,
"latency_ms": 400.6,
"result": {
"reply": "Deepfake este o tehnică de manipulare a conținutului multimedia care utilizează inteligența artificială pentru a înlocua fața sau vocea unei persoane reale cu cea a alteia, creând astfel imagini sau videoclipuri extrem de realiste dar false."
},
"checked_at": "2026-06-27T14:43:58.666410+03:00"
},
{
"id": "llm_completions",
"name": "LLM · completions",
"endpoint": "POST /v1/completions",
"http_code": 200,
"ok": true,
"latency_ms": 122.0,
"result": {
"text": " Bucuresti.\n\n<think>\n\n</think>\n\nAfirm"
},
"checked_at": "2026-06-27T14:43:58.666410+03:00"
},
{
"id": "llm_info",
"name": "LLM · info",
"endpoint": "GET /v1/info",
"http_code": 200,
"ok": true,
"latency_ms": 12.9,
"result": {
"keys": [
"resource",
"models",
"functions"
]
},
"checked_at": "2026-06-27T14:43:58.666410+03:00"
},
{
"id": "embeddings",
"name": "Embeddings (bge-m3)",
"endpoint": "POST /v1/embeddings",
"http_code": 200,
"ok": true,
"latency_ms": 41.8,
"result": {
"dim": 1024
},
"checked_at": "2026-06-27T14:43:58.666410+03:00"
},
{
"id": "rerank",
"name": "Rerank (bge-reranker-v2-m3)",
"endpoint": "POST /v1/rerank",
"http_code": 200,
"ok": true,
"latency_ms": 39.7,
"result": {
"top_index": 1,
"score": 0.626
},
"checked_at": "2026-06-27T14:43:58.666410+03:00"
},
{
"id": "audio",
"name": "Audio (Whisper large-v3-turbo)",
"endpoint": "POST /v1/audio/transcriptions",
"http_code": 200,
"ok": true,
"latency_ms": 372.5,
"result": {
"text": "Nici nu mai știi cine este de fapt agresorul și agresatul. De ce? Simplu. Pentru că în acest moment, Uniunea Europeană...",
"language": "ro"
},
"checked_at": "2026-06-27T14:43:58.666410+03:00"
},
{
"id": "video",
"name": "Video/BusterX (deepfake)",
"endpoint": "POST /analyze/video",
"http_code": 200,
"ok": true,
"latency_ms": 1384.5,
"result": {
"verdict": "REAL",
"frames_analyzed": 16
},
"checked_at": "2026-06-27T14:43:58.666410+03:00"
},
{
"id": "video_semantic",
"name": "Video · semantic",
"endpoint": "POST /analyze/video/semantic",
"http_code": 200,
"ok": true,
"latency_ms": 27228.4,
"result": {
"num_chunks": 5,
"summary": "[Aggregation failed: Client error '404 Not Found' for url 'http://didiai-llm-api:14011/v1/v1/chat/completions'\nFor more "
},
"checked_at": "2026-06-27T14:43:58.666410+03:00"
},
{
"id": "extractors_metadata",
"name": "Extractors · metadata",
"endpoint": "POST /v1/metadata",
"http_code": 200,
"ok": true,
"latency_ms": 320.3,
"result": {
"media_type": "video",
"sha256": "27d6c00786e63905"
},
"checked_at": "2026-06-27T14:43:58.666410+03:00"
},
{
"id": "extractors_ner",
"name": "Extractors · NER (GLiNER)",
"endpoint": "POST /v1/ner",
"http_code": 200,
"ok": true,
"latency_ms": 115.1,
"result": {
"entities": [
"Klaus Iohannis:persoană",
"Emmanuel Macron:persoană",
"Bucuresti:locație"
]
},
"checked_at": "2026-06-27T14:43:58.666410+03:00"
},
{
"id": "extractors_detect",
"name": "Extractors · detect (YOLOv8)",
"endpoint": "POST /v1/detect",
"http_code": 200,
"ok": true,
"latency_ms": 53.7,
"result": {
"objects": 4,
"labels": [
"person",
"person",
"person",
"person"
]
},
"checked_at": "2026-06-27T14:43:58.666410+03:00"
},
{
"id": "extractors_ocr",
"name": "Extractors · OCR (LLM vision)",
"endpoint": "POST /v1/ocr",
"http_code": 200,
"ok": true,
"latency_ms": 701.6,
"result": {
"text": "Asa arată\nANTI-ROMÂNISMUL\n22:08\nVOCEA LIBERTĂȚII\n15 AUG. 25\nALASKA\nTRUMP VS PUTIN\nTikTok\n@ ovi1623\n>> Da. Deci acuma nici nu mai știi cine"
},
"checked_at": "2026-06-27T14:43:58.666410+03:00"
},
{
"id": "extractors_sentiment",
"name": "Extractors · sentiment (LLM)",
"endpoint": "POST /v1/sentiment",
"http_code": 200,
"ok": true,
"latency_ms": 552.9,
"result": {
"label": "pozitiv",
"score": 0.9
},
"checked_at": "2026-06-27T14:43:58.666410+03:00"
},
{
"id": "forensic",
"name": "Forensic · module list",
"endpoint": "GET /api/forensic-modules",
"http_code": 200,
"ok": true,
"latency_ms": 8.7,
"result": {
"modules": [
"m25",
"m26",
"m27",
"m28",
"m29"
]
},
"checked_at": "2026-06-27T14:43:58.666410+03:00"
},
{
"id": "forensic_evidence",
"name": "Forensic · analiza reala",
"endpoint": "POST /api/forensic-evidence",
"http_code": 200,
"ok": true,
"latency_ms": 9039.9,
"result": {
"modules_run": [
"m25",
"m26",
"m27",
"m28",
"m29"
],
"frames": 131
},
"checked_at": "2026-06-27T14:43:58.666410+03:00"
},
{
"id": "web",
"name": "Web · search (SearXNG)",
"endpoint": "POST /v1/search",
"http_code": 200,
"ok": true,
"latency_ms": 841.4,
"result": {
"results": 3,
"first": "AI Deepfake Detection Online Free - Images, Videos, Voices"
},
"checked_at": "2026-06-27T14:43:58.666410+03:00"
},
{
"id": "web_fetch",
"name": "Web · fetch",
"endpoint": "POST /v1/fetch",
"http_code": 200,
"ok": true,
"latency_ms": 554.3,
"result": {
"bytes": 113707
},
"checked_at": "2026-06-27T14:43:58.666410+03:00"
},
{
"id": "web_gather",
"name": "Web · gather (evidence)",
"endpoint": "POST /v1/gather",
"http_code": 200,
"ok": true,
"latency_ms": 2911.9,
"result": {
"evidence_items": 5,
"urls_found": 5
},
"checked_at": "2026-06-27T14:43:58.666410+03:00"
},
{
"id": "catalog",
"name": "Catalog · status",
"endpoint": "GET /catalog/v1/status",
"http_code": 200,
"ok": true,
"latency_ms": 73.8,
"result": {
"status": "healthy",
"components": [
"llm-inference",
"audio-transcription",
"video-analysis",
"web-factcheck"
]
},
"checked_at": "2026-06-27T14:43:58.666410+03:00"
},
{
"id": "catalog_components",
"name": "Catalog · components",
"endpoint": "GET /catalog/v1/components",
"http_code": 200,
"ok": true,
"latency_ms": 70.6,
"result": {
"components": 4
},
"checked_at": "2026-06-27T14:43:58.666410+03:00"
},
{
"id": "catalog_models",
"name": "Catalog · models",
"endpoint": "GET /catalog/v1/models",
"http_code": 200,
"ok": true,
"latency_ms": 55.7,
"result": {
"models": 3
},
"checked_at": "2026-06-27T14:43:58.666410+03:00"
},
{
"id": "gateway",
"name": "Gateway (auth+rutare)",
"endpoint": "GET /health + 401",
"http_code": 200,
"ok": true,
"latency_ms": 7.1,
"result": {
"health": 200,
"web_no_auth": 401
},
"checked_at": "2026-06-27T14:43:58.666410+03:00"
},
{
"id": "brain_atoms",
"name": "Brain · analysis_atom",
"endpoint": "GET /v1/analysis_atom/stats/extended",
"http_code": 200,
"ok": true,
"latency_ms": 36.1,
"result": {
"total_atoms": 9,
"by_tier": {
"gold": 4,
"silver": 3,
"bronze": 2
}
},
"checked_at": "2026-06-27T14:43:58.666410+03:00"
},
{
"id": "brain_facts",
"name": "Brain · fact_status",
"endpoint": "GET /v1/fact_status/list",
"http_code": 200,
"ok": true,
"latency_ms": 13.3,
"result": {
"total": 7
},
"checked_at": "2026-06-27T14:43:58.666410+03:00"
},
{
"id": "brain_vcache",
"name": "Brain · verification_cache",
"endpoint": "GET /v1/verification_cache/list",
"http_code": 200,
"ok": true,
"latency_ms": 13.7,
"result": {
"total": 5
},
"checked_at": "2026-06-27T14:43:58.666410+03:00"
},
{
"id": "brain_search",
"name": "Brain · search (RAG)",
"endpoint": "POST /v1/search",
"http_code": 200,
"ok": true,
"latency_ms": 3040.4,
"result": {
"total_results": 0,
"cache": "MISS"
},
"checked_at": "2026-06-27T14:43:58.666410+03:00"
},
{
"id": "brain_gather",
"name": "Brain · gather (RAG)",
"endpoint": "POST /v1/gather",
"http_code": 200,
"ok": true,
"latency_ms": 3042.0,
"result": {
"evidence_items": 0,
"urls": 0
},
"checked_at": "2026-06-27T14:43:58.666410+03:00"
},
{
"id": "dashboard",
"name": "Dashboard · health",
"endpoint": "GET /health",
"http_code": 200,
"ok": true,
"latency_ms": 14.0,
"result": {
"status": "healthy",
"db": true
},
"checked_at": "2026-06-27T14:43:58.666410+03:00"
},
{
"id": "dashboard_monitoring",
"name": "Dashboard · monitoring 11/11",
"endpoint": "GET /api/monitoring/services",
"http_code": 200,
"ok": true,
"latency_ms": 129.6,
"result": {
"healthy": "11/11"
},
"checked_at": "2026-06-27T14:43:58.666410+03:00"
},
{
"id": "cloak",
"name": "Cloak · SERP scraper",
"endpoint": "POST /v1/search",
"http_code": 200,
"ok": true,
"latency_ms": 2014.3,
"result": {
"results": 3,
"first": "Deepfake - Wikipedia"
},
"checked_at": "2026-06-27T14:43:58.666410+03:00"
}
]
}

View file

@ -0,0 +1,422 @@
{
"generated_at": "2026-06-27T14:46:37.623279+03:00",
"host": "10.11.10.18",
"summary": {
"total": 30,
"passed": 30,
"failed": 0
},
"tests": [
{
"id": "llm",
"name": "LLM (Qwen3.5-35B-A3B)",
"endpoint": "POST /v1/chat/completions",
"http_code": 200,
"ok": true,
"latency_ms": 342.2,
"result": {
"reply": "Un deepfake este o media manipulat, realizat prin tehnologia inteligenței artificiale, care înlocuiează cu realism înspăimântător fața sau vocea unei persoane cu cea a altcuiva."
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "llm_completions",
"name": "LLM · completions",
"endpoint": "POST /v1/completions",
"http_code": 200,
"ok": true,
"latency_ms": 111.8,
"result": {
"text": " un oras care atrage foarte multi turisti si"
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "llm_info",
"name": "LLM · info",
"endpoint": "GET /v1/info",
"http_code": 200,
"ok": true,
"latency_ms": 14.5,
"result": {
"keys": [
"resource",
"models",
"functions"
]
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "embeddings",
"name": "Embeddings (bge-m3)",
"endpoint": "POST /v1/embeddings",
"http_code": 200,
"ok": true,
"latency_ms": 43.8,
"result": {
"dim": 1024
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "rerank",
"name": "Rerank (bge-reranker-v2-m3)",
"endpoint": "POST /v1/rerank",
"http_code": 200,
"ok": true,
"latency_ms": 39.3,
"result": {
"top_index": 1,
"score": 0.626
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "audio",
"name": "Audio (Whisper large-v3-turbo)",
"endpoint": "POST /v1/audio/transcriptions",
"http_code": 200,
"ok": true,
"latency_ms": 371.3,
"result": {
"text": "Nici nu mai știi cine este de fapt agresorul și agresatul. De ce? Simplu. Pentru că în acest moment, Uniunea Europeană...",
"language": "ro"
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "video",
"name": "Video/BusterX (deepfake)",
"endpoint": "POST /analyze/video",
"http_code": 200,
"ok": true,
"latency_ms": 1389.8,
"result": {
"verdict": "REAL",
"frames_analyzed": 16
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "video_semantic",
"name": "Video · semantic",
"endpoint": "POST /analyze/video/semantic",
"http_code": 200,
"ok": true,
"latency_ms": 18390.9,
"result": {
"num_chunks": 5,
"summary": "The video presents a continuous, polemical monologue delivered by a male speaker in a studio setting, likely a podcast o"
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "extractors_metadata",
"name": "Extractors · metadata",
"endpoint": "POST /v1/metadata",
"http_code": 200,
"ok": true,
"latency_ms": 284.1,
"result": {
"media_type": "video",
"sha256": "27d6c00786e63905"
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "extractors_ner",
"name": "Extractors · NER (GLiNER)",
"endpoint": "POST /v1/ner",
"http_code": 200,
"ok": true,
"latency_ms": 262.9,
"result": {
"entities": [
"Klaus Iohannis:persoană",
"Emmanuel Macron:persoană",
"Bucuresti:locație"
]
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "extractors_detect",
"name": "Extractors · detect (YOLOv8)",
"endpoint": "POST /v1/detect",
"http_code": 200,
"ok": true,
"latency_ms": 77.1,
"result": {
"objects": 4,
"labels": [
"person",
"person",
"person",
"person"
]
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "extractors_ocr",
"name": "Extractors · OCR (LLM vision)",
"endpoint": "POST /v1/ocr",
"http_code": 200,
"ok": true,
"latency_ms": 684.6,
"result": {
"text": "Asa arată\nANTI-ROMÂNISMUL\n22:08\nVOCEA LIBERTĂȚII\n15 AUG. 25\nALASKA\nTRUMP VS PUTIN\nTikTok\n@ ovi1623\n>> Da. Deci acuma nici nu mai știi cine"
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "extractors_sentiment",
"name": "Extractors · sentiment (LLM)",
"endpoint": "POST /v1/sentiment",
"http_code": 200,
"ok": true,
"latency_ms": 551.5,
"result": {
"label": "pozitiv",
"score": 0.9
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "forensic",
"name": "Forensic · module list",
"endpoint": "GET /api/forensic-modules",
"http_code": 200,
"ok": true,
"latency_ms": 9.5,
"result": {
"modules": [
"m25",
"m26",
"m27",
"m28",
"m29"
]
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "forensic_evidence",
"name": "Forensic · analiza reala",
"endpoint": "POST /api/forensic-evidence",
"http_code": 200,
"ok": true,
"latency_ms": 9563.6,
"result": {
"modules_run": [
"m25",
"m26",
"m27",
"m28",
"m29"
],
"frames": 131
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "web",
"name": "Web · search (SearXNG)",
"endpoint": "POST /v1/search",
"http_code": 200,
"ok": true,
"latency_ms": 1303.6,
"result": {
"results": 3,
"first": "AI Deepfake Detection Online Free - Images, Videos, Voices"
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "web_fetch",
"name": "Web · fetch",
"endpoint": "POST /v1/fetch",
"http_code": 200,
"ok": true,
"latency_ms": 731.4,
"result": {
"bytes": 113707
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "web_gather",
"name": "Web · gather (evidence)",
"endpoint": "POST /v1/gather",
"http_code": 200,
"ok": true,
"latency_ms": 3115.4,
"result": {
"evidence_items": 5,
"urls_found": 5
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "catalog",
"name": "Catalog · status",
"endpoint": "GET /catalog/v1/status",
"http_code": 200,
"ok": true,
"latency_ms": 81.8,
"result": {
"status": "healthy",
"components": [
"llm-inference",
"audio-transcription",
"video-analysis",
"web-factcheck"
]
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "catalog_components",
"name": "Catalog · components",
"endpoint": "GET /catalog/v1/components",
"http_code": 200,
"ok": true,
"latency_ms": 76.0,
"result": {
"components": 4
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "catalog_models",
"name": "Catalog · models",
"endpoint": "GET /catalog/v1/models",
"http_code": 200,
"ok": true,
"latency_ms": 67.9,
"result": {
"models": 3
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "gateway",
"name": "Gateway (auth+rutare)",
"endpoint": "GET /health + 401",
"http_code": 200,
"ok": true,
"latency_ms": 8.8,
"result": {
"health": 200,
"web_no_auth": 401
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "brain_atoms",
"name": "Brain · analysis_atom",
"endpoint": "GET /v1/analysis_atom/stats/extended",
"http_code": 200,
"ok": true,
"latency_ms": 15.3,
"result": {
"total_atoms": 9,
"by_tier": {
"gold": 4,
"silver": 3,
"bronze": 2
}
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "brain_facts",
"name": "Brain · fact_status",
"endpoint": "GET /v1/fact_status/list",
"http_code": 200,
"ok": true,
"latency_ms": 14.5,
"result": {
"total": 7
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "brain_vcache",
"name": "Brain · verification_cache",
"endpoint": "GET /v1/verification_cache/list",
"http_code": 200,
"ok": true,
"latency_ms": 11.3,
"result": {
"total": 5
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "brain_search",
"name": "Brain · search (RAG)",
"endpoint": "POST /v1/search",
"http_code": 200,
"ok": true,
"latency_ms": 3044.4,
"result": {
"total_results": 0,
"cache": "MISS"
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "brain_gather",
"name": "Brain · gather (RAG)",
"endpoint": "POST /v1/gather",
"http_code": 200,
"ok": true,
"latency_ms": 3038.3,
"result": {
"evidence_items": 0,
"urls": 0
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "dashboard",
"name": "Dashboard · health",
"endpoint": "GET /health",
"http_code": 200,
"ok": true,
"latency_ms": 19.3,
"result": {
"status": "healthy",
"db": true
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "dashboard_monitoring",
"name": "Dashboard · monitoring 11/11",
"endpoint": "GET /api/monitoring/services",
"http_code": 200,
"ok": true,
"latency_ms": 107.6,
"result": {
"healthy": "11/11"
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
},
{
"id": "cloak",
"name": "Cloak · SERP scraper",
"endpoint": "POST /v1/search",
"http_code": 200,
"ok": true,
"latency_ms": 1991.0,
"result": {
"results": 3,
"first": "Deepfake - Wikipedia"
},
"checked_at": "2026-06-27T14:46:37.623279+03:00"
}
]
}

View file

@ -0,0 +1,271 @@
#!/usr/bin/env python3
"""DiDi LOT 1 — runner automat de teste end-to-end.
Rulează un test funcțional pe fiecare componentă și scrie rezultatele într-un
raport JSON cu timestamp (reports/test_report_<data>.json). Fiecare intrare
conține: componentă, endpoint, http_code, ok (verdict funcțional), latență,
extras din răspuns + request-ul folosit ca dovadă vizibilă.
Rulare: python3 run_tests.py (HOST_IP din .env sau 10.11.10.18)
HOST_IP=1.2.3.4 python3 run_tests.py
"""
from __future__ import annotations
import json, os, subprocess, sys, time, datetime, pathlib
HERE = pathlib.Path(__file__).resolve().parent
ASSETS = HERE / "test_assets"
REPORTS = HERE / "reports"
# --- config din .env (fallback la defaults) ---------------------------------
def load_env() -> dict:
env = {}
f = HERE / ".env"
if f.exists():
for line in f.read_text().splitlines():
line = line.strip()
if line and not line.startswith("#") and "=" in line:
k, v = line.split("=", 1)
env[k.strip()] = v.strip()
return env
ENV = load_env()
H = os.environ.get("HOST_IP") or ENV.get("HOST_IP") or "10.11.10.18"
TOKEN = ENV.get("GATEWAY_API_TOKEN", "didi-local-dev-token-123")
WHISPER = ENV.get("WHISPER_MODEL", "large-v3-turbo")
# --- docker shim (pt pregatirea asset-urilor; merge si prin sg docker) -------
def docker(args: list[str]) -> subprocess.CompletedProcess:
if subprocess.run(["docker", "ps"], capture_output=True).returncode == 0:
return subprocess.run(["docker", *args], capture_output=True, text=True)
return subprocess.run(["sg", "docker", "-c", "docker " + " ".join(args)],
capture_output=True, text=True)
def ensure_assets() -> None:
ASSETS.mkdir(exist_ok=True)
mp4 = ASSETS / "sample.mp4"
if not mp4.exists():
import glob
cands = sorted(glob.glob(str(HERE.parent.parent / "artefacte_lot1/runs_deepfake/**/*.mp4"), recursive=True),
key=lambda p: os.path.getsize(p))
if cands:
subprocess.run(["cp", cands[0], str(mp4)])
if not (ASSETS / "frame.jpg").exists() or not (ASSETS / "audio.wav").exists():
# genereaza un cadru + audio din containerul extractors (are ffmpeg)
docker(["cp", str(mp4), "didiAI-extractors:/tmp/s.mp4"])
docker(["exec", "didiAI-extractors", "sh", "-c",
"ffmpeg -y -i /tmp/s.mp4 -frames:v 1 /tmp/frame.jpg -vn -ar 16000 -ac 1 -t 8 /tmp/audio.wav"])
docker(["cp", "didiAI-extractors:/tmp/frame.jpg", str(ASSETS / "frame.jpg")])
docker(["cp", "didiAI-extractors:/tmp/audio.wav", str(ASSETS / "audio.wav")])
# --- helper request prin curl -----------------------------------------------
def run_curl(args: list[str], timeout: int) -> tuple[int, str, float]:
"""Return (http_code, body, latency_ms)."""
full = ["curl", "-s", "-m", str(timeout), "-w", "\n%{http_code}", *args]
t0 = time.monotonic()
try:
p = subprocess.run(full, capture_output=True, text=True, timeout=timeout + 10)
out = p.stdout
except subprocess.TimeoutExpired:
return (0, "TIMEOUT", round((time.monotonic() - t0) * 1000, 1))
lat = round((time.monotonic() - t0) * 1000, 1)
code = 0; body = out
if "\n" in out:
body, _, last = out.rpartition("\n")
try: code = int(last.strip())
except ValueError: code = 0
return (code, body, lat)
def jparse(body: str):
try: return json.loads(body)
except Exception: return None
# --- definitia testelor ------------------------------------------------------
def t_llm():
c, b, l = run_curl([f"http://{H}:14011/v1/chat/completions", "-H", "Content-Type: application/json",
"-d", json.dumps({"model":"qwen3.5","messages":[{"role":"user","content":"Ce este un deepfake? O propozitie."}],"max_tokens":120})], 90)
d = jparse(b); txt = (d or {}).get("choices",[{}])[0].get("message",{}).get("content","") if d else ""
return c, l, bool(txt) and not txt.lower().startswith("thinking"), {"reply": txt[:300]}
def t_embeddings():
c,b,l = run_curl([f"http://{H}:14100/v1/embeddings","-H","Content-Type: application/json","-d",json.dumps({"model":"bge-m3","input":"test"})],30)
d=jparse(b); dim=len((d or {}).get("data",[{}])[0].get("embedding",[])) if d else 0
return c,l,dim==1024,{"dim":dim}
def t_rerank():
c,b,l=run_curl([f"http://{H}:14200/v1/rerank","-H","Content-Type: application/json","-d",json.dumps({"model":"bge-reranker-v2-m3","query":"deepfake detection","documents":["o pisica","sistem detectie deepfake video"]})],30)
d=jparse(b); r=(d or {}).get("results",[]) ; top=r[0] if r else {}
return c,l,top.get("index")==1,{"top_index":top.get("index"),"score":round(top.get("relevance_score",0),4)}
def t_audio():
c,b,l=run_curl([f"http://{H}:54300/v1/audio/transcriptions","-F",f"file=@{ASSETS}/audio.wav","-F",f"model={WHISPER}"],120)
d=jparse(b); txt=(d or {}).get("text","") if d else ""
return c,l,bool(txt.strip()),{"text":txt[:200],"language":(d or {}).get("language")}
def t_video():
c,b,l=run_curl([f"http://{H}:54600/analyze/video","-F",f"file=@{ASSETS}/sample.mp4"],180)
d=jparse(b); v=(d or {}).get("verdict")
return c,l,v in ("REAL","FAKE","UNCERTAIN"),{"verdict":v,"frames_analyzed":(d or {}).get("frames_analyzed")}
def t_metadata():
c,b,l=run_curl([f"http://{H}:54400/v1/metadata","-F",f"file=@{ASSETS}/sample.mp4"],40)
d=jparse(b); ok=bool((d or {}).get("analyses",{}).get("integrity",{}).get("ok"))
return c,l,ok,{"media_type":(d or {}).get("media_type"),"sha256":(d or {}).get("sha256","")[:16]}
def t_ner():
c,b,l=run_curl([f"http://{H}:54400/v1/ner","-H","Content-Type: application/json","-d",json.dumps({"text":"Klaus Iohannis s-a intalnit cu Emmanuel Macron la Bucuresti."})],60)
d=jparse(b); ents=(d or {}).get("results",{}).get("entities",[])
return c,l,len(ents)>=2,{"entities":[e.get("text")+":"+e.get("label","") for e in ents][:5]}
def t_detect():
c,b,l=run_curl([f"http://{H}:54400/v1/detect","-F",f"file=@{ASSETS}/frame.jpg"],90)
d=jparse(b); det=(d or {}).get("results",{}).get("detections",[])
return c,l,len(det)>=1,{"objects":len(det),"labels":[x.get("label") for x in det][:5]}
def t_ocr():
c,b,l=run_curl([f"http://{H}:54400/v1/ocr","-F",f"file=@{ASSETS}/frame.jpg"],90)
d=jparse(b); ok=bool((d or {}).get("ok")); txt=(d or {}).get("results",{}).get("text","")
return c,l,ok,{"text":txt[:150]}
def t_sentiment():
c,b,l=run_curl([f"http://{H}:54400/v1/sentiment","-H","Content-Type: application/json","-d",json.dumps({"text":"Produsul este excelent, sunt foarte multumit!"})],60)
d=jparse(b); ok=bool((d or {}).get("ok"))
return c,l,ok,{"label":(d or {}).get("results",{}).get("label"),"score":(d or {}).get("results",{}).get("score")}
def t_forensic():
c,b,l=run_curl([f"http://{H}:8085/api/forensic-modules"],15)
d=jparse(b); m=(d or {}).get("available_modules",[])
return c,l,len(m)>=3,{"modules":[x.get("id") for x in m]}
def t_web():
c,b,l=run_curl([f"http://{H}:51100/v1/search","-H","Content-Type: application/json","-d",json.dumps({"queries":["deepfake detection"],"max_results":3})],40)
d=jparse(b); r=(d or {}).get("results",[])
return c,l,len(r)>0,{"results":len(r),"first":(r[0].get("title","")[:60] if r else None)}
def t_catalog():
c,b,l=run_curl([f"http://{H}:11000/catalog/v1/status","-H",f"Authorization: Bearer {TOKEN}"],15)
d=jparse(b); st=(d or {}).get("status")
return c,l,st in ("healthy","degraded"),{"status":st,"components":[x.get("component_id") for x in (d or {}).get("components",[])]}
def t_gateway():
c,b,l=run_curl([f"http://{H}:11000/health"],10)
c2,_,_=run_curl([f"http://{H}:11000/web/v1/info"],10) # fara token -> 401
return c,l,(c==200 and c2==401),{"health":c,"web_no_auth":c2}
def t_brain_atoms():
c,b,l=run_curl([f"http://{H}:8090/v1/analysis_atom/stats/extended"],15)
d=jparse(b)
return c,l,(d is not None and "total_atoms" in d),{"total_atoms":(d or {}).get("total_atoms"),"by_tier":(d or {}).get("by_tier")}
def t_brain_facts():
c,b,l=run_curl([f"http://{H}:8090/v1/fact_status/list?page=1&page_size=10"],15)
d=jparse(b)
return c,l,(d is not None and "total" in d),{"total":(d or {}).get("total")}
def t_brain_vcache():
c,b,l=run_curl([f"http://{H}:8090/v1/verification_cache/list?page=1&page_size=10"],15)
d=jparse(b)
return c,l,(d is not None and "total" in d),{"total":(d or {}).get("total")}
def t_llm_completions():
c,b,l=run_curl([f"http://{H}:14011/v1/completions","-H","Content-Type: application/json","-d",json.dumps({"model":"qwen3.5","prompt":"Capitala Romaniei este","max_tokens":10})],60)
d=jparse(b); txt=(d or {}).get("choices",[{}])[0].get("text","") if d else ""
return c,l,bool(txt.strip()),{"text":txt[:120]}
def t_llm_info():
c,b,l=run_curl([f"http://{H}:14011/v1/info"],15); d=jparse(b)
return c,l,isinstance(d,dict) and len(d)>0,{"keys":list(d.keys())[:6] if isinstance(d,dict) else None}
def t_video_semantic():
c,b,l=run_curl([f"http://{H}:54600/analyze/video/semantic","-F",f"file=@{ASSETS}/sample.mp4"],180)
d=jparse(b); ok=bool(d) and ("final_summary" in d or "chunk_results" in d)
return c,l,ok,{"num_chunks":(d or {}).get("num_chunks"),"summary":(str((d or {}).get("final_summary",""))[:120])}
def t_web_fetch():
c,b,l=run_curl([f"http://{H}:51100/v1/fetch","-H","Content-Type: application/json","-d",json.dumps({"urls":["https://en.wikipedia.org/wiki/Deepfake"]})],60)
ok=(c==200 and len(b)>1500 and ("title" in b or "content" in b or "text" in b))
return c,l,ok,{"bytes":len(b)}
def t_web_gather():
c,b,l=run_curl([f"http://{H}:51100/v1/gather","-H","Content-Type: application/json","-d",json.dumps({"claim":"Deepfakes can be detected by AI","max_search_results":5})],150)
d=jparse(b); ev=(d or {}).get("evidence",[]); n=(d or {}).get("total_evidence_items",len(ev) if isinstance(ev,list) else 0)
return c,l,bool(d) and (n>0 or (isinstance(ev,list) and len(ev)>0)),{"evidence_items":n,"urls_found":(d or {}).get("total_urls_found")}
def t_forensic_evidence():
c,b,l=run_curl([f"http://{H}:8085/api/forensic-evidence","-F",f"video=@{ASSETS}/sample.mp4","-F","modules=m29","-F","every_n_frames=30"],150)
d=jparse(b); mr=(d or {}).get("modules_run",[])
return c,l,bool(mr),{"modules_run":mr,"frames":(d or {}).get("n_frames_extracted")}
def t_catalog_components():
c,b,l=run_curl([f"http://{H}:11000/catalog/v1/components","-H",f"Authorization: Bearer {TOKEN}"],15)
d=jparse(b); comp=(d or {}).get("components",d if isinstance(d,list) else [])
return c,l,c==200 and len(comp)>0,{"components":len(comp)}
def t_catalog_models():
c,b,l=run_curl([f"http://{H}:11000/catalog/v1/models","-H",f"Authorization: Bearer {TOKEN}"],15)
d=jparse(b)
return c,l,c==200 and d is not None,{"models":len((d or {}).get("models",d if isinstance(d,list) else []))}
def t_brain_search():
c,b,l=run_curl([f"http://{H}:8090/v1/search","-H","Content-Type: application/json","-d",json.dumps({"queries":["deepfake"],"max_results":2})],60)
d=jparse(b)
return c,l,bool(d) and "results" in d,{"total_results":(d or {}).get("total_results"),"cache":(d or {}).get("brain_meta",{}).get("cache_status")}
def t_brain_gather():
c,b,l=run_curl([f"http://{H}:8090/v1/gather","-H","Content-Type: application/json","-d",json.dumps({"claim":"Deepfakes can be detected by AI"})],120)
d=jparse(b)
return c,l,bool(d) and "evidence" in d,{"evidence_items":(d or {}).get("total_evidence_items"),"urls":(d or {}).get("total_urls_found")}
def t_dashboard_health():
c,b,l=run_curl([f"http://{H}:51300/health"],10); d=jparse(b)
return c,l,(d or {}).get("status") in ("healthy","ok"),{"status":(d or {}).get("status"),"db":(d or {}).get("db")}
def t_dashboard_monitoring():
c,b,l=run_curl([f"http://{H}:51300/api/monitoring/services"],20); d=jparse(b)
s=d if isinstance(d,list) else (d or {}).get("services",[])
h=sum(1 for x in s if x.get("status")=="healthy")
return c,l,len(s)>0 and h==len(s),{"healthy":f"{h}/{len(s)}"}
def t_cloak():
c,b,l=run_curl([f"http://{H}:8770/v1/search","-H","Content-Type: application/json","-d",json.dumps({"queries":["deepfake"],"engines":["ddg"],"max_results_per_engine":3})],45)
d=jparse(b); r=(d or {}).get("results",[])
return c,l,len(r)>0,{"results":len(r),"first":(r[0].get("title","")[:55] if r else None)}
TESTS = [
("llm","LLM (Qwen3.5-35B-A3B)","POST /v1/chat/completions",t_llm),
("llm_completions","LLM · completions","POST /v1/completions",t_llm_completions),
("llm_info","LLM · info","GET /v1/info",t_llm_info),
("embeddings","Embeddings (bge-m3)","POST /v1/embeddings",t_embeddings),
("rerank","Rerank (bge-reranker-v2-m3)","POST /v1/rerank",t_rerank),
("audio","Audio (Whisper large-v3-turbo)","POST /v1/audio/transcriptions",t_audio),
("video","Video/BusterX (deepfake)","POST /analyze/video",t_video),
("video_semantic","Video · semantic","POST /analyze/video/semantic",t_video_semantic),
("extractors_metadata","Extractors · metadata","POST /v1/metadata",t_metadata),
("extractors_ner","Extractors · NER (GLiNER)","POST /v1/ner",t_ner),
("extractors_detect","Extractors · detect (YOLOv8)","POST /v1/detect",t_detect),
("extractors_ocr","Extractors · OCR (LLM vision)","POST /v1/ocr",t_ocr),
("extractors_sentiment","Extractors · sentiment (LLM)","POST /v1/sentiment",t_sentiment),
("forensic","Forensic · module list","GET /api/forensic-modules",t_forensic),
("forensic_evidence","Forensic · analiza reala","POST /api/forensic-evidence",t_forensic_evidence),
("web","Web · search (SearXNG)","POST /v1/search",t_web),
("web_fetch","Web · fetch","POST /v1/fetch",t_web_fetch),
("web_gather","Web · gather (evidence)","POST /v1/gather",t_web_gather),
("catalog","Catalog · status","GET /catalog/v1/status",t_catalog),
("catalog_components","Catalog · components","GET /catalog/v1/components",t_catalog_components),
("catalog_models","Catalog · models","GET /catalog/v1/models",t_catalog_models),
("gateway","Gateway (auth+rutare)","GET /health + 401",t_gateway),
("brain_atoms","Brain · analysis_atom","GET /v1/analysis_atom/stats/extended",t_brain_atoms),
("brain_facts","Brain · fact_status","GET /v1/fact_status/list",t_brain_facts),
("brain_vcache","Brain · verification_cache","GET /v1/verification_cache/list",t_brain_vcache),
("brain_search","Brain · search (RAG)","POST /v1/search",t_brain_search),
("brain_gather","Brain · gather (RAG)","POST /v1/gather",t_brain_gather),
("dashboard","Dashboard · health","GET /health",t_dashboard_health),
("dashboard_monitoring","Dashboard · monitoring 11/11","GET /api/monitoring/services",t_dashboard_monitoring),
("cloak","Cloak · SERP scraper","POST /v1/search",t_cloak),
]
def main() -> int:
ensure_assets()
ts = datetime.datetime.now().astimezone()
stamp = ts.strftime("%Y%m%d_%H%M%S")
results = []
print(f"\n DiDi LOT 1 — test run @ {ts.isoformat()} (host {H})\n")
for tid, name, ep, fn in TESTS:
try:
code, lat, ok, extra = fn()
except Exception as e: # noqa: BLE001
code, lat, ok, extra = 0, 0.0, False, {"error": str(e)}
mark = "\033[32mPASS\033[0m" if ok else "\033[31mFAIL\033[0m"
print(f" [{mark}] {name:34} {ep:38} HTTP {code:>3} {lat:>7.0f}ms {json.dumps(extra, ensure_ascii=False)[:80]}")
results.append({"id":tid,"name":name,"endpoint":ep,"http_code":code,
"ok":ok,"latency_ms":lat,"result":extra,"checked_at":ts.isoformat()})
passed = sum(1 for r in results if r["ok"])
report = {
"generated_at": ts.isoformat(),
"host": H,
"summary": {"total": len(results), "passed": passed, "failed": len(results) - passed},
"tests": results,
}
REPORTS.mkdir(exist_ok=True)
out = REPORTS / f"test_report_{stamp}.json"
out.write_text(json.dumps(report, indent=2, ensure_ascii=False))
latest = REPORTS / "latest.json"
latest.write_text(json.dumps(report, indent=2, ensure_ascii=False))
print(f"\n === {passed}/{len(results)} PASS ===")
print(f" Raport JSON: {out}")
print(f" (si copie: {latest})\n")
return 0 if passed == len(results) else 1
if __name__ == "__main__":
sys.exit(main())

View file

@ -0,0 +1,49 @@
#!/usr/bin/env bash
# Brain stack helpers: render env, align postgres password, create atomic token,
# seed the canonical taxonomy. Idempotent. Requires lib.sh sourced + master .env
# loaded ($POSTGRES_PASSWORD, $ATOMIC_TOKEN, $HOST_IP, $ENV_FILE, $BRAIN_DIR).
# Write didi_brain/.env (read by brain-api/scheduler) + infra/.env (compose interpolation)
brain_write_env() {
local pw=$1 tok=$2
cp "${BRAIN_DIR}/.env.example" "${BRAIN_DIR}/.env"
sed -i \
-e "s#^LLM_ROUTER_URL=.*#LLM_ROUTER_URL=http://didiAI-llm-api:14011#" \
-e "s#^LLM_VLLM_URL=.*#LLM_VLLM_URL=http://didiAI-vllm-qwen3.5:14001#" \
-e "s#^EMBEDDING_URL=.*#EMBEDDING_URL=http://didiAI-embeddings-api:14100#" \
-e "s#^RERANKER_URL=.*#RERANKER_URL=http://didiAI-rerank-api:14200#" \
-e "s#^MODEL_FAST_ENABLED=.*#MODEL_FAST_ENABLED=true#" \
-e "s#^POSTGRES_PASSWORD=.*#POSTGRES_PASSWORD=${pw}#" \
"${BRAIN_DIR}/.env"
if grep -q '^ATOMIC_TOKEN=' "${BRAIN_DIR}/.env"; then
sed -i "s#^ATOMIC_TOKEN=.*#ATOMIC_TOKEN=${tok}#" "${BRAIN_DIR}/.env"
else
echo "ATOMIC_TOKEN=${tok}" >> "${BRAIN_DIR}/.env"
fi
printf 'POSTGRES_USER=atomic\nPOSTGRES_PASSWORD=%s\nPOSTGRES_DB=atomic\n' "$pw" > "${BRAIN_DIR}/infra/.env"
}
# Force the postgres role password to match (volume may have been initialized with a different one)
brain_align_pg() {
local pw=$1
docker exec didibrain-postgres psql -U atomic -d atomic \
-c "ALTER USER atomic WITH PASSWORD '${pw}';" >/dev/null 2>&1 || true
}
# Create an atomic API token if the master .env doesn't have one yet. Echos the token.
brain_ensure_token() {
local pw=$1 cur=$2
if [ -n "$cur" ]; then printf '%s' "$cur"; return 0; fi
local out tok
out=$(docker exec didibrain-atomic atomic-server token create --name didibrain \
--storage postgres --database-url "postgres://atomic:${pw}@postgres:5432/atomic" 2>&1)
tok=$(printf '%s' "$out" | grep -oE 'at_[A-Za-z0-9_.-]{20,}' | head -1)
printf '%s' "$tok"
}
# Seed canonical taxonomy into atomic (idempotent; tags created if missing).
brain_seed_taxonomy() {
docker cp "${BRAIN_DIR}/scripts" didibrain-api:/app/scripts >/dev/null 2>&1 || true
# local _tag_ids.json write is read-only in the image — harmless, atomic gets the tags
docker exec -e PYTHONPATH=/app didibrain-api python /app/scripts/04_seed_taxonomy.py >/dev/null 2>&1 || true
}

View file

@ -0,0 +1,36 @@
#!/usr/bin/env bash
# Idempotent seed of the dashboard DB-backed Model & Extractor catalog.
# Defines seed_catalog(); requires lib.sh sourced + $HOST_IP set.
seed_catalog() {
local base="http://${HOST_IP}:51300/api/catalog"
# existing names (idempotency)
local existing
existing=$(curl -s -m10 "$base" | python3 -c 'import sys,json
try:
d=json.load(sys.stdin); items=d if isinstance(d,list) else d.get("items",d.get("data",[]))
print("\n".join(x.get("name","") for x in items))
except Exception: pass' 2>/dev/null)
local entries=(
'{"kind":"model","name":"qwen3.5","display_name":"Qwen3.5-35B-A3B","service":"llm","capabilities":["chat","completion","reasoning"],"context_length":32000,"supports_gpu":true,"endpoint":"http://HOST:14011","notes":"vLLM MoE, GPU 0"}'
'{"kind":"model","name":"bge-m3","display_name":"BGE-M3 Embeddings","service":"embeddings","capabilities":["embeddings"],"context_length":8192,"supports_gpu":true,"endpoint":"http://HOST:14100","notes":"1024-dim, vLLM GPU 1"}'
'{"kind":"model","name":"bge-reranker-v2-m3","display_name":"BGE Reranker v2-m3","service":"rerank","capabilities":["rerank"],"context_length":8192,"supports_gpu":true,"endpoint":"http://HOST:14200","notes":"cross-encoder, vLLM GPU 1"}'
'{"kind":"model","name":"busterx","display_name":"BusterX++ (deepfake)","service":"video","capabilities":["deepfake-detection","video-vlm"],"context_length":32768,"supports_gpu":true,"endpoint":"http://HOST:54600","notes":"Qwen2.5-VL based, vLLM GPU 1"}'
'{"kind":"model","name":"whisper-large-v3-turbo","display_name":"Whisper large-v3-turbo","service":"audio","capabilities":["transcription","stt"],"supports_gpu":true,"endpoint":"http://HOST:54300","notes":"faster-whisper, GPU 0"}'
'{"kind":"extractor","name":"gliner-multi-v2.1","display_name":"GLiNER NER (multilingual)","service":"extractors","capabilities":["ner"],"supports_gpu":true,"endpoint":"http://HOST:54400/v1/ner","notes":"validat RO"}'
'{"kind":"extractor","name":"yolov8n","display_name":"YOLOv8 Object Detection","service":"extractors","capabilities":["object-detection"],"supports_gpu":true,"endpoint":"http://HOST:54400/v1/detect"}'
'{"kind":"extractor","name":"metadata","display_name":"Media Metadata (EXIF/ELA/ffprobe/C2PA)","service":"extractors","capabilities":["exif","ela","spectrogram","integrity"],"supports_cpu":true,"endpoint":"http://HOST:54400/v1/metadata"}'
'{"kind":"extractor","name":"forensic-suite","display_name":"Forensic detectors (rPPG/lip-sync/forgery/lighting)","service":"forensic","capabilities":["rppg","lip-sync","forgery-heatmap","lighting"],"supports_gpu":true,"endpoint":"http://HOST:8085/api/forensic-evidence"}'
)
local added=0 skipped=0 e name
for e in "${entries[@]}"; do
name=$(printf '%s' "$e" | python3 -c 'import sys,json;print(json.load(sys.stdin)["name"])')
if printf '%s\n' "$existing" | grep -qx "$name"; then skipped=$((skipped+1)); continue; fi
e=${e//HOST/$HOST_IP}
if [ "$(curl -s -m10 -o /dev/null -w '%{http_code}' -X POST "$base" -H 'Content-Type: application/json' -d "$e")" = "200" ]; then
added=$((added+1))
fi
done
echo "catalog: +$added adaugate, $skipped existau deja"
}