clossers/didi-lot1-ai
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
didi-lot1-ai/ai_platform/ML_PROJECTS_DEPLOY_KEYS_AND_REQUIREMENTS.md

13 KiB
Raw Blame History

ML Projects — Deploy Keys, Env & Requirements

Generated: 2026-04-30 Source mașină: /home/admin365/ml-projects/ Arhivă cod: /home/admin365/ml-projects-20260424-1013.tar.gz (264 MB, conține și .env-urile)

⚠️ Document conține secrete în clear text. Tratează-l ca pe un seif — NU pe email/Slack public, NU în repo public, NU pe wiki. Pentru transfer: scp criptat sau message DM.

Notă topologie: acest document descrie topologia multi-host de referință (URL-uri/IP-uri distribuite pe mai multe mașini). Valorile reale pentru deployment-ul single-host livrat se află în local_gpu_stack/.env.example.

Cuprins

  1. Cerințe sistem (host)
  2. Servicii externe accesabile (VPN/LAN)
  3. API keys terți
  4. Credentiale interne (DB, tokens generate)
  5. .env files complet, per modul
  6. Porturi expuse host-side
  7. Ordine deploy + comenzi

Cerinte sistem (host)

Minim, pe orice mașină pe care vrei să rulezi stack-ul didiAI:

Tool Versiune Purpose
Linux Ubuntu 22.04+ Host OS
Docker Engine 24.0+ Container runtime
Docker Compose v2 (plugin) Orchestration
Python 3.10+ Pentru scripturile bootstrap (didi_brain)
python3-venv match Python ver Pentru bootstrap_deploy.sh
curl orice Health checks
openssl orice Generare passwords

Opțional:

  • NVIDIA Driver 535+ + NVIDIA Container Toolkit — doar pentru mașinile cu GPU (vLLM, Whisper). Dacă nu ai GPU, mașina asta rulează doar componentele CPU (web-api, dashboard, didi-brain, video-api as proxy).

Spațiu disk:

  • ~2 GB pentru imagini Docker
  • ~1 GB pentru postgres data + atomic atoms (crește cu 100 KB/claim ingerat)

Servicii externe accesabile (VPN/LAN)

Stack-ul depinde de aceste hosts care rulează GPU-side:

Service URL Folosit de
LLM Router (gateway Qwen) http://10.11.10.17:14011 web, didi_brain, dashboard
vLLM Qwen3.5-35B (direct) http://10.11.10.17:14001 dashboard (provider stats)
llama.cpp Qwen3.5-35B-A3B #1 http://10.11.10.18:14001 LLM router upstream
llama.cpp Qwen3.5-35B-A3B #2 http://10.11.10.19:14001 LLM router upstream
BGE-M3 embeddings http://10.11.10.15:8200 didi_brain (Atomic auto-embeds)
BGE-reranker-v2-m3 http://10.11.10.15:8100 didi_brain (gather rerank)
Whisper STT (host-local, audio module) audio module

Ping test rapid înainte de deploy:

for url in http://10.11.10.17:14011/health \
          http://10.11.10.15:8100/health \
          http://10.11.10.15:8200/v1/models; do
  echo -n "$url ... "
  curl -sf --connect-timeout 5 "$url" > /dev/null && echo OK || echo FAIL
done

API keys terti

Search providers (premium tier web-api + dashboard live quota):

Provider Key Folosit pentru
SerpAPI 8a84f4de853b5ec0f7eca97b30f27db45c576cfde24e979ad219ff8b9ef61ad6 Google search (premium rotation)
Tavily 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 AI-optimized search
LinkUp 60f33e97-6cb0-4dd0-8364-b120f43efac6 Web content search
Brave Search BSAYcuZE8365Ms-BEj3yPmDjgdx_UAL Premium search alternative

LLM providers:

Provider Key Folosit pentru
OpenRouter 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 Premium tier LLM (Gemini Flash)

HuggingFace (opțional, pentru download modele gated):

  • HF_TOKEN= (gol în .env actual; setați doar dacă descărcați modele gated)

Credentiale interne (DB, tokens generate)

Generate per-deploy — diferă între mașini, NU le copia direct:

Var Valoare actuală Notă
DASHBOARD_DB_PASSWORD /8FQ36GKvCnaNNHexJLhaVH0GiPsjWHd Postgres dashboard. Generează nou pe altă mașină: openssl rand -base64 24
GATEWAY_API_TOKEN 0x2-m0W5oG7MjZUu5v3-ejBWMExwLeGDv4aRKqBI7bM Bearer token pentru gateway nginx
ATOMIC_TOKEN at_bsbLAS1tO0wQ0tulyAdJvhzZlNdBqDEyayf_GBzNTUk Generat automat de scripts/02_bootstrap_atomic.py la primul deploy. Nu copia — rerularea bootstrap-ului îl regenerează.
POSTGRES_PASSWORD (didi_brain) atomic_dev_changeme Default insecure. Schimbă înainte de prod.

.env files complet, per modul

modules/web/deploy/.env

# REQUIRED - Search (SearXNG) - Docker internal hostname
WEB_SEARXNG_BASE_URL=http://didiAI-web-searxng:8080
WEB_EXTERNAL_URL=http://localhost:51100

# REQUIRED - LLM Integration
WEB_LLM_BASE_URL=http://10.11.10.17:14011
WEB_TEXT_MODEL=qwen3.5

# Vision model - same Qwen3.5 (native multimodal)
WEB_VISION_BASE_URL=http://10.11.10.17:14011
WEB_VISION_MODEL=qwen3.5

# OPTIONAL - Premium search providers
WEB_SERPAPI_API_KEY=8a84f4de853b5ec0f7eca97b30f27db45c576cfde24e979ad219ff8b9ef61ad6
WEB_TAVILY_API_KEY=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
WEB_LINKUP_API_KEY=60f33e97-6cb0-4dd0-8364-b120f43efac6
WEB_BRAVE_API_KEY=BSAYcuZE8365Ms-BEj3yPmDjgdx_UAL

# OPTIONAL - OpenRouter (premium tier LLM)
WEB_OPENROUTER_API_KEY=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
WEB_OPENROUTER_MODEL=google/gemini-3.1-flash-lite-preview

# OPTIONAL - Dashboard event sink
WEB_DASHBOARD_URL=http://didiAI-dashboard:51300

# OPTIONAL - Brain cache integration (premium tier only)
WEB_BRAIN_URL=http://didibrain-api:8090
WEB_BRAIN_CACHE_READ_ENABLED=true
WEB_BRAIN_INGEST_ENABLED=true
WEB_BRAIN_INGEST_MIN_EVIDENCE=3
WEB_BRAIN_INGEST_MIN_CREDIBILITY=0.7
WEB_BRAIN_INGEST_MIN_EXECUTION_MS=2000
WEB_BRAIN_HIT_MIN_EVIDENCE=3
WEB_BRAIN_HIT_MIN_RELEVANCE=0.7

modules/dashboard/deploy/.env

# Database
DASHBOARD_DB_USER=dashboard
DASHBOARD_DB_PASSWORD=/8FQ36GKvCnaNNHexJLhaVH0GiPsjWHd   # GENEREAZĂ NOU pe altă mașină
DASHBOARD_DB_NAME=dashboard

# Server
DASHBOARD_EXTERNAL_URL=http://localhost:51300

# Provider API keys (mirror cu web)
DASHBOARD_SERPAPI_API_KEY=8a84f4de853b5ec0f7eca97b30f27db45c576cfde24e979ad219ff8b9ef61ad6
DASHBOARD_TAVILY_API_KEY=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
DASHBOARD_LINKUP_API_KEY=60f33e97-6cb0-4dd0-8364-b120f43efac6
DASHBOARD_BRAVE_API_KEY=BSAYcuZE8365Ms-BEj3yPmDjgdx_UAL
DASHBOARD_OPENROUTER_API_KEY=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

# Upstream URLs
DASHBOARD_WEB_API_URL=http://didiAI-web-api:51100
DASHBOARD_SEARXNG_URL=http://didiAI-web-searxng:8080
DASHBOARD_LLM_API_URL=http://10.11.10.17:14011
DASHBOARD_VLLM_QWEN_URL=http://10.11.10.17:14001
DASHBOARD_LLAMACPP_URLS=http://10.11.10.18:14001,http://10.11.10.19:14001

# Retention
DASHBOARD_HISTORY_RETENTION_DAYS=30
DASHBOARD_PROVIDER_STATS_CACHE_SECONDS=30

modules/didi_brain/.env

# LLM router (single entry point)
LLM_ROUTER_URL=http://10.11.10.17:14011
LLM_ROUTER_API_KEY=
LLM_VLLM_URL=http://localhost:14001
LLM_LLAMACPP_URLS=http://10.11.10.18:14001,http://10.11.10.19:14001

# Models
MODEL_REASONING=Qwen/Qwen3.5-35B-A3B
MODEL_REASONING_BACKEND=llamacpp
MODEL_FAST=qwen3.5
MODEL_FAST_BACKEND=vllm
MODEL_FAST_ENABLED=false
# Vision: nu exista model vision separat — OCR/vision se face prin Qwen3.5 (gateway LLM)
MODEL_VISION=qwen3.5
MODEL_VISION_URL=http://10.11.10.17:14011
MODEL_VISION_ENABLED=false

# Embeddings
EMBEDDING_URL=http://10.11.10.15:8200
EMBEDDING_API_KEY=
EMBEDDING_MODEL=BAAI/bge-m3
EMBEDDING_DIM=1024
EMBEDDING_MAX_TOKENS=8192

# Reranker
RERANKER_URL=http://10.11.10.15:8100
RERANKER_API_KEY=
RERANKER_MODEL=BAAI/bge-reranker-v2-m3

# Atomic server (token GENERAT de bootstrap; nu copia, rerulează scripts/02)
ATOMIC_URL=http://localhost:8088
ATOMIC_TOKEN=at_bsbLAS1tO0wQ0tulyAdJvhzZlNdBqDEyayf_GBzNTUk

# Postgres (schimbă password pe deploy nou)
POSTGRES_USER=atomic
POSTGRES_PASSWORD=atomic_dev_changeme
POSTGRES_DB=atomic
POSTGRES_PORT=5434

LOG_LEVEL=INFO

modules/llm-inference/deploy/.env

LLM_DEFAULT_BACKEND=vllm
LLM_ENABLE_VLLM=true
LLM_ENABLE_LLAMACPP=true
LLM_EXTERNAL_URL=http://localhost:14011
LLM_LLAMACPP_BASE_URLS=http://10.11.10.18:14001,http://10.11.10.19:14001

HF_CACHE_DIR=/home/didiai/.cache/huggingface
VLLM_MODEL=Qwen/Qwen3.5-35B-A3B
MODELS_DIR=/home/didiai/.cache/huggingface
HF_TOKEN=

modules/audio/deploy/.env

AUDIO_MODEL=large-v3-turbo
AUDIO_DEVICE=cuda
AUDIO_COMPUTE_TYPE=int8
AUDIO_CACHE_DIR=/home/didiai/.cache/huggingface
AUDIO_EXTERNAL_URL=http://localhost:54300

modules/catalog-api/deploy/.env

CATALOG_EXTERNAL_URL=http://localhost
CATALOG_PORT=11000
CATALOG_LLM_URL=http://didiAI-llm-api:14011
CATALOG_AUDIO_URL=http://didiAI-audio:54300
CATALOG_WEB_URL=http://didiAI-web-api:51100
CATALOG_VIDEO_URL=

modules/gateway/deploy/.env

GATEWAY_API_TOKEN=0x2-m0W5oG7MjZUu5v3-ejBWMExwLeGDv4aRKqBI7bM

Porturi expuse host-side

Port Serviciu Container
51100 Web API (gather, search, fetch) didiAI-web-api
51300 Dashboard UI + API didiAI-dashboard
15432 Dashboard Postgres (debug only) didiAI-dashboard-db
54300 Audio STT didiAI-audio
54600 Video Analysis didiAI-video-api
8080 (intern) SearXNG metasearch didiAI-web-searxng
8090 didi-brain HTTP API didibrain-api
8088 didi-brain Atomic didibrain-atomic
5434 didi-brain Postgres (debug only) didibrain-postgres

Ordine deploy + comenzi

1. Pune codul pe mașină

scp /home/admin365/ml-projects-20260424-1013.tar.gz user@new-host:~/
ssh user@new-host
tar xzf ml-projects-20260424-1013.tar.gz
cd ml-projects

2. Verifică conectivitatea VPN/LAN

curl -sf --connect-timeout 5 http://10.11.10.17:14011/health
curl -sf --connect-timeout 5 http://10.11.10.15:8100/health

Dacă FAIL: stack-ul nu va funcționa — verifică VPN.

3. Verifică / ajustează .env-urile

Pentru CPU-only host (fără GPU):

  • Editează modules/web/deploy/.env → WEB_LLM_BASE_URL la IP-ul corect (default 10.11.10.17:14011)
  • Editează modules/didi_brain/.env → LLM_ROUTER_URL la fel
  • Generează DB password nou: 
    NEW_PASS=$(openssl rand -base64 24)
sed -i "s|DASHBOARD_DB_PASSWORD=.*|DASHBOARD_DB_PASSWORD=${NEW_PASS}|" modules/dashboard/deploy/.env

4. Deploy SearXNG (dependență web)

cd modules/web/deploy/metasearch
docker compose up -d
cd -

5. Deploy dashboard (postgres + API)

cd modules/dashboard/deploy
docker compose --profile dashboard up -d
cd -

6. Deploy web-api

cd modules/web/deploy
docker compose --profile api up -d
cd -

7. Deploy didi-brain (cu bootstrap interactiv)

cd modules/didi_brain
./scripts/bootstrap_deploy.sh

Bootstrap-ul:

  • Build imagine didibrain-api
  • Up postgres + atomic-server + brain-api
  • Generează ATOMIC_TOKEN nou și-l scrie în .env
  • Seed taxonomy (79 tags)
  • (Opțional) Import Wikipedia seed + run extraction

Pentru deploy non-interactiv, dezactivează corpus seed:

BRAIN_IMPORT_CORPUS=0 BRAIN_RUN_EXTRACTION=0 ./scripts/bootstrap_deploy.sh

8. Conectează web-api la rețeaua didibrain

Web-api rulează pe network didi-network, didi-brain pe didibrain. Trebuie unit:

docker network connect didibrain didiAI-web-api

Sau permanent în modules/web/deploy/docker-compose.yml (deja făcut: rețeaua didibrain e marcată ca external: true pe web-api).

9. Smoke test final

echo "=== web-api ==="; curl -sf http://localhost:51100/health
echo "=== dashboard ==="; curl -sf http://localhost:51300/health
echo "=== brain ==="; curl -sf http://localhost:8090/health
echo "=== brain cache test ==="
curl -sf -X POST http://localhost:8090/v1/gather \
  -H 'Content-Type: application/json' \
  -d '{"claim":"vaccines cause autism","max_evidence":3,"run_nli":false}' | jq '.brain_meta.cache_status'

Așteptat: "HIT" pe vaccines (e în corpus seed).

Pe scurt — ce trebuie pe noua mașină

Copy-paste:

  • Toate API keys din secțiunea API keys terti — identice pe orice deploy
  • Toate URL-urile interne (10.11.10.*) — dacă VPN-ul e același

Generează nou:

  • DASHBOARD_DB_PASSWORD (random 24 chars)
  • ATOMIC_TOKEN (auto, prin bootstrap script)
  • GATEWAY_API_TOKEN (random; doar dacă deploy-ezi gateway-ul)
  • POSTGRES_PASSWORD didi_brain (default insecure, schimbă în prod)

Configurabil opțional:

  • WEB_BRAIN_* thresholds (defaults safe)
  • DASHBOARD_HISTORY_RETENTION_DAYS (default 30)
  • LOG_LEVEL (INFO)