clossers/didi-lot1-ai
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
didi-lot1-ai/ai_platform/INDEX.md

49 KiB
Raw Blame History

INDEX - Documentatie tehnica completa ml-projects

Ultima actualizare: 2026-03-18

Acest fisier descrie fiecare modul, fiecare fisier, fiecare clasa, fiecare functie, fiecare ruta, fiecare container si fiecare port din acest monorepo. Nimic nu este omis.

Cuprins

  1. Arhitectura generala
  2. Gateway (nginx)
  3. Catalog API
  4. LLM Inference
  5. Embeddings
  6. Rerank
  7. Audio
  8. Video Analysis
  9. Web
  10. Harta porturilor
  11. Harta GPU
  12. Retea Docker

1. Arhitectura generala

Monorepo cu 8 module independente. Fiecare modul este un pachet Python instalabil cu FastAPI, containerizat in Docker, conectat pe reteaua comuna didi-network. Toate comunica prin HTTP intern. Singurul port expus extern este 11000 (gateway nginx).

Flux tipic de request extern:

Client -> Gateway (nginx :11000) -> Serviciu intern (llm/audio/web/catalog)

Flux intern (intre servicii):

Web API -> LLM Inference API -> vLLM (Qwen3.5)
Video API -> vLLM (BusterX)
Catalog API -> (interogheaza toate celelalte servicii pe /v1/info)

Modele ML servite:

  • Qwen3.5-35B-A3B (text + vision, MoE) - vLLM pe GPU 0
  • BAAI/bge-m3 (embeddings) - vLLM sau llama.cpp
  • BAAI/bge-reranker-v2-m3 (reranking) - vLLM sau llama.cpp
  • Whisper large-v3-turbo (speech-to-text) - faster-whisper pe GPU 0
  • BusterX / Qwen2.5-VL-7B (deepfake detection) - vLLM pe GPU 1

2. Gateway (nginx) - punctul unic de intrare

Locatie: modules/gateway/ Container: didiAI-gateway Port extern: 11000 Imagine: nginx:1.27-alpine

Gateway-ul este un reverse proxy nginx care ruteaza toate request-urile catre serviciile interne. Toate rutele (in afara de /health) necesita autentificare Bearer token.

Fisiere

deploy/nginx.conf.template - Template nginx cu variabile de mediu

Defineste 4 upstream-uri:

  • llm -> didiAI-llm-api:14011
  • audio -> didiAI-audio-api:54300
  • web -> didiAI-web-api:51100
  • catalog -> didiAI-catalog-api:11000

Autentificarea: nginx map compara header-ul Authorization cu Bearer ${GATEWAY_API_TOKEN}. Daca nu coincide, returneaza 401 JSON.

Rute:

  • GET /health - fara autentificare, returneaza {"status":"ok"} direct din nginx
  • /llm/ -> proxy catre llm upstream, cu SSE streaming (proxy_buffering off, chunked transfer)
  • /audio/ -> proxy catre audio upstream, cu body buffer 10M pentru upload-uri mari
  • /web/ -> proxy catre web upstream
  • /catalog/ -> proxy catre catalog upstream
  • / (orice altceva) -> 404 cu lista rutelor disponibile

Timeout-uri proxy: connect 60s, send 300s, read 600s. Upload maxim: 500MB (client_max_body_size). Toate request-urile primesc header X-Request-ID generat de nginx.

deploy/docker-compose.yml - Un singur serviciu gateway

Container didiAI-gateway pe imaginea nginx:1.27-alpine. La pornire, ruleaza envsubst care inlocuieste ${GATEWAY_API_TOKEN} in template si genereaza nginx.conf final. Healthcheck cu wget pe /health.

deploy/deploy.sh - Script bash

Valideaza variabila GATEWAY_API_TOKEN (fail-fast). Suporta actiunile up, down, logs. Incarca .env din directorul deploy/.

deploy/.env.example - O singura variabila required: GATEWAY_API_TOKEN.

3. Catalog API - agregator de servicii

Locatie: modules/catalog-api/ Container: didiAI-catalog-api Port intern: 11000 (accesat prin gateway la /catalog/)

Ce face

Interogheaza periodic sau la cerere endpoint-ul /v1/info de pe fiecare serviciu intern (LLM, Audio, Video, Web). Colecteaza metadata (modele disponibile, functii, starea de sanatate) si le expune intr-un singur loc. Genereaza si un OpenAPI spec agregat care combina spec-urile tuturor componentelor.

Fisiere sursa

src/catalog_api/settings.py

Clasa CatalogSettings(BaseSettings) cu prefix CATALOG_:

  • external_url: str (REQUIRED, fara default) - URL extern pentru OpenAPI
  • host, port, log_level - setari server (cu default-uri)
  • llm_url, audio_url, video_url, web_url - URL-uri interne Docker ale serviciilor
  • llm_external_port, audio_external_port, video_external_port, web_external_port - porturi externe

Clasa Component(BaseSettings):

  • id, url, external_url, timeout - metadata despre un serviciu

Metoda get_components() -> list[Component] - construieste lista componentelor configurate, omitand cele cu URL gol (ex: video_url="" dezactiveaza video).

Instanta globala settings = CatalogSettings().

src/catalog_api/app.py

Aplicatia FastAPI principala. Constanta AGGREGATED_OPENAPI_VERSION = "0.1.0".

Rute:

  • GET /health -> health() - returneaza {"status": "ok"}

  • GET /v1/components -> list_components() - face fetch async la /v1/info pe fiecare component din settings.get_components(). Adauga component_id si base_url la fiecare raspuns. Returneaza {components: [...], total: N, errors: [...]}. Gestioneaza timeout-uri si erori HTTP per component.

  • GET /v1/components/{component_id} -> get_component(component_id) - cauta componenta dupa ID, face fetch la /v1/info, returneaza metadata. 404 daca ID-ul nu exista.

  • GET /v1/models -> list_models() - colecteaza array-ul models din /v1/info al fiecarei componente. Adauga component_id si component_name la fiecare model. Ignora silentios componentele cu erori.

  • GET /v1/functions -> list_functions() - la fel ca models, dar extrage array-ul functions.

  • GET /v1/status -> get_status() - verifica conectivitatea cu fiecare componenta. Statusul general: "healthy" (toate ok), "degraded" (unele ok), "unhealthy" (niciuna). Returneaza {status, components: [...], healthy_count, total_count}.

  • GET /v1/openapi -> get_aggregated_openapi() - face fetch la /openapi.json de pe fiecare componenta. Combina spec-urile intr-un singur OpenAPI 3.1.0 cu titlul "didiAI - Aggregated ML Services API". Prefixeaza path-urile cu /{component_id} si schema-urile cu {component_id}_ pentru a evita coliziuni.

  • GET /v1/docs -> get_aggregated_swagger_ui() - pagina HTML cu Swagger UI care incarca /v1/openapi.

  • GET /v1/redoc -> get_aggregated_redoc() - pagina HTML cu ReDoc.

  • GET /v1/openapi/component/{component_id} -> get_component_openapi(component_id) - returneaza OpenAPI spec-ul brut al unei componente specifice.

Functii helper:

  • _merge_openapi_schemas(base_spec, component_spec, component_id, component_url) - combina spec-ul unei componente in spec-ul agregat. Prefixeaza path-urile, schema-urile, operationId-urile.
  • _update_refs(obj, component_id) - actualizeaza recursiv referintele $ref din obiectele OpenAPI.

La final, uvicorn.run() porneste serverul.

Deploy

deploy/Dockerfile - Multi-stage build pe python:3.11-slim cu uv. Port default 11000. CMD: uvicorn catalog_api.app:app.

deploy/docker-compose.yml - Serviciu catalog-api, container didiAI-catalog-api, port 11000, retea didi-network. Healthcheck pe /health.

4. LLM Inference - router LLM unificat

Locatie: modules/llm-inference/ Containere: didiAI-llm-api (port 14011), didiAI-vllm-qwen3.5 (port 14001)

Ce face

Gateway unificat pentru inferenta LLM. Primeste cereri OpenAI-compatibile si le ruteaza catre unul din 3 backend-uri: LiteLLM (100+ provideri cloud), vLLM (GPU local), llama.cpp (CPU local). Suporta streaming SSE, retry cu backoff exponential, rate limiting, concurrency limiting, autentificare Bearer, load balancing pentru llama.cpp.

Fisiere sursa

src/llm_inference/types.py - Tipuri de baza

  • BackendType(str, Enum) - LITELLM, VLLM, LLAMACPP
  • ChatMessage(BaseModel) - role (system/user/assistant/function/tool), content (str sau list multimodal), name optional
  • Usage(BaseModel) - prompt_tokens, completion_tokens, total_tokens
  • Choice(BaseModel) - index, message, finish_reason (raspuns non-streaming)
  • Delta(BaseModel) - role, content (raspuns streaming)
  • StreamChoice(BaseModel) - index, delta, finish_reason
  • ModelInfo(BaseModel) - id, backend, loaded, context_length, capabilities

src/llm_inference/schemas.py - Scheme API

  • CompletionRequest(BaseModel) - messages (1-1000), model, temperature (0-2, default 0.7), max_tokens, stream (bool), backend (override optional), top_p, frequency_penalty, presence_penalty, stop. Validator: non-assistant messages trebuie sa aiba content.
  • CompletionResponse(BaseModel) - id, object="chat.completion", created (unix timestamp), model, choices, usage, backend
  • CompletionChunk(BaseModel) - id, object="chat.completion.chunk", created, model, choices (StreamChoice)
  • ModelListResponse, ModelLoadRequest, ModelLoadResponse - management modele
  • BackendHealth, HealthResponse, ReadinessResponse, BackendListResponse - monitoring

src/llm_inference/config.py - Configurare Pydantic Settings

Clasa LLMSettings(BaseSettings) cu prefix LLM_:

Campuri REQUIRED (fara default):

  • default_backend: Literal["litellm", "vllm", "llamacpp"]
  • enable_vllm: bool
  • enable_llamacpp: bool
  • external_url: str

Campuri cu default:

  • host="0.0.0.0", port=14011
  • default_model="gpt-3.5-turbo"
  • openrouter_api_key, openai_api_key, anthropic_api_key - chei API optionale
  • vllm_base_url="http://localhost:14001", vllm_api_key
  • llamacpp_base_url="http://localhost:8102", llamacpp_base_urls (lista, comma-separated, pt load balancing)
  • llamacpp_health_check_interval=30 (interval verificare sanatate servere)
  • models_dir="/models"
  • request_timeout=120.0, connect_timeout=10.0
  • max_retries=3, retry_min_wait=1.0, retry_max_wait=60.0
  • rate_limit_rps=10.0, rate_limit_burst=20
  • max_concurrent_completions=10
  • api_tokens: frozenset[str] | None - tokeni Bearer (comma-separated)
  • log_level="INFO", log_json=False

Proprietati: auth_enabled, llamacpp_urls. Validatori: parse_llamacpp_base_urls() (parseaza string comma-separated in lista), parse_api_tokens(), validate_api_keys() (avertizeaza daca litellm fara chei API).

Clasa SettingsCache - singleton thread-safe cu get(), clear(), set().

src/llm_inference/exceptions.py - Exceptii custom

Baza: LLMInferenceError(Exception). Derivate:

  • AuthenticationError - autentificare esuata
  • BackendNotAvailableError(backend, reason) - backend indisponibil
  • BackendNotEnabledError(backend) - backend neactivat in config
  • ModelNotFoundError(model, backend) - model negasit
  • CompletionError(message, backend, model) - eroare la generare
  • ModelLoadError(model, backend, reason) - eroare la incarcare model
  • ModelListError(backend, reason) - eroare la listare modele
  • LLMRateLimitError(message, backend, retry_after) - rate limit atins
  • LLMTimeoutError(message, backend, timeout) - timeout
  • LLMConnectionError(backend, reason) - conexiune esuata

src/llm_inference/retry.py - Logica de retry

Importa conditionat exceptiile din litellm, openai, httpx. Defineste doua tupluri globale:

  • RETRYABLE_EXCEPTIONS - RateLimitError, Timeout, ServiceUnavailableError, TimeoutException, ConnectError
  • NON_RETRYABLE_EXCEPTIONS - AuthenticationError, BadRequestError, NotFoundError

Functii:

  • is_retryable_exception(exc) - verifica daca exceptia e retryable
  • extract_retry_after(exc) - extrage headerul Retry-After din exceptii de rate limit
  • translate_exception(exc, backend, model) - traduce exceptii provider-specifice in exceptii custom
  • retry_with_backoff(func, max_retries, min_wait, max_wait, backend, model) - executa functie async cu retry exponential. Formula: min_wait * 2^attempt, capped la max_wait, cu jitter 0-25%. Respecta Retry-After headers.

src/llm_inference/logging.py - Logging structurat

  • request_id_ctx: ContextVar[str | None] - propagare request ID prin context
  • RequestIdFilter(logging.Filter) - adauga request_id la log records
  • JsonFormatter(logging.Formatter) - formatare JSON cu timestamp, level, logger, message, request_id, exceptie
  • configure_logging(level, json_format) - configureaza root logger "llm_inference"
  • get_logger(name) - returneaza logger cu prefix "llm_inference."
  • set_request_id(request_id) / get_request_id() - management context

src/llm_inference/utils.py

  • safe_close_stream(stream, logger) - inchide sigur un stream async (incearca aclose(), fallback pe close())

src/llm_inference/image_processing.py - Procesare imagini multimodale

Client HTTP global partajat _http_client pentru download.

Functii:

  • _get_http_client() - returneaza/creeaza clientul HTTP partajat
  • _guess_mime_type(url, content_type) - determina MIME type din URL sau Content-Type
  • _download_and_encode(url) - descarca imagine si returneaza ca data URI base64
  • _process_content_item(item) - proceseaza un element de continut (descarca imagini HTTP, lasa base64 si non-HTTP neschimbate)
  • _process_text_with_urls(text) - detecteaza URL-uri de imagini in text plain si le converteste in format multimodal [{type: "text"}, {type: "image_url"}]
  • process_messages(messages: list[ChatMessage]) - proceseaza toate mesajele, descarcand URL-urile de imagini si convertindu-le in base64

src/llm_inference/client.py - Client de nivel inalt

Clasa LLMClient:

  • __init__(settings) - initializeaza cu settings si BackendRegistry
  • complete(messages, model, backend, **kwargs) - generare chat completion. Proceseaza imagini, rezolva backend-ul, apeleaza backend.complete()
  • stream(messages, model, backend, **kwargs) - generare streaming. Yield-uieste CompletionChunk
  • list_models(backend) - listeaza modele de la un backend sau toate
  • load_model(model, backend) - incarca model pe backend local
  • unload_model(model, backend) - descarca model
  • list_backends() - listeaza tipurile de backend disponibile
  • health_check() - returneaza starea de sanatate per backend
  • _resolve_backend_for_model(model) - interogheaza fiecare backend local sa vada care serveste modelul
  • _parse_messages(messages) - converteste dict-uri in ChatMessage

src/llm_inference/cli.py - Punct de intrare CLI

Clasa GracefulShutdown:

  • Inregistreaza handlere SIGTERM/SIGINT
  • Primul semnal: shutdown graceful
  • Al doilea semnal: exit fortat

Functia main():

  • Argumente: --host, --port, --workers, --reload, --graceful-timeout
  • Porneste uvicorn cu factory mode llm_inference.api.app:create_app

Backend-uri

src/llm_inference/backends/base.py - Clasa abstracta LLMBackend

Metode abstracte: name (property), complete(), stream(), list_models() Metode concrete (cu default): load_model() (NotImplementedError), unload_model() (NotImplementedError), health_check() (True)

src/llm_inference/backends/registry.py - Registru de backend-uri

Clasa BackendRegistry:

  • __init__(settings) - initializeaza backend-urile activate (LiteLLM mereu, vLLM/llama.cpp optional)
  • get(backend_type) - returneaza instanta backend (default daca None)
  • list_backends() - lista tipurilor disponibile
  • is_available(backend_type) - verifica disponibilitatea

src/llm_inference/backends/litellm_backend.py - Backend LiteLLM

Clasa LiteLLMBackend(LLMBackend):

  • Interfata cu 100+ provideri cloud (OpenAI, Anthropic, OpenRouter, Azure, Google, AWS)
  • _configure_litellm() - seteaza cheile API pe modulul litellm
  • complete() - apeleaza litellm.acompletion() cu retry_with_backoff
  • stream() - streaming prin litellm.acompletion(stream=True), yield-uieste CompletionChunk
  • list_models() - cache TTL 3600s. Fetcheaza dinamic de la OpenAI si OpenRouter API, fallback pe liste curate (hardcoded)
  • health_check() - verifica conectivitatea la cel putin un provider
  • Liste fallback: OpenAI (gpt-4o, gpt-4o-mini, gpt-4-turbo, gpt-3.5-turbo), Anthropic (Claude 3.5 Sonnet, 3 Opus, 3 Haiku)

src/llm_inference/backends/vllm_backend.py - Backend vLLM

Clasa VLLMBackend(LLMBackend):

  • Conectare la server vLLM prin API OpenAI-compatible (pachetul openai)
  • Creeaza AsyncOpenAI client care pointeaza la vllm_base_url
  • complete() - client.chat.completions.create() cu retry
  • stream() - streaming prin acelasi client, cu safe_close_stream la erori
  • list_models() - client.models.list()
  • health_check() - incearca models.list(), True daca reuseste

src/llm_inference/backends/llamacpp_backend.py - Backend llama.cpp cu load balancing

Dataclass LlamaCppServer:

  • url, healthy, request_count, error_count, last_check, response_times (ultimele 50)
  • Property: avg_response_ms, short_name

Clasa LlamaCppBackend(LLMBackend):

  • Creeaza obiecte LlamaCppServer si clienti AsyncOpenAI pentru fiecare URL din configurare
  • Round-robin cu failover automat
  • _get_server() - returneaza urmatorul server sanatos (round-robin). Daca toate sunt nesanatoase, incearca pe toate.
  • _mark_unhealthy(server) / _mark_healthy(server) - actualizeaza starea serverului
  • _periodic_health_check(interval) - task async care verifica periodic toate serverele
  • _complete_on_server(server, client, messages, model) - executa completare pe un server specific, masoara timp de raspuns
  • complete() - incearca servere sanatoase round-robin, failover la urmatorul pe erori de conexiune/timeout
  • stream() - streaming cu failover
  • list_models() - interogheaza primul server sanatos
  • health_check() - True daca orice server e disponibil
  • get_servers_status() - returneaza starea tuturor serverelor (URL, health, request_count, error_count, avg_response_ms)

API (FastAPI)

src/llm_inference/api/app.py - Factory aplicatie

Functia create_app():

  • Creeaza FastAPI cu titlu "LLM Inference API", versiune 0.1.0
  • Lifespan manager: la startup configureaza logging, initializeaza ConcurrencyLimiter si LLMClient. La shutdown logheaza.
  • Middleware (ordinea conteaza, primul adaugat = cel mai exterior):
    1. RateLimitMiddleware - token bucket, exclude /health si /ready
    2. RequestIdMiddleware - genereaza/extrage X-Request-ID
  • Routere montate: health, completions (/v1), models (/v1), info (/v1)

src/llm_inference/api/dependencies.py - Dependinte FastAPI

  • get_client(request) - returneaza LLMClient din app.state
  • get_settings(request) - returneaza settings din app.state
  • verify_bearer_token(request, authorization) - valideaza token Bearer. Comparatie constant-time cu hmac.compare_digest. 401 daca invalid.

Clasa ConcurrencyLimiter:

  • Limiteaza numarul de completari concurente cu asyncio.Semaphore
  • acquire(blocking=False) - context manager async. Non-blocking (default): 503 imediat daca nu sunt sloturi. Blocking: asteapta slot.
  • current_count, available - proprietati de monitorizare

Functii globale: init_concurrency_limiter(), get_concurrency_limiter(), require_completion_slot() (dependinta FastAPI).

src/llm_inference/api/middleware.py - Middleware

Clasa RequestIdMiddleware(BaseHTTPMiddleware):

  • Extrage X-Request-ID din header sau genereaza UUID
  • Seteaza in context (pt logging), in request.state, si in response headers

Clasa TokenBucket:

  • Algoritm token bucket pentru rate limiting
  • acquire() - incearca sa consume un token. Returneaza True/False.
  • retry_after() - secunde pana cand un token e disponibil.

Clasa RateLimitMiddleware(BaseHTTPMiddleware):

  • Foloseste TokenBucket. Exclude /health si /ready.
  • Returneaza 429 Too Many Requests cu header Retry-After cand limita e atinsa.
  • ATENTIE: rate limiting per-proces, nu distribuit. Fiecare replica are limita proprie.

src/llm_inference/api/routes/completions.py - Ruta de completari

Endpoint POST /v1/chat/completions:

  • Necesita Bearer token (daca auth activat)
  • Pentru streaming: creeaza generator SSE care yield-uieste chunks JSON, tine slot de concurenta pe intreaga durata stream-ului, ping la fiecare 15s
  • Pentru non-streaming: achizitioneaza slot, ruleaza completare, elibereaza slot
  • Erori: BackendNotAvailableError/BackendNotEnabledError -> 400, CompletionError -> 500

Functii helper:

  • _stream_generator(client, request) - generator SSE, trimite [DONE] la final, eroare ca eveniment SSE
  • _stream_with_slot(generator, limiter) - wrapper care tine slot-ul pe durata stream-ului
  • _build_completion_kwargs(request) - construieste kwargs din CompletionRequest

src/llm_inference/api/routes/models.py - Management modele

  • GET /v1/models - query param optional backend pentru filtrare
  • POST /v1/models/load - incarca model pe backend local (body: ModelLoadRequest)
  • POST /v1/models/unload - descarca model
  • GET /v1/backends - listeaza backend-urile disponibile

src/llm_inference/api/routes/health.py - Health checks

  • GET /health - status per-backend, status general (healthy/degraded/unhealthy)
  • GET /ready - probe Kubernetes, verifica backend-ul default

src/llm_inference/api/routes/info.py - Informatii component

  • GET /v1/info - returneaza metadata completa pentru catalog: resource (name, slug, config, auth, rate_limits, tags), models (cu capabilities, provider, endpoint), functions (Chat Completions, List Models, List Backends, Load Model, Unload Model cu input/output schema)

Deploy

deploy/docker-compose.yml - Defineste 3 servicii:

  1. llm-api (didiAI-llm-api, port 14011) - API-ul FastAPI. Profile: api, vllm. Variabile: backend config, chei API, concurrency.
  2. vllm-qwen3.5 (didiAI-vllm-qwen3.5, port 14001) - Server vLLM cu Qwen/Qwen3.5-35B-A3B. Profile: vllm. GPU 0. Image: vllm/vllm-openai:qwen3_5. Parametri: max-model-len 32000, gpu-memory-utilization 0.65, enable-prefix-caching, enable-auto-tool-choice (Hermes parser). Healthcheck cu 600s start_period (modelul se incarca lent).
  3. llamacpp (optional) - Image: ghcr.io/ggml-org/llama.cpp:server. Profile: llamacpp, full. GGUF model din MODELS_DIR. ctx-size 4096.

deploy/Dockerfile - Multi-stage: python:3.11-slim cu uv. Port intern 14011. CMD: python -m llm_inference.cli.

5. Embeddings - API de embeddings

Locatie: modules/embeddings/ Containere: didiAI-embeddings-api (port 14100/54100), didiAI-embeddings-vllm (port 14101/54101), didiAI-embeddings-llamacpp (port 14110/54110)

Ce face

API OpenAI-compatibil de embeddings cu suport pentru doua backend-uri: vLLM (GPU) si llama.cpp (CPU/GGUF). Modelul principal: BAAI/bge-m3 (max 8192 tokeni).

Fisiere sursa

Structura e identica cu llm-inference (acelasi tipar arhitectural). Diferentele principale:

src/embeddings/types.py

  • BackendType(str, Enum) - VLLM, LLAMACPP (fara LITELLM)
  • EmbeddingUsage(BaseModel) - prompt_tokens, total_tokens
  • EmbeddingData(BaseModel) - object="embedding", index, embedding (list[float])
  • ModelInfo(BaseModel) - id, backend, loaded, dimensions, max_input_tokens

src/embeddings/schemas.py

  • EmbeddingRequest - input (list[str] sau str), model, encoding_format ("float"/"base64"), dimensions (optional), backend (override). Validatori: ensure_list() converteste str in list, validate_input() verifica ca input-ul nu e gol.
  • EmbeddingResponse - object="list", data (list[EmbeddingData]), model, usage, backend
  • encode_embedding_base64(embedding) - encodeaza vector embedding ca base64 (little-endian floats)

src/embeddings/config.py - EmbeddingSettings cu prefix EMB_:

  • Required: default_backend, enable_vllm, enable_llamacpp, external_url
  • Default-uri: port=54100, vllm_base_url="http://localhost:54101", llamacpp_base_url="http://localhost:54110"
  • Rate limiting: 20 RPS, burst 40, max 20 concurrent

src/embeddings/backends/base.py - EmbeddingBackend(ABC):

  • embed(texts, model, dimensions) -> tuple[list[list[float]], EmbeddingUsage]
  • list_models(), health_check()

src/embeddings/backends/vllm_backend.py - VLLMEmbeddingBackend:

  • Foloseste AsyncOpenAI client catre serverul vLLM
  • embed() - client.embeddings.create(), returneaza vectori si usage

src/embeddings/backends/llamacpp_backend.py - LlamaCppEmbeddingBackend:

  • Identic cu vLLM dar pointeaza la serverul llama.cpp

src/embeddings/client.py - EmbeddingClient:

  • embed(texts, model, backend, dimensions) - genereaza embeddings
  • list_models(backend), list_backends(), health_check()

src/embeddings/api/routes/embeddings.py - POST /v1/embeddings:

  • Primeste EmbeddingRequest, apeleaza client.embed()
  • Erori: 400 (backend invalid), 429 (rate limit), 504 (timeout), 503 (conexiune), 500 (eroare generala)

Celelalte fisiere (cli.py, logging.py, exceptions.py, middleware.py, dependencies.py, routes/models.py, routes/health.py) sunt structurate identic cu llm-inference, adaptate pentru embeddings.

Deploy

deploy/docker-compose.yml - 3 servicii:

  1. embeddings-api (didiAI-embeddings-api) - FastAPI API. Profile: api, vllm, llamacpp.
  2. vllm-embed (didiAI-embeddings-vllm) - Image: vllm/vllm-openai:v0.8.5. Model: BAAI/bge-m3 (configurabil). Task: embed. gpu-memory-utilization configurable (default 0.50). max-model-len configurable (default 8192). Profile: vllm.
  3. llamacpp-embed (didiAI-embeddings-llamacpp) - Image: ghcr.io/ggml-org/llama.cpp:server. Model GGUF. Mod embedding activat. ctx-size 8192, threads 4, parallel 4. Profile: llamacpp.

6. Rerank - API de reranking

Locatie: modules/rerank/ Containere: didiAI-rerank-api (port 14200/54200), didiAI-rerank-vllm (port 14201/54201), didiAI-rerank-llamacpp (port 14210/54210)

Ce face

API compatibil Cohere/Jina pentru reranking documente. Primeste un query si o lista de documente, returneaza documentele sortate dupa relevanta cu scoruri. Doua backend-uri: vLLM (GPU) si llama.cpp (CPU).

Fisiere sursa

Structura identica cu embeddings. Diferente specifice:

src/rerank/types.py

  • RerankUsage(BaseModel) - total_tokens
  • RerankResult(BaseModel) - index (pozitia originala), relevance_score, document (optional)
  • ModelInfo - id, backend, loaded, max_input_tokens

src/rerank/schemas.py

  • RerankRequest - model, query (min_length=1), documents (1-1000, fara stringuri goale), top_n (optional), return_documents (bool, default False), backend (override)
  • RerankResponse - id (generat: "rerank-{uuid12}"), model, results (list[RerankResult]), usage, backend

src/rerank/config.py - RerankSettings cu prefix RERANK_:

  • Required: default_backend, enable_vllm, enable_llamacpp, external_url
  • Default-uri: port=54200, vllm_base_url="http://localhost:54201", llamacpp_base_url="http://localhost:54210"
  • Rate limiting: 20 RPS, burst 50, max 20 concurrent

src/rerank/backends/vllm_backend.py - VLLMRerankBackend:

  • Foloseste httpx.AsyncClient pentru POST la /rerank (nu API OpenAI)
  • rerank(query, documents, model, top_n) - trimite cerere, parseaza rezultatele, returneaza [(index, score), ...] si usage

src/rerank/backends/llamacpp_backend.py - LlamaCppRerankBackend:

  • POST la /rerank. Gestioneaza field-uri alternative: "relevance_score" sau "score".
  • Sorteaza descrescator dupa scor, aplica top_n.

src/rerank/client.py - RerankClient:

  • rerank(query, documents, model, backend, top_n, return_documents) - obtine backend, apeleaza rerank, construieste RerankResult-uri
  • list_models(), list_backends(), health_check()

src/rerank/api/routes/rerank.py - Doua routere:

  • POST /v1/rerank si POST /v2/rerank (alias) - ambele apeleaza _handle_rerank() care achizitioneaza slot de concurenta, apeleaza client.rerank()

Deploy

deploy/docker-compose.yml - 3 servicii:

  1. rerank-api (didiAI-rerank-api). Profile: api, vllm, llamacpp.
  2. vllm-rerank (didiAI-rerank-vllm) - Image: vllm/vllm-openai:v0.8.5. Task: score. Model: BAAI/bge-reranker-v2-m3. Profile: vllm. GPU configurable. 300s start_period.
  3. llamacpp-rerank (didiAI-rerank-llamacpp) - Image: ghcr.io/ggml-org/llama.cpp:server. Model GGUF. Mod reranking activat. Profile: llamacpp.

7. Audio - transcriere audio

Locatie: modules/audio/ Container: didiAI-audio-api (port 54300)

Ce face

Serviciu speech-to-text folosind faster-whisper (de 4x mai rapid decat Whisper original). API OpenAI-compatibil. Suporta 99+ limbi, detectie automata limba, VAD filtering.

Fisiere sursa

src/audio/settings.py - Settings(BaseSettings) cu prefix AUDIO_:

  • model="large-v3-turbo" - modelul Whisper
  • device="cuda" - cuda sau cpu
  • compute_type="int8" - tip de cuantizare (int8, float16, int8_float16)
  • cache_dir="/root/.cache/huggingface"
  • beam_size=5, best_of=5, temperature=0.0
  • host, port=8200, log_level, external_url (REQUIRED)
  • max_file_size_mb=500

src/audio/schemas.py

  • TranscriptionSegment - id, seek, start, end, text, tokens, temperature, avg_logprob, compression_ratio, no_speech_prob
  • TranscriptionResponse - text, language, duration, segments (optional, doar pt verbose_json)
  • TranscriptionRequest - model, language, prompt, response_format ("json"/"text"/"verbose_json"), temperature

src/audio/transcriber.py

Clasa Transcriber:

  • __init__() - incarca WhisperModel cu model, device, compute_type, cache_dir din settings
  • transcribe(audio_path, language, initial_prompt, temperature) - apeleaza self.model.transcribe() cu beam_size, best_of, VAD filter (min_silence 500ms). Colecteaza segmente. Returneaza (text_complet, metadata).
  • Metadata: language, language_probability, duration, duration_after_vad, all_language_probs, segments

Functia get_transcriber() - singleton, instantiaza Transcriber la primul apel.

src/audio/app.py

Aplicatie FastAPI "Audio Transcription API".

La startup (startup_event) incarca modelul Whisper in memorie.

Rute:

  • GET /health -> {"status": "ok"}
  • GET /v1/models -> lista cu un singur model (cel configurat), format OpenAI-compatibil
  • POST /v1/audio/transcriptions -> endpoint principal de transcriere
    • Parametri form: file (UploadFile) SAU url (str), model, language, prompt, response_format, temperature
    • _get_audio_content(file, url) - obtine continut audio din upload sau URL. Valideaza dimensiune contra max_file_size_mb.
    • _download_url(url) - descarca audio de la URL cu httpx
    • Flux: obtine audio -> salveaza in fisier temporar -> transcrie -> formateaza raspuns -> sterge fisier temp
    • Formate raspuns: "text" (PlainTextResponse), "json" (TranscriptionResponse), "verbose_json" (cu segmente detaliate)
  • GET /v1/info -> metadata pentru catalog (resource, models, functions)

Deploy

deploy/Dockerfile - Bazat pe nvidia/cuda:12.1.0-runtime-ubuntu22.04. Instaleaza Python 3.10, ffmpeg. Nu foloseste uv, ci pip direct. Port 54300.

deploy/docker-compose.yml - Serviciu audio-api, container didiAI-audio-api. GPU 0 (CUDA_VISIBLE_DEVICES=0). Volum pentru cache modele. Profile: api. Start period 60s.

deploy/deploy.sh - Valideaza AUDIO_MODEL, AUDIO_DEVICE, AUDIO_CACHE_DIR. Suporta profile api si api-nginx.

8. Video Analysis - analiza video

Locatie: modules/video-analysis/ Containere: didiAI-video-api (port 54600), didiAI-video-vllm-buster (port 54500)

Ce face

Doua functionalitati:

  1. Detectie deepfake - extrage 16 frame-uri uniforme, le trimite la BusterX (model fine-tuned pe Qwen2.5-VL-7B), obtine verdict REAL/FAKE/INCONCLUSIVE
  2. Analiza semantica - divide video-ul in chunk-uri temporale (default 10s), extrage 24 frame-uri/chunk, descrie fiecare chunk cu LLM vision, optional agrega intr-un summary final

Fisiere sursa

src/video_analysis/settings.py - Settings(BaseSettings) cu prefix VIDEO_ANALYSIS_:

Required:

  • vllm_base_url - URL server vLLM pt deepfake (ex: http://vllm-buster:8000)
  • vllm_model - nume model (ex: "busterx")
  • runs_dir - director artefacte
  • external_url - URL extern OpenAPI

Optional:

  • semantic_vllm_base_url, semantic_vllm_model - vLLM separat pt analiza semantica
  • frames=16 - nr frame-uri pt sampling uniform (1-64)
  • max_side=960 - dimensiune maxima frame (100-2048)
  • jpeg_quality=85 - calitate JPEG (1-100)
  • max_tokens=750, temperature=1e-6, repetition_penalty=1.05
  • analysis_prompt - prompt pt deepfake ("analyze whether...")
  • semantic_prompt - prompt pt descriere chunk
  • aggregation_prompt_template - template pt agregare
  • semantic_chunk_duration_s=10.0, semantic_frames_per_chunk=24
  • semantic_enable_aggregation=True
  • semantic_aggregation_model="qwen3.5" - modelul pt agregare (LLM text, nu vision)
  • semantic_llm_base_url - URL LLM text pt agregare

Suporta configurare din deploy/config.yaml (YAML), cu override din variabile de mediu.

src/video_analysis/video_sampling.py - Utilitare pentru sampling frame-uri

  • get_video_props(cap) - extrage total_frames, fps, duration_s, width, height din cv2.VideoCapture
  • compute_uniform_indices(total, num_frames) - calculeaza indici uniformi. Formula: round(i * (total-1) / (num_frames-1))
  • sample_frames_uniform(video_path, num_frames=16) - deschide video cu OpenCV, selecteaza frame-uri uniform. Daca total_frames necunoscut, citeste pana la 2000 frame-uri si subsampleaza. Returneaza (liste frame-uri, metadata cu timpi si indici)
  • sample_frames_chunked(video_path, chunk_duration_s=10.0, frames_per_chunk=24) - divide video in chunk-uri temporale. Calculeaza nr chunk-uri = ceil(duration/chunk_duration). Pentru fiecare chunk: calculeaza interval temporal, converteste in indici frame, extrage frame-uri (uniform daca chunk > frames_per_chunk). Returneaza (lista de liste de frame-uri, metadata)

src/video_analysis/buster_client.py - Client vision LLM

  • frame_to_data_url_b64jpeg(frame_bgr, max_side, jpeg_quality) - converteste frame BGR la RGB PIL Image, scaleaza la max_side, encodeaza JPEG, returneaza data URI base64
  • call_vllm_chat(base_url, model, data_urls, prompt, max_tokens, temperature, repetition_penalty, timeout_s=180) - construieste payload cu imagini + text, POST la /v1/chat/completions, masoara timpul. Returneaza (response JSON, elapsed_seconds)
  • parse_verdict_and_explanation(model_text) - verifica primele 20 caractere (uppercase) pt prefix verdict. REAL/FAKE/altceva=INCONCLUSIVE.

src/video_analysis/schemas.py

  • Verdict = Literal["REAL", "FAKE", "INCONCLUSIVE"]
  • Usage - prompt_tokens, completion_tokens, total_tokens
  • LatencyS - sampling_time_s, encode_time_s, model_inference_time_s
  • Meta - fps, total_frames, duration_s, sampled, indices, timestamps_s
  • AnalyzeResponse - request_id (UUID), run_dir, verdict, explanation, usage, latency_s, meta
  • ChunkResult - chunk_idx, time_range, description, frames_analyzed, inference_time_s, usage
  • SemanticMeta - fps, total_frames, duration_s, chunk_duration_s, frames_per_chunk, total_frames_sampled
  • SemanticAnalysisResponse - request_id, run_dir, analysis_type="semantic", video_duration_s, num_chunks, chunk_results, final_summary, aggregation_time_s, total_latency_s, meta

src/video_analysis/app.py

Rute:

  • GET /health -> {"status": "ok"}

  • POST /analyze/video -> deepfake detection

    1. Genereaza UUID, creeaza run_dir
    2. Salveaza video, calculeaza SHA256
    3. sample_frames_uniform() cu settings.frames
    4. frame_to_data_url_b64jpeg() pt fiecare frame
    5. Salveaza request metadata in JSON
    6. call_vllm_chat() cu data_urls + analysis_prompt
    7. parse_verdict_and_explanation()
    8. Salveaza result in JSON, returneaza AnalyzeResponse

  • POST /analyze/video/semantic -> analiza semantica

    1. Parametri form: file, chunk_duration_s, frames_per_chunk, enable_aggregation
    2. Selecteaza semantic vLLM daca configurat, altfel fallback la vLLM principal
    3. sample_frames_chunked() - divide in chunk-uri
    4. Per chunk: encodeaza frame-uri, call vLLM chat, extrage text, creeaza ChunkResult
    5. Daca aggregation activat si >1 chunk: construieste prompt cu descrierile chunk-urilor, apeleaza LLM text (semantic_llm_base_url) pt summary final
    6. Returneaza SemanticAnalysisResponse

  • GET /v1/info -> metadata catalog

Functii helper: safe_mkdir(), write_json(), sha256_file().

Deploy

deploy/docker-compose.yml - 2 servicii:

  1. vllm-buster (didiAI-video-vllm-buster, port 54500) - Image: vllm/vllm-openai:latest. Model: l8cv/BusterX_plusplus (served as "busterx"). GPU 1. max-model-len 32768, gpu-memory-utilization 0.25, prefix caching activat. Profile: api-vllm. 600s start_period.
  2. video-analysis-api (didiAI-video-api, port 54600) - FastAPI. Volum ../runs montat la /app/runs. Profile: api, api-vllm.

deploy/Dockerfile - Multi-stage python:3.11-slim cu uv. Port 54600.

9. Web - cautare web si fact-checking

Locatie: modules/web/ Container: didiAI-web-api (port 51100)

Ce face

Modul complex de fact-checking cu pipeline complet: detectie context -> cautare web (SearXNG) -> extragere continut (HTTP/Playwright/Vision) -> impachetare dovezi (deduplicare, extragere snippete cu LLM, scoring relevanta). Pipeline cu fallback automat si cautare multi-round bazata pe context.

Fisiere sursa

src/web/config.py - WebSettings(BaseSettings) cu prefix WEB_:

Required:

  • searxng_base_url - URL SearXNG (ex: http://localhost:55100)
  • llm_base_url - URL LLM inference server
  • external_url - URL extern OpenAPI

Campuri cu default (selectie principala):

  • port=51100, host="0.0.0.0"
  • vision_model="qwen-vl", text_model="qwen3-235b" - modele LLM
  • llm_api_key, openai_api_key, anthropic_api_key - chei API
  • fetch_timeout=30, fetch_user_agent - setari HTTP
  • browse_timeout=30000, browse_viewport_width=1280 - setari Playwright
  • vision_max_tokens=2000, vision_concurrency=3 - setari Vision
  • evidence_max_items=30, evidence_dedup_threshold=0.9 - setari Evidence
  • rate_limit_rps=10.0, rate_limit_burst=20
  • api_tokens: frozenset[str] | None - auth
  • context_detection_enabled=True - detectie context activata/dezactivata

src/web/exceptions.py - Exceptii custom:

  • WebError, AuthenticationError, ProviderError, ProviderNotAvailableError, SearchError, RateLimitError, WebTimeoutError, WebConnectionError

src/web/orchestrator.py - Orchestratorul principal

Clasa Orchestrator:

  • Proprietati lazy-loaded: search_client, fetch_client, browse_client, vision_client, evidence_packer, context_detector

Metoda gather(request, request_id):

  • Ruleaza pipeline-ul complet cu timeout global
  • Inregistreaza duratele fiecarui stage

Metoda _run_pipeline(request, request_id, stages):

Stage 0 - Context Detection (_run_context_stage):

  • Analizeaza claim-ul cu LLM-ul local
  • Detecteaza tara, limba, entitati, genereaza query-uri optimizate

Stage 1 - Search (_run_search_stage):

  • Cautare multi-round (cand contextul e disponibil):
    • Round 1: surse oficiale + media din tara detectata
    • Round 2: surse internationale de fact-checking
    • Round 3: cautare normala nerestrictata
  • Rezultatele se combina si se deduplica

Stage 2 - Fetch (_run_fetch_stage):

  • Lant de fallback: HTTP fetch -> Browse (Playwright) -> Vision (screenshot + LLM)
  • Conditii de escaladare:
    • Text extras prea scurt (< fetch_min_text_length)
    • Pagina necesita JavaScript (detectat prin indicatori SPA)
    • Erori HTTP 401/403
    • URL-uri PDF sunt sarite complet

Stage 3 - Evidence (_run_evidence_stage):

  • Deduplicare, extragere snippete, scoring relevanta

src/web/search/searxng.py - Client SearXNG

Clasa SearXNGClient:

  • search(request, request_id) - executa query-uri in paralel, combina rezultatele
  • _search_single(query, ...) - cautare singura cu rate limiting
  • _build_query(query, site_allowlist, site_blocklist) - adauga filtre de site (format site:example.com)
  • _execute_request(params, ...) - cu retry si backoff exponential
  • _parse_results(data, query) - extrage SearchResult din raspunsul SearXNG
  • image_search(request, request_id) - cautare imagini prin SearXNG
  • health_check() - probe /healthz

src/web/fetch/client.py - Client HTTP

Clasa FetchClient:

  • MAX_PAGE_SIZE = 5MB
  • fetch(request, request_id) - fetch paralel pe URL-uri
  • _fetch_single(url, ...) - fetch cu extragere continut
  • _extract_content(html, url) - extragere text cu 3 nivele de fallback:
    1. readability-lxml (calitate cea mai buna)
    2. BeautifulSoup4 (fallback)
    3. Regex (ultima sansa)
  • _detect_javascript_required(html, text) - detecteaza pagini JS-heavy: "enable javascript", <noscript>, indicatori SPA (react-root, ng-app, __next)

src/web/browse/client.py - Client Playwright

Clasa BrowseClient:

  • browse(request, request_id) - navigare paralela cu browser headless
  • _browse_single(url, ...) - navigare cu asteptare continut dinamic (networkidle, selectori custom), screenshot optional
  • _extract_content(page) - manipulare DOM + extragere text
  • Suport: data publicare, URL canonic, screenshot base64

src/web/vision/client.py - Client Vision LLM

Clasa VisionClient:

  • Foloseste LLMProviderChain pt fallback provider (local -> OpenAI -> Anthropic)
  • extract(request, request_id) - extragere paralela
  • _extract_single(url, ...) - screenshot + apel vision LLM
  • _call_vision_llm(messages, provider, ...) - apeleaza modelul vision cu imagini
  • _extract_images(page) - analizeaza imaginile de pe pagina

src/web/evidence/packer.py - Impachetare dovezi

Clasa EvidencePacker:

Algoritm de deduplicare:

  • SimHash fingerprinting (64-bit) pentru comparare rapida O(n)
  • Distanta Hamming ca prag de candidati
  • SequenceMatcher pentru comparare precisa
  • Multi-nivel: hash exact SHA256, SimHash, similaritate precisa (ratio lungime, prefix/sufix, shingles pt texte lungi)

Metode:

  • pack(request, request_id) - pipeline complet: deduplicare -> extragere snippete -> scoring
  • _deduplicate(pages) - deduplicare pe baza de SimHash
  • _create_evidence_item(page, claim) - creeaza EvidenceItem din PageContent
  • _extract_snippet_llm(text, claim) - extrage snippet relevant cu LLM
  • _extract_snippet_and_score_llm(text, claim) - snippet + scor relevanta intr-un singur apel
  • _score_relevance(text, claim) - scoring relevanta cu LLM
  • _score_credibility_simple(url) - scoring credibilitate pe baza de domeniu (Reuters, BBC, Nature etc. primesc scor mare)
  • _extract_snippets_batch(pages, claim) - grupuri de 2-3 pagini per apel LLM
  • _summarize_batch(pages, claim) - sumarizare in batch

Tratament special: elimina taguri <think> din output-ul modelelor de reasoning. Circuit breaker: cache negativ de 60s daca LLM-ul nu e disponibil.

src/web/llm/provider.py - Lant de provideri LLM

Clasa LLMProviderChain:

  • 3 provideri in ordine: local (vLLM), OpenAI, Anthropic
  • call_chat(messages, provider, max_tokens, temperature) - apeleaza providerul specificat
  • _call_local(messages, ...) - apel HTTP direct la vLLM-ul local
  • _call_openai(messages, ...) - API OpenAI
  • _call_anthropic(messages, ...) - API Anthropic (converteste formatul mesajelor)
  • Suport multimodal: data URI-uri si URL-uri de imagini

src/web/context/detector.py - Detectie context

Clasa ContextDetector:

  • detect(claim) - analizeaza claim-ul inainte de cautare
  • _detect_with_llm(claim) - apeleaza LLM local pt a extrage: tara principala (ISO 3166-1), limba, entitati (persoane, institutii, locatii), query-uri optimizate de cautare
  • _is_llm_available() - verifica disponibilitatea LLM-ului cu probe la /v1/models, cache negativ 60s
  • Fallback: returneaza SearchContext gol daca LLM indisponibil

src/web/context/sources.py - Surse pe tari

Surse predefinite per tara:

  • RO (Romania): gov.ro, cdep.ro, senat.ro, digi24.ro, hotnews.ro etc.
  • US (SUA): whitehouse.gov, congress.gov, nytimes.com, apnews.com etc.
  • Surse fact-check internationale: Reuters, Snopes, PolitiFact, FactCheck.org, FullFact, BBC, AFP, Veridica.ro

src/web/validation.py - Validare URL (protectie SSRF)

  • validate_url(url) - verificari sincrone: schema (http/https), hostname blocklist (localhost, metadata.google.internal, 169.254.169.254), IP-uri private
  • validate_url_dns(url) - verificare DNS async
  • validate_urls_async(urls) - validare DNS in paralel
  • Blocate: 127.0.0.0/8, 10.0.0.0/8, 192.168.0.0/16, 172.16.0.0/12

Scheme (schemas/)

  • common.py - FailedUrl, PageImage, PageContent (url, title, text, hash, extraction_method, timestamps), ProviderHealth, HealthResponse, ErrorDetail, ErrorResponse
  • search.py - SearchResult (query, url, title, snippet, rank, site, published_at), SearchRequest (queries, max_results, site_allowlist/blocklist, language, country, freshness, safe_search), SearchResponse
  • fetch.py - FetchRequest (urls, auto_fallback, method, min_text_length, parallel_fetches), FetchPageResult (status_code, content_type, needs_fallback), FetchResponse
  • browse.py - BrowseRequest (urls, wait, timeout, screenshot), BrowsePageResult (final_url, screenshot_base64, viewport), BrowseResponse
  • vision.py - ImageContext, VisionExtractRequest (urls, context_query, model, screenshots, provider), VisionPageResult (extracted_text, images, tokens_used), VisionExtractResponse
  • evidence.py - EvidenceItem (url, title, publisher, snippet, summary, full_text, relevance_score, credibility_score, provenance), EvidencePackRequest (pages, claim, dedup, LLM flags, limits), EvidenceStats, EvidencePackResponse
  • gather.py - GatherRequest (claim 10-1000 chars, search/fetch/evidence options, context detection, timeout), GatherStageResult (stage, success, counts, duration, error), GatherResponse (evidence, stats, search_results, stages, execution_time)
  • image_search.py - ImageSearchResult (image_url, thumbnail, source_url, title, dimensions), ImageSearchRequest, ImageSearchResponse
  • context.py - EntitySet (persons, institutions, locations), SearchContext (primary_country, entities, detected_language, search_queries)

API

Rute:

  • GET /health - starea serviciului
  • GET /ready - readiness probe
  • POST /v1/search - cautare web prin SearXNG
  • POST /v1/image-search - cautare imagini
  • POST /v1/fetch - fetch HTTP cu extragere continut
  • POST /v1/gather - pipeline complet de fact-checking (endpointul principal)
  • GET /v1/info - metadata catalog

Deploy

deploy/docker-compose.yml - Serviciu didiAI-web-api, port 51100. SHM 2GB (pt Playwright browsers). Profile: api. Depinde de SearXNG si llm-inference.

deploy/Dockerfile - Multi-stage cu python:3.11-slim. Instaleaza browsere Playwright. Creeaza user non-root. Port 51100.

10. Harta completa a porturilor

PRODUCTION (1xxxx):
  11000  Gateway (nginx)         - singurul port expus extern
  14001  vLLM Qwen3.5-35B-A3B   - server LLM text+vision
  14011  LLM Inference API       - router LLM unificat
  14100  Embeddings API          - API embeddings
  14101  vLLM Embed Server       - backend GPU embeddings
  14110  llama.cpp Embed Server  - backend CPU embeddings
  14200  Rerank API              - API reranking
  14201  vLLM Rerank Server      - backend GPU reranking
  14210  llama.cpp Rerank Server - backend CPU reranking

DEVELOPMENT (5xxxx):
  51100  Web API                 - fact-checking + cautare web
  54100  Embeddings API Dev
  54101  vLLM Embed Server Dev
  54110  llama.cpp Embed Server Dev
  54200  Rerank API Dev
  54201  vLLM Rerank Server Dev
  54210  llama.cpp Rerank Server Dev
  54300  Audio API               - transcriere Whisper
  54500  BusterX vLLM            - server vision deepfake
  54600  Video Analysis API      - analiza video

Schema porturi: 5 cifre. Prima cifra: 1=prod, 5=dev. A doua cifra: 1=API/Gateway, 4=LLM/AI.

11. Alocare GPU

GPU Ce ruleaza VRAM folosit VRAM total
GPU 0 Qwen3.5-35B-A3B (~57GB) + Whisper large-v3-turbo (~2GB) ~59GB 143GB
GPU 1 BusterX / Qwen2.5-VL-7B (~22GB) ~22GB 143GB

12. Retea Docker

Toate containerele sunt pe reteaua externa didi-network. Comunicarea interna se face prin DNS Docker (nume containere):

didiAI-gateway          -> didiAI-llm-api, didiAI-audio-api, didiAI-web-api, didiAI-catalog-api
didiAI-llm-api          -> didiAI-vllm-qwen3.5
didiAI-catalog-api      -> didiAI-llm-api, didiAI-audio-api, didiAI-video-api, didiAI-web-api
didiAI-web-api          -> SearXNG, didiAI-llm-api
didiAI-video-api        -> didiAI-video-vllm-buster, didiAI-llm-api (pt agregare semantica)
didiAI-embeddings-api   -> didiAI-embeddings-vllm, didiAI-embeddings-llamacpp
didiAI-rerank-api       -> didiAI-rerank-vllm, didiAI-rerank-llamacpp

Naming convention containere: didiAI-{modul}-{serviciu}.

Recent Changes (2026-05-05)

  • Login Keycloak SSO functional la /admin-ai/: realm didi-admins (mutat din didi-clients), client ai-platform-dashboard (creat in didi-admins ca clona), required role admin. SSO comun cu admin-dashboard backend (1 login = ambele dashboard-uri).
  • AI dashboard env: VITE_KEYCLOAK_URL=https://sso.clossers.com, VITE_KEYCLOAK_REALM=didi-admins, VITE_KEYCLOAK_CLIENT_ID=ai-platform-dashboard, VITE_KEYCLOAK_REQUIRED_ROLE=admin. Dual var pentru build (VITE_) + runtime (DASHBOARD_).
  • Schema config DB-overridable: tabel nou config_schema_override (auto-creat la startup), helper _merged_schema(session) in routes/config.py, endpoint-uri admin GET /api/config/schema/_overrides, PUT /api/config/schema/{key}, DELETE /api/config/schema/{key}. Audit trail (action config.schema.upsert/delete/seed).
  • Migrare automata 98 chei -> DB: la primul startup, seed_schema_if_empty() populeaza tabelul din KNOWN_KEYS (idempotent). Codul KNOWN_KEYS ramane fallback daca DB e sters. DB = single source of truth pentru schema acum.
  • didi_brain endpoint nou: GET /v1/fact_status/due_for_recheck?limit=N&volatility=X (facts cu next_check_at <= now, nelocked). Plus rate-limit pe POST /v1/cache/invalidate (10/h per actor, dry-run free).