clossers/didi-lot2-backend
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
didi-lot2-backend/backend/services/data-layer/didiCache/INDEX.md
EVOTECH IT SRL 8ecc78e729 livrare lot 2
2026-07-10 03:39:53 -07:00

12 KiB
Raw Blame History

didiCache - Index

Productia ruleaza pe Redis LOCAL (container didi-cache pe didi-network). Decizie: stabilitate + zero dependinte externe + acces rapid. Clusterul Redis RAG (managed extern, HAProxy VIP peste rag01/02/03) ramane configurat ca fallback de urgenta pentru HA — activabil cu redis-switch.sh cluster, nu este folosit operational acum.

Productie (LOCAL — activ):

  • Container: didi-cache (imagine redis:7-alpine)
  • Host intern Docker: didi-cache:6379
  • DB: 0
  • Parola: REDIS_PASSWORD din .env (default redis123 in dev)
  • Memorie: 512MB (eviction: allkeys-lru)
  • Retea: didi-network; portul 6379 este expus pe host (0.0.0.0:6379->6379)
  • Persistenta: RDB + AOF

Fallback HA (cluster RAG — disponibil dar inactiv):

  • Host: 10.11.50.100 (HAProxy VIP)
  • Port: 16379
  • DB: 0
  • User: didi
  • Parola: din .cluster-credentials.env (gitignored)

Switch intre LOCAL si cluster: backend/services/orchestration-layer/scripts/redis-switch.sh {cluster|local|status} rescrie fisierele .env ale serviciilor (agent-v3, didiFramework) + restart containere. Status curent: local.

Helper de conectare folosit de toate serviciile:

  • agent-v3 -> src/shared/redis/connection.ts (createRedisConnection())
  • didiFramework -> src/config/redis.ts (createRedisConnection())

Ce face

Stocheaza date temporare si configurari pentru platforma DIDI:

  • Cache framework (parametri tehnici, verdicts, ponderi) -- scrise de didiFramework via sync-redis
  • Configurare HIL Moderation (triage thresholds, brain client config, sensitive topics, roluri) -- scrise de didiFramework
  • Sesiuni analiza (AnalysisSession JSON, TTL 7 zile) -- scrise de agent-v3
  • Rezultate intermediare per etapa (TTL 7 zile) -- scrise de agent-v3
  • Stare coada async (progres workeri) -- scrise de agent-v3
  • Lock-uri concurenta workeri (TTL 30s-5min) -- scrise de agent-v3
  • Chei API extensie browser (cache validare) -- scrise de didiFramework
  • Configurare modele viziune -- scrise de didiFramework

Sync-redis scrie ~51 chei de configurare (crescut de la 48 dupa adaugarea cheilor HIL Moderation in 2026-05-01). Plus framework_keys: 7 chei standard sau 8 daca include providers.

Cine scrie in Redis

Serviciu Ce scrie Chei Redis
didiFramework (sync-redis) Ierarhie tehnici, claims, verdicts, ponderi, surse, provideri didi:framework:*
didiFramework (sync-redis) Configurare componente pe etape didi:config:*
didiFramework (extension-keys) Cache validare chei API extensie didi:extension:key:*
agent-v3 (sesiuni) Sesiune completa JSON didi:pipeline:{sessionId}:status
agent-v3 (executori) Rezultate intermediare per etapa agent:result:{sessionId}:{component}:{stage}
agent-v3 (coada) Stare procesare async didi:queue:session:{sessionId}
agent-v3 (workeri) Lock-uri concurenta didi:queue:lock:*

Cine citeste din Redis

Serviciu Ce citeste Chei Redis
agent-v3 (executori) Configurare framework (tehnici, ponderi, verdicts) didi:framework:*
agent-v3 (executori) Modele disponibile, stage assignments, prompturi didi:config:*
agent-v3 (rute) Sesiuni pentru polling status didi:pipeline:{sessionId}:status
agent-v3 (rute) Rezultate intermediare agent:result:{sessionId}:*
agent-v3 (pipeline) Validare cheie extensie didi:extension:key:*
didiFramework (debug) Verificare date sincronizate didi:framework:*

Chei Redis principale

Pattern Scop TTL Scris de
didi:framework:manifest Index categorii + timestamp sync permanent didiFramework
didi:framework:techniques Ierarhie completa tehnici (denormalizata) permanent didiFramework
didi:framework:claims Parametri claims (tipuri, statusuri, confidence) permanent didiFramework
didi:framework:verdicts Categorii verdict + risk mappings + severity permanent didiFramework
didi:framework:weights Ponderi componente + scenarii + multiplicatori permanent didiFramework
didi:framework:sources Evaluare surse permanent didiFramework
didi:framework:providers Configurare LLM permanent didiFramework
didi:framework:dimensions_compact Lista compacta dimensiuni (screening) permanent didiFramework
didi:config:{component}:v1:* Config componenta (modele, etape, prompturi) permanent didiFramework
didi:config:techniques:v3:stage_assignments TIER-NESTED {stage: {free: {models}, premium: {models}}} — chain-uri LLM per tier permanent didiFramework
didi:config:ai-tampered:v1:stage_assignments TIER-NESTED stage assignments AI-Tampered permanent didiFramework
didi:config:claims:v1:stage_assignments TIER-NESTED stage assignments Claims permanent didiFramework
didi:config:source-assessment:v1:stage_assignments TIER-NESTED stage assignments Source Assessment permanent didiFramework
didi:config:vision:v1:stage_assignments TIER-NESTED (Etapa 4) stage image_analysis — citit de callVision() cu tier param permanent didiFramework
didi:config:verdict:v1:stage_assignments TIER-NESTED (Etapa 5) stage verdict_review — citit de verdict-explanation.ts loadModels(tier) permanent didiFramework
didi:config:ai-tampered:v1:vision_models Legacy flat vision config (fallback pentru vision.ts daca didi:config:vision:v1:stage_assignments lipseste) permanent didiFramework
didi:config:vision:v1:prompts:extraction Prompt viziune: extragere text din imagini permanent didiFramework
didi:config:vision:v1:prompts:video_frames Prompt viziune: analiza cadre video permanent didiFramework
didi:config:vision:v1:prompts:ai_detection Prompt viziune: detectie AI imagini permanent didiFramework
didi:config:pipeline:v1:component_config Config componente pipeline + video track weights permanent didiFramework
didi:config:pipeline:v1:session_config Config sesiune pipeline permanent didiFramework
didi:config:pipeline:v1:verdict_config Override-uri verdict, synergy, confidence (globale). JSONB sincronizat din PG bos_parammgmt.component_config WHERE (component_code='pipeline', config_key='verdict_config'). Contine: synergy + override-uri (false_claims, severe_techniques, undisclosed_ai, untrusted_domain, domain_red_flags) + confidence + confidence_levels. Editabil din admin dashboard via /api/verdicts/runtime-config (GET/PUT/PATCH). permanent didiFramework
didi:config:pipeline:v1:input_profiles Profiluri verdict per input type (6 profile: ponderi, override-uri, INCONCLUSIVE) permanent didiFramework
didi:config:techniques:v3:scoring_config Parametri scoring techniques (count_scaler, intensity, severe_threshold) permanent didiFramework
didi:config:ai-tampered:v1:scoring_config Parametri scoring AI (blend_weights, disclosure_impact, thresholds) permanent didiFramework
didi:config:claims:v1:scoring_config Parametri scoring claims (status_weights, unverified behavior) permanent didiFramework
didi:config:source-assessment:v1:scoring_config Parametri scoring source (axis_weights, verdict_thresholds) permanent didiFramework
didi:config:source-assessment:v1:available_models Modele LLM source assessment (union free+premium) permanent didiFramework
didi:config:source-assessment:v1:prompts:extraction Prompt extractie metadata sursa permanent didiFramework
didi:config:source-assessment:v1:prompts:evaluation Prompt evaluare sursa permanent didiFramework
didi:config:verdict:v1:available_models Legacy fallback pentru verdict reviewer (folosit daca stage_assignments lipseste) permanent didiFramework

HIL Moderation (synced from PG by sync-redis)

Adaugat in 2026-05-01. Citit de agent-v3 in triage.ts (cache local 60s) si brain/client.ts pentru configurare live.

Pattern Scop TTL Scris de
didi:config:moderation:v1:settings Single-row config: triage thresholds + brain client config (brain_enabled, brain_url, lookup/write timeouts, brain_confidence_min_silver, brain_semantic_threshold, brain_per_component) permanent didiFramework
didi:config:moderation:v1:sensitive_topics Active topics list (elections, health, war, covid, climate) permanent didiFramework
didi:config:moderation:v1:roles Keycloak role -> permissions (moderator, senior_moderator) cu toggle flags permanent didiFramework

Nota structura tier-nested (aplicabila tuturor cheilor stage_assignments):

{
  "techniques_screening": {
    "free":    { "models": [{"order":1,"model_key":"qwen35:Qwen3.5-397B-A17B",...}, ...] },
    "premium": { "models": [{"order":1,"model_key":"openrouter:google/gemini-3-flash-preview",...}, ...] }
  },
  "techniques_deep": { "free": {...}, "premium": {...} }
}

Sync-redis scrie cheia intr-un singur SET (atomic). agent-v3 citeste cheia o singura data, apoi rezolva tier-ul local cu stages[stageCode][tier] || stages[stageCode].free. | didi:pipeline:{sessionId}:status | Status executie pipeline (JSON: PipelineStatus) | 7 zile | agent-v3 | | didi:pipeline:{sessionId}:{component} | Rezultat componenta (JSON: TechniquesResult etc.) | 7 zile | agent-v3 | | didi:pipeline:{sessionId}:verdict | Rezultat verdict final (JSON: VerdictResult) | 7 zile | agent-v3 | | didi:pipeline:history:entry:{sessionId} | Date intrare sesiune (input + metadata) | 7 zile | agent-v3 | | didi:pipeline:history:user:{userId} | Sorted set istoric utilizator (score=timestamp) | 7 zile | agent-v3 | | agent:result:{sessionId}:{comp}:{stage} | Rezultat intermediar etapa | 7 zile | agent-v3 | | didi:queue:session:{sessionId} | Stare coada async | 24 ore | agent-v3 | | didi:queue:lock:{sessionId}:{comp} | Lock worker per componenta | 5 min | agent-v3 | | didi:queue:aggregator:{sessionId} | Lock agregator verdict | 30s | agent-v3 | | didi:extension:key:{key} | Cache cheie API extensie | permanent | didiFramework |

Configurare

redis.conf

  • Bind: 0.0.0.0 (acces din Docker network)
  • Protected mode: activat
  • Max memorie: 512MB
  • Eviction: allkeys-lru (sterge cele mai vechi chei cand se umple)
  • Persistenta RDB: snapshot la 60s/10000 keys, 300s/10 keys, 900s/1 key
  • Persistenta AOF: activat, sync everysec
  • Max clienti: 10000
  • 16 baze de date
  • Slow log: 10ms threshold
  • Parola: obligatorie (requirepass)

Pornire in productie

Nota: didi-cache NU este definit in data-layer/docker-compose.yml. Containerul Redis este definit in production/docker-compose.yml (alaturi de Kong si Keycloak).

# din production/docker-compose.yml
didi-cache:
  image: redis:7-alpine
  command: redis-server --requirepass ${REDIS_PASSWORD} --save 60 1 --save 300 10
  volumes:
    - didi-cache-data:/data
  networks:
    - didi-network
  healthcheck:
    test: redis-cli -a ${REDIS_PASSWORD} ping

Nota: redis.conf din directorul didiCache nu este montat in container in productie. Containerul foloseste parametrii din command line.

Conexiune

Productie (LOCAL — activ):

  • Host: didi-cache (Docker DNS pe didi-network)
  • Port: 6379 (expus si pe host: 0.0.0.0:6379->6379)
  • DB: 0
  • Parola: redis123 (din .env, REDIS_PASSWORD)

Fallback HA (cluster RAG — disponibil dar inactiv, doar dupa redis-switch.sh cluster):

  • Host: 10.11.50.100 (HAProxy VIP rag01/02/03)
  • Port: 16379
  • DB: 0
  • Username: didi
  • Parola: din .cluster-credentials.env (gitignored)

Administrare

  • Conexiune directa: redis-cli -h didi-cache -a redis123 (sau 127.0.0.1:6379 de pe host).

Fisiere in directorul didiCache

redis.conf          -- Configurare Redis completa (183 linii, nu e montata in productie)
.env.example        -- Template variabile de mediu
.gitignore          -- Exclude .env, data/, *.rdb, *.aof
README.md           -- Documentatie

Zero cod custom. Zero module Redis. Zero scripturi. Doar configurare.

Ce NU face

  • Nu e Redis Cluster (instanta singulara)
  • Nu e Redis Sentinel (fara failover automat)
  • Nu are module custom
  • Nu are replicare
  • Nu are ACL (doar autentificare cu parola)
  • Nu proceseaza nimic -- doar stocheaza si serveste date scrise de alte servicii