216 lines
12 KiB
Markdown
216 lines
12 KiB
Markdown
# didiCache - Index
|
|
|
|
**Productia ruleaza pe Redis LOCAL** (container `didi-cache` pe `didi-network`). Decizie: stabilitate + zero dependinte externe + acces rapid. Clusterul Redis RAG (managed extern, HAProxy VIP peste rag01/02/03) ramane configurat ca **fallback de urgenta pentru HA** — activabil cu `redis-switch.sh cluster`, nu este folosit operational acum.
|
|
|
|
**Productie (LOCAL — activ)**:
|
|
- Container: `didi-cache` (imagine `redis:7-alpine`)
|
|
- Host intern Docker: `didi-cache:6379`
|
|
- DB: `0`
|
|
- Parola: `REDIS_PASSWORD` din `.env` (default `redis123` in dev)
|
|
- Memorie: 512MB (eviction: allkeys-lru)
|
|
- Retea: `didi-network`; portul `6379` este expus pe host (`0.0.0.0:6379->6379`)
|
|
- Persistenta: RDB + AOF
|
|
|
|
**Fallback HA (cluster RAG — disponibil dar inactiv)**:
|
|
- Host: `10.11.50.100` (HAProxy VIP)
|
|
- Port: `16379`
|
|
- DB: `0`
|
|
- User: `didi`
|
|
- Parola: din `.cluster-credentials.env` (gitignored)
|
|
|
|
**Switch intre LOCAL si cluster**: `backend/services/orchestration-layer/scripts/redis-switch.sh {cluster|local|status}` rescrie fisierele `.env` ale serviciilor (agent-v3, didiFramework) + restart containere. Status curent: `local`.
|
|
|
|
**Helper de conectare folosit de toate serviciile**:
|
|
- agent-v3 -> `src/shared/redis/connection.ts` (`createRedisConnection()`)
|
|
- didiFramework -> `src/config/redis.ts` (`createRedisConnection()`)
|
|
|
|
---
|
|
|
|
## Ce face
|
|
|
|
Stocheaza date temporare si configurari pentru platforma DIDI:
|
|
- Cache framework (parametri tehnici, verdicts, ponderi) -- scrise de didiFramework via sync-redis
|
|
- Configurare HIL Moderation (triage thresholds, brain client config, sensitive topics, roluri) -- scrise de didiFramework
|
|
- Sesiuni analiza (AnalysisSession JSON, TTL 7 zile) -- scrise de agent-v3
|
|
- Rezultate intermediare per etapa (TTL 7 zile) -- scrise de agent-v3
|
|
- Stare coada async (progres workeri) -- scrise de agent-v3
|
|
- Lock-uri concurenta workeri (TTL 30s-5min) -- scrise de agent-v3
|
|
- Chei API extensie browser (cache validare) -- scrise de didiFramework
|
|
- Configurare modele viziune -- scrise de didiFramework
|
|
|
|
**Sync-redis scrie ~51 chei de configurare** (crescut de la 48 dupa adaugarea cheilor HIL Moderation in 2026-05-01). Plus `framework_keys`: 7 chei standard sau 8 daca include `providers`.
|
|
|
|
---
|
|
|
|
## Cine scrie in Redis
|
|
|
|
| Serviciu | Ce scrie | Chei Redis |
|
|
|----------|----------|------------|
|
|
| didiFramework (sync-redis) | Ierarhie tehnici, claims, verdicts, ponderi, surse, provideri | didi:framework:* |
|
|
| didiFramework (sync-redis) | Configurare componente pe etape | didi:config:* |
|
|
| didiFramework (extension-keys) | Cache validare chei API extensie | didi:extension:key:* |
|
|
| agent-v3 (sesiuni) | Sesiune completa JSON | didi:pipeline:{sessionId}:status |
|
|
| agent-v3 (executori) | Rezultate intermediare per etapa | agent:result:{sessionId}:{component}:{stage} |
|
|
| agent-v3 (coada) | Stare procesare async | didi:queue:session:{sessionId} |
|
|
| agent-v3 (workeri) | Lock-uri concurenta | didi:queue:lock:* |
|
|
|
|
## Cine citeste din Redis
|
|
|
|
| Serviciu | Ce citeste | Chei Redis |
|
|
|----------|-----------|------------|
|
|
| agent-v3 (executori) | Configurare framework (tehnici, ponderi, verdicts) | didi:framework:* |
|
|
| agent-v3 (executori) | Modele disponibile, stage assignments, prompturi | didi:config:* |
|
|
| agent-v3 (rute) | Sesiuni pentru polling status | didi:pipeline:{sessionId}:status |
|
|
| agent-v3 (rute) | Rezultate intermediare | agent:result:{sessionId}:* |
|
|
| agent-v3 (pipeline) | Validare cheie extensie | didi:extension:key:* |
|
|
| didiFramework (debug) | Verificare date sincronizate | didi:framework:* |
|
|
|
|
---
|
|
|
|
## Chei Redis principale
|
|
|
|
| Pattern | Scop | TTL | Scris de |
|
|
|---------|------|-----|----------|
|
|
| didi:framework:manifest | Index categorii + timestamp sync | permanent | didiFramework |
|
|
| didi:framework:techniques | Ierarhie completa tehnici (denormalizata) | permanent | didiFramework |
|
|
| didi:framework:claims | Parametri claims (tipuri, statusuri, confidence) | permanent | didiFramework |
|
|
| didi:framework:verdicts | Categorii verdict + risk mappings + severity | permanent | didiFramework |
|
|
| didi:framework:weights | Ponderi componente + scenarii + multiplicatori | permanent | didiFramework |
|
|
| didi:framework:sources | Evaluare surse | permanent | didiFramework |
|
|
| didi:framework:providers | Configurare LLM | permanent | didiFramework |
|
|
| didi:framework:dimensions_compact | Lista compacta dimensiuni (screening) | permanent | didiFramework |
|
|
| didi:config:{component}:v1:* | Config componenta (modele, etape, prompturi) | permanent | didiFramework |
|
|
| didi:config:techniques:v3:stage_assignments | **TIER-NESTED** `{stage: {free: {models}, premium: {models}}}` — chain-uri LLM per tier | permanent | didiFramework |
|
|
| didi:config:ai-tampered:v1:stage_assignments | **TIER-NESTED** stage assignments AI-Tampered | permanent | didiFramework |
|
|
| didi:config:claims:v1:stage_assignments | **TIER-NESTED** stage assignments Claims | permanent | didiFramework |
|
|
| didi:config:source-assessment:v1:stage_assignments | **TIER-NESTED** stage assignments Source Assessment | permanent | didiFramework |
|
|
| didi:config:vision:v1:stage_assignments | **TIER-NESTED** (Etapa 4) stage `image_analysis` — citit de `callVision()` cu tier param | permanent | didiFramework |
|
|
| didi:config:verdict:v1:stage_assignments | **TIER-NESTED** (Etapa 5) stage `verdict_review` — citit de `verdict-explanation.ts loadModels(tier)` | permanent | didiFramework |
|
|
| didi:config:ai-tampered:v1:vision_models | Legacy flat vision config (fallback pentru vision.ts daca `didi:config:vision:v1:stage_assignments` lipseste) | permanent | didiFramework |
|
|
| didi:config:vision:v1:prompts:extraction | Prompt viziune: extragere text din imagini | permanent | didiFramework |
|
|
| didi:config:vision:v1:prompts:video_frames | Prompt viziune: analiza cadre video | permanent | didiFramework |
|
|
| didi:config:vision:v1:prompts:ai_detection | Prompt viziune: detectie AI imagini | permanent | didiFramework |
|
|
| didi:config:pipeline:v1:component_config | Config componente pipeline + video track weights | permanent | didiFramework |
|
|
| didi:config:pipeline:v1:session_config | Config sesiune pipeline | permanent | didiFramework |
|
|
| didi:config:pipeline:v1:verdict_config | Override-uri verdict, synergy, confidence (globale). JSONB sincronizat din PG `bos_parammgmt.component_config WHERE (component_code='pipeline', config_key='verdict_config')`. Contine: `synergy` + override-uri (`false_claims`, `severe_techniques`, `undisclosed_ai`, `untrusted_domain`, `domain_red_flags`) + `confidence` + `confidence_levels`. Editabil din admin dashboard via `/api/verdicts/runtime-config` (GET/PUT/PATCH). | permanent | didiFramework |
|
|
| didi:config:pipeline:v1:input_profiles | **Profiluri verdict per input type (6 profile: ponderi, override-uri, INCONCLUSIVE)** | permanent | didiFramework |
|
|
| didi:config:techniques:v3:scoring_config | Parametri scoring techniques (count_scaler, intensity, severe_threshold) | permanent | didiFramework |
|
|
| didi:config:ai-tampered:v1:scoring_config | Parametri scoring AI (blend_weights, disclosure_impact, thresholds) | permanent | didiFramework |
|
|
| didi:config:claims:v1:scoring_config | Parametri scoring claims (status_weights, unverified behavior) | permanent | didiFramework |
|
|
| didi:config:source-assessment:v1:scoring_config | Parametri scoring source (axis_weights, verdict_thresholds) | permanent | didiFramework |
|
|
| didi:config:source-assessment:v1:available_models | Modele LLM source assessment (union free+premium) | permanent | didiFramework |
|
|
| didi:config:source-assessment:v1:prompts:extraction | Prompt extractie metadata sursa | permanent | didiFramework |
|
|
| didi:config:source-assessment:v1:prompts:evaluation | Prompt evaluare sursa | permanent | didiFramework |
|
|
| didi:config:verdict:v1:available_models | Legacy fallback pentru verdict reviewer (folosit daca `stage_assignments` lipseste) | permanent | didiFramework |
|
|
|
|
### HIL Moderation (synced from PG by sync-redis)
|
|
|
|
Adaugat in 2026-05-01. Citit de agent-v3 in `triage.ts` (cache local 60s) si `brain/client.ts` pentru configurare live.
|
|
|
|
| Pattern | Scop | TTL | Scris de |
|
|
|---------|------|-----|----------|
|
|
| didi:config:moderation:v1:settings | Single-row config: triage thresholds + brain client config (brain_enabled, brain_url, lookup/write timeouts, brain_confidence_min_silver, brain_semantic_threshold, brain_per_component) | permanent | didiFramework |
|
|
| didi:config:moderation:v1:sensitive_topics | Active topics list (elections, health, war, covid, climate) | permanent | didiFramework |
|
|
| didi:config:moderation:v1:roles | Keycloak role -> permissions (moderator, senior_moderator) cu toggle flags | permanent | didiFramework |
|
|
|
|
**Nota structura tier-nested** (aplicabila tuturor cheilor `stage_assignments`):
|
|
```json
|
|
{
|
|
"techniques_screening": {
|
|
"free": { "models": [{"order":1,"model_key":"qwen35:Qwen3.5-397B-A17B",...}, ...] },
|
|
"premium": { "models": [{"order":1,"model_key":"openrouter:google/gemini-3-flash-preview",...}, ...] }
|
|
},
|
|
"techniques_deep": { "free": {...}, "premium": {...} }
|
|
}
|
|
```
|
|
|
|
Sync-redis scrie cheia intr-un singur SET (atomic). agent-v3 citeste cheia o singura data, apoi rezolva tier-ul local cu `stages[stageCode][tier] || stages[stageCode].free`.
|
|
| didi:pipeline:{sessionId}:status | Status executie pipeline (JSON: PipelineStatus) | 7 zile | agent-v3 |
|
|
| didi:pipeline:{sessionId}:{component} | Rezultat componenta (JSON: TechniquesResult etc.) | 7 zile | agent-v3 |
|
|
| didi:pipeline:{sessionId}:verdict | Rezultat verdict final (JSON: VerdictResult) | 7 zile | agent-v3 |
|
|
| didi:pipeline:history:entry:{sessionId} | Date intrare sesiune (input + metadata) | 7 zile | agent-v3 |
|
|
| didi:pipeline:history:user:{userId} | Sorted set istoric utilizator (score=timestamp) | 7 zile | agent-v3 |
|
|
| agent:result:{sessionId}:{comp}:{stage} | Rezultat intermediar etapa | 7 zile | agent-v3 |
|
|
| didi:queue:session:{sessionId} | Stare coada async | 24 ore | agent-v3 |
|
|
| didi:queue:lock:{sessionId}:{comp} | Lock worker per componenta | 5 min | agent-v3 |
|
|
| didi:queue:aggregator:{sessionId} | Lock agregator verdict | 30s | agent-v3 |
|
|
| didi:extension:key:{key} | Cache cheie API extensie | permanent | didiFramework |
|
|
|
|
---
|
|
|
|
## Configurare
|
|
|
|
### redis.conf
|
|
|
|
- Bind: 0.0.0.0 (acces din Docker network)
|
|
- Protected mode: activat
|
|
- Max memorie: 512MB
|
|
- Eviction: allkeys-lru (sterge cele mai vechi chei cand se umple)
|
|
- Persistenta RDB: snapshot la 60s/10000 keys, 300s/10 keys, 900s/1 key
|
|
- Persistenta AOF: activat, sync everysec
|
|
- Max clienti: 10000
|
|
- 16 baze de date
|
|
- Slow log: 10ms threshold
|
|
- Parola: obligatorie (requirepass)
|
|
|
|
### Pornire in productie
|
|
|
|
Nota: didi-cache NU este definit in data-layer/docker-compose.yml. Containerul Redis este definit in `production/docker-compose.yml` (alaturi de Kong si Keycloak).
|
|
|
|
```yaml
|
|
# din production/docker-compose.yml
|
|
didi-cache:
|
|
image: redis:7-alpine
|
|
command: redis-server --requirepass ${REDIS_PASSWORD} --save 60 1 --save 300 10
|
|
volumes:
|
|
- didi-cache-data:/data
|
|
networks:
|
|
- didi-network
|
|
healthcheck:
|
|
test: redis-cli -a ${REDIS_PASSWORD} ping
|
|
```
|
|
|
|
Nota: redis.conf din directorul didiCache nu este montat in container in productie. Containerul foloseste parametrii din command line.
|
|
|
|
### Conexiune
|
|
|
|
**Productie (LOCAL — activ)**:
|
|
- Host: `didi-cache` (Docker DNS pe `didi-network`)
|
|
- Port: `6379` (expus si pe host: `0.0.0.0:6379->6379`)
|
|
- DB: `0`
|
|
- Parola: `redis123` (din `.env`, `REDIS_PASSWORD`)
|
|
|
|
**Fallback HA (cluster RAG — disponibil dar inactiv, doar dupa `redis-switch.sh cluster`)**:
|
|
- Host: `10.11.50.100` (HAProxy VIP rag01/02/03)
|
|
- Port: `16379`
|
|
- DB: `0`
|
|
- Username: `didi`
|
|
- Parola: din `.cluster-credentials.env` (gitignored)
|
|
|
|
### Administrare
|
|
|
|
- Conexiune directa: `redis-cli -h didi-cache -a redis123` (sau `127.0.0.1:6379` de pe host).
|
|
|
|
---
|
|
|
|
## Fisiere in directorul didiCache
|
|
|
|
```
|
|
redis.conf -- Configurare Redis completa (183 linii, nu e montata in productie)
|
|
.env.example -- Template variabile de mediu
|
|
.gitignore -- Exclude .env, data/, *.rdb, *.aof
|
|
README.md -- Documentatie
|
|
```
|
|
|
|
Zero cod custom. Zero module Redis. Zero scripturi. Doar configurare.
|
|
|
|
---
|
|
|
|
## Ce NU face
|
|
|
|
- Nu e Redis Cluster (instanta singulara)
|
|
- Nu e Redis Sentinel (fara failover automat)
|
|
- Nu are module custom
|
|
- Nu are replicare
|
|
- Nu are ACL (doar autentificare cu parola)
|
|
- Nu proceseaza nimic -- doar stocheaza si serveste date scrise de alte servicii
|