livrare lot 2
This commit is contained in:
commit
8ecc78e729
763 changed files with 164593 additions and 0 deletions
870
backend/services/orchestration-layer/didiFramework/INDEX.md
Normal file
870
backend/services/orchestration-layer/didiFramework/INDEX.md
Normal file
|
|
@ -0,0 +1,870 @@
|
|||
# didiFramework - Index
|
||||
|
||||
Backend CRUD pentru managementul tuturor parametrilor platformei DIDI. Stocheaza configuratia in PostgreSQL si o sincronizeaza in Redis pentru acces rapid de catre agent-v3. Gestioneaza si utilizatorii, creditele, abonamentele si fisierele.
|
||||
|
||||
**Port**: 3005
|
||||
**Framework**: Express 4 + TypeScript
|
||||
**Container**: didi-framework
|
||||
**Schema principala**: bos_parammgmt
|
||||
|
||||
---
|
||||
|
||||
## Ce face serviciul
|
||||
|
||||
1. **CRUD parametri** -- dimensiuni, tehnici, indicatori, reguli, verdicts, ponderi, surse, claims
|
||||
2. **Sincronizare Redis** -- incarca ierarhia completa in Redis ca agent-v3 sa o citeasca instant
|
||||
3. **Managementul utilizatorilor** -- integrare Keycloak, auto-inregistrare, credite, abonamente
|
||||
4. **Stocare fisiere** -- upload/download via MinIO, bucket-uri per utilizator
|
||||
5. **Istoric analize** -- acces la rezultatele salvate in bos_analysis
|
||||
6. **Configurare LLM** -- provideri, modele, assignments pe componente
|
||||
7. **API keys extensie browser** -- CRUD chei API pentru extensia Chrome
|
||||
|
||||
---
|
||||
|
||||
## Structura fisierelor
|
||||
|
||||
```
|
||||
src/
|
||||
server.ts -- Express app, montare rute, middleware, error handling
|
||||
config/
|
||||
database.ts -- Pool PostgreSQL, query(), queryOne(), transaction()
|
||||
minio.ts -- Client MinIO, operatii bucket, upload/download, bucket-uri user
|
||||
jwt-verify.ts -- jwtVerifyGate(): middleware global RS256 vs Keycloak JWKS (Modul 7 Auth)
|
||||
types/
|
||||
index.ts -- Interfete TypeScript pentru toate entitatile
|
||||
utils/
|
||||
crud-factory.ts -- Generator automat de rute CRUD (GET/POST/PUT/DELETE)
|
||||
dependency-checker.ts -- Verificare dependente inainte de stergere (safe delete)
|
||||
routes/
|
||||
dimensions.ts -- CRUD dimensiuni (nivel 1 ierarhie tehnici)
|
||||
subdimensions.ts -- CRUD subdimensiuni (nivel 2)
|
||||
techniques.ts -- CRUD tehnici (nivel 3, suporta cascade delete)
|
||||
indicators.ts -- CRUD indicatori tehnica (leaf, bulk create)
|
||||
validation-rules.ts -- CRUD reguli validare tehnica (leaf, bulk create)
|
||||
verdicts.ts -- CRUD categorii verdict + risk mappings + severity
|
||||
weights.ts -- CRUD ponderi componente + scenarii + multiplicatori
|
||||
platforms.ts -- CRUD platforme social media
|
||||
sources.ts -- CRUD tipuri sursa
|
||||
source-assessment.ts -- CRUD platf. modifiers, credib. sursa, varsta domeniu, risk, red flags, autori
|
||||
claims.ts -- CRUD status claim, tip claim, confidence, interpretare
|
||||
sync-redis.ts -- Sincronizare framework PostgreSQL -> Redis
|
||||
sync-analysis.ts -- Sincronizare rezultate analiza Redis -> PostgreSQL (legacy)
|
||||
auth.ts -- Autentificare Keycloak, profil, credite, auto-inregistrare
|
||||
admin.ts -- Management admin utilizatori + abonamente
|
||||
history.ts -- Istoric analize (list paginat + detaliu)
|
||||
subscriptions.ts -- Info abonament + credite ramase
|
||||
uploads.ts -- Upload/download fisiere MinIO
|
||||
providers.ts -- Configurare provideri LLM, modele, assignments, API keys
|
||||
extension-keys.ts -- CRUD chei API extensie browser
|
||||
prompts.ts -- Servire fisiere prompt (markdown) pentru pipeline
|
||||
waitlist.ts -- Waitlist public (signup email)
|
||||
overview.ts -- Statistici framework + health checks
|
||||
input-profiles.ts -- CRUD profiluri verdict per input type + scoring-config GET/PUT
|
||||
moderation-config.ts -- CRUD single-row config HIL triage + brain client
|
||||
sensitive-topics.ts -- CRUD topics care declanseaza HIL review
|
||||
moderation-roles.ts -- CRUD Keycloak role -> permisiuni HIL
|
||||
skills.ts -- Catalog resurse AI (Modul 1): analysis_components + extractor_skills (probe live platforma Lot 1) + code_jobs
|
||||
notifications.ts -- Health/test SMTP + trigger manual credit-reset (email notifications)
|
||||
webhooks/stripe.ts -- Webhook Stripe (raw body) — plati/abonamente (suplimentar)
|
||||
admin/social.ts -- Postare social media (Facebook DESI 6) din admin-dashboard (suplimentar)
|
||||
data/ -- Fisiere date (seed, export)
|
||||
scripts/ -- Scripturi utilitare
|
||||
openapi.yaml -- Spec OpenAPI la radacina (documenteaza API-ul; Modulele 5-7)
|
||||
sql/
|
||||
migrations/ -- Migratii SQL schema
|
||||
```
|
||||
|
||||
Nota: `input-profiles.ts` este montat sub DOUA prefixe — `/api/input-profiles` si aliasul `/api/pipelines` (Modul 1: definitiile de pipeline = `input_type_profile`).
|
||||
|
||||
---
|
||||
|
||||
## Ierarhia de date (tehnici de manipulare)
|
||||
|
||||
```
|
||||
dimension
|
||||
└── subdimension
|
||||
└── technique
|
||||
├── technique_indicator (leaf)
|
||||
└── technique_validation_rule (leaf)
|
||||
```
|
||||
|
||||
Stergerea unui parinte este blocata daca are copii (safe delete).
|
||||
Exceptie: `DELETE /techniques/:id?cascade=true` sterge copiii inainte.
|
||||
|
||||
---
|
||||
|
||||
## API - Toate endpoint-urile
|
||||
|
||||
### Health si documentatie
|
||||
|
||||
| Metoda | Path | Ce face | Logica in fisier |
|
||||
|--------|------|---------|------------------|
|
||||
| GET | / | Documentatie API completa | server.ts |
|
||||
| GET | /health | Health check simplu | server.ts |
|
||||
| GET | /health/all | Health PostgreSQL + MinIO | server.ts |
|
||||
|
||||
### Overview (`/api/overview`)
|
||||
|
||||
| Metoda | Path | Ce face | Logica in fisier |
|
||||
|--------|------|---------|------------------|
|
||||
| GET | /stats | Numar dimensiuni, tehnici, verdicte, platforme | routes/overview.ts |
|
||||
| GET | /health | Health check baza de date | routes/overview.ts |
|
||||
| GET | /docker-health | Status container Docker | routes/overview.ts |
|
||||
|
||||
### Dimensiuni (`/api/dimensions`)
|
||||
|
||||
| Metoda | Path | Ce face | Logica in fisier |
|
||||
|--------|------|---------|------------------|
|
||||
| GET | / | Lista toate dimensiunile | routes/dimensions.ts |
|
||||
| GET | /with-counts | Lista cu numar subdimensiuni si tehnici | routes/dimensions.ts |
|
||||
| GET | /:id | O singura dimensiune | routes/dimensions.ts |
|
||||
| GET | /:id/dependencies | Verifica daca are subdimensiuni | routes/dimensions.ts |
|
||||
| POST | / | Creeaza dimensiune | routes/dimensions.ts |
|
||||
| PUT | /:id | Actualizeaza dimensiune | routes/dimensions.ts |
|
||||
| DELETE | /:id | Sterge (blocat daca are copii) | routes/dimensions.ts |
|
||||
|
||||
### Subdimensiuni (`/api/subdimensions`)
|
||||
|
||||
| Metoda | Path | Ce face | Logica in fisier |
|
||||
|--------|------|---------|------------------|
|
||||
| GET | / | Lista toate subdimensiunile | routes/subdimensions.ts |
|
||||
| GET | /with-counts | Lista cu numar tehnici | routes/subdimensions.ts |
|
||||
| GET | /by-dimension/:dimensionId | Filtrate dupa dimensiune parinte | routes/subdimensions.ts |
|
||||
| GET | /:id | O singura subdimensiune | routes/subdimensions.ts |
|
||||
| GET | /:id/dependencies | Verifica daca are tehnici | routes/subdimensions.ts |
|
||||
| POST | / | Creeaza subdimensiune | routes/subdimensions.ts |
|
||||
| PUT | /:id | Actualizeaza subdimensiune | routes/subdimensions.ts |
|
||||
| DELETE | /:id | Sterge (blocat daca are copii) | routes/subdimensions.ts |
|
||||
|
||||
Nota: Coloana din DB se numeste `subdmiension_name` (typo). Codul corecteaza in API ca `subdimension_name`.
|
||||
|
||||
### Tehnici (`/api/techniques`)
|
||||
|
||||
| Metoda | Path | Ce face | Logica in fisier |
|
||||
|--------|------|---------|------------------|
|
||||
| GET | / | Lista toate tehnicile | routes/techniques.ts |
|
||||
| GET | /with-hierarchy | Lista cu dimensiune + subdimensiune + contor indicatori/reguli | routes/techniques.ts |
|
||||
| GET | /by-subdimension/:subdimensionId | Filtrate dupa subdimensiune | routes/techniques.ts |
|
||||
| GET | /:id | O singura tehnica | routes/techniques.ts |
|
||||
| GET | /:id/dependencies | Verifica indicatori si reguli | routes/techniques.ts |
|
||||
| POST | / | Creeaza tehnica | routes/techniques.ts |
|
||||
| PUT | /:id | Actualizeaza tehnica | routes/techniques.ts |
|
||||
| DELETE | /:id | Sterge (blocat daca are copii) | routes/techniques.ts |
|
||||
| DELETE | /:id?cascade=true | Sterge cu toti copiii | routes/techniques.ts |
|
||||
|
||||
### Indicatori (`/api/indicators`)
|
||||
|
||||
| Metoda | Path | Ce face | Logica in fisier |
|
||||
|--------|------|---------|------------------|
|
||||
| GET | / | Lista toti indicatorii | routes/indicators.ts |
|
||||
| GET | /by-technique/:id | Filtrati dupa tehnica | routes/indicators.ts |
|
||||
| GET | /missing | Tehnici fara indicatori | routes/indicators.ts |
|
||||
| GET | /stats | Statistici acoperire indicatori | routes/indicators.ts |
|
||||
| POST | / | Creeaza indicator | routes/indicators.ts |
|
||||
| POST | /bulk | Creeaza mai multi indicatori | routes/indicators.ts |
|
||||
| POST | /bulk-for-technique/:id | Creeaza indicatori pentru o tehnica | routes/indicators.ts |
|
||||
| PUT | /:techniqueId/:indicatorId | Actualizeaza indicator | routes/indicators.ts |
|
||||
| DELETE | /:techniqueId/:indicatorId | Sterge indicator | routes/indicators.ts |
|
||||
| DELETE | /by-technique/:id | Sterge toti indicatorii unei tehnici | routes/indicators.ts |
|
||||
|
||||
### Reguli validare (`/api/validation-rules`)
|
||||
|
||||
| Metoda | Path | Ce face | Logica in fisier |
|
||||
|--------|------|---------|------------------|
|
||||
| GET | / | Lista toate regulile | routes/validation-rules.ts |
|
||||
| GET | /with-techniques | Lista cu numele tehnicii | routes/validation-rules.ts |
|
||||
| GET | /by-technique/:techniqueId | Filtrate dupa tehnica | routes/validation-rules.ts |
|
||||
| GET | /stats | Statistici acoperire reguli | routes/validation-rules.ts |
|
||||
| GET | /:id | O singura regula | routes/validation-rules.ts |
|
||||
| POST | / | Creeaza regula | routes/validation-rules.ts |
|
||||
| POST | /bulk-for-technique/:techniqueId | Creeaza reguli pentru o tehnica | routes/validation-rules.ts |
|
||||
| PUT | /:id | Actualizeaza regula | routes/validation-rules.ts |
|
||||
| DELETE | /:id | Sterge regula | routes/validation-rules.ts |
|
||||
| DELETE | /by-technique/:techniqueId | Sterge toate regulile unei tehnici | routes/validation-rules.ts |
|
||||
|
||||
### Verdicts (`/api/verdicts`)
|
||||
|
||||
| Metoda | Path | Ce face | Logica in fisier |
|
||||
|--------|------|---------|------------------|
|
||||
| GET | /all | Toate cele 3 tipuri combinate | routes/verdicts.ts |
|
||||
| GET | /categories | Categorii verdict (RELIABLE, MIXED, DISINFO, etc.) | routes/verdicts.ts |
|
||||
| GET | /categories/:id | O categorie | routes/verdicts.ts |
|
||||
| POST | /categories | Creeaza categorie | routes/verdicts.ts |
|
||||
| PUT | /categories/:id | Actualizeaza categorie | routes/verdicts.ts |
|
||||
| DELETE | /categories/:id | Sterge categorie | routes/verdicts.ts |
|
||||
| GET | /risk | Risk mappings (LOW, MODERATE, HIGH, CRITICAL) | routes/verdicts.ts |
|
||||
| GET | /risk/:id | Un risk mapping | routes/verdicts.ts |
|
||||
| POST | /risk | Creeaza risk mapping | routes/verdicts.ts |
|
||||
| PUT | /risk/:id | Actualizeaza risk mapping | routes/verdicts.ts |
|
||||
| DELETE | /risk/:id | Sterge risk mapping | routes/verdicts.ts |
|
||||
| GET | /severity | Severity assessments | routes/verdicts.ts |
|
||||
| POST,PUT,DELETE | /severity/... | CRUD severity | routes/verdicts.ts |
|
||||
| GET | /runtime-config | Citeste jsonb din `component_config` (component_code='pipeline', config_key='verdict_config') | routes/verdicts.ts |
|
||||
| PUT | /runtime-config | Update full body (synergy + overrides + confidence + confidence_levels) cu validare chei obligatorii | routes/verdicts.ts |
|
||||
| PATCH | /runtime-config/:section | Update partial pe sectiune (synergy, overrides, confidence, confidence_levels, false_claims, severe_techniques, undisclosed_ai, untrusted_domain, domain_red_flags) | routes/verdicts.ts |
|
||||
|
||||
### Ponderi (`/api/weights`)
|
||||
|
||||
| Metoda | Path | Ce face | Logica in fisier |
|
||||
|--------|------|---------|------------------|
|
||||
| GET | /all | Toate cele 3 tipuri combinate | routes/weights.ts |
|
||||
| GET | /components | Ponderi componente (techniques 35%, claims 25%, etc.) | routes/weights.ts |
|
||||
| POST,PUT,DELETE | /components/... | CRUD ponderi | routes/weights.ts |
|
||||
| GET | /scenarios | Scenarii ponderi (combinatii per context) | routes/weights.ts |
|
||||
| POST,PUT,DELETE | /scenarios/... | CRUD scenarii | routes/weights.ts |
|
||||
| GET | /multipliers | Multiplicatori (topic, temporal, reach) | routes/weights.ts |
|
||||
| GET | /multipliers/type/:type | Multiplicatori filtrati dupa tip | routes/weights.ts |
|
||||
| POST,PUT,DELETE | /multipliers/... | CRUD multiplicatori | routes/weights.ts |
|
||||
|
||||
### Platforme (`/api/platforms`)
|
||||
|
||||
| Metoda | Path | Ce face | Logica in fisier |
|
||||
|--------|------|---------|------------------|
|
||||
| GET | / | Lista platforme cu modifier info | routes/platforms.ts |
|
||||
| GET | /:id | O platforma | routes/platforms.ts |
|
||||
| POST | / | Creeaza platforma | routes/platforms.ts |
|
||||
| PUT | /:id | Actualizeaza platforma | routes/platforms.ts |
|
||||
| DELETE | /:id | Sterge platforma | routes/platforms.ts |
|
||||
|
||||
### Surse (`/api/sources`)
|
||||
|
||||
| Metoda | Path | Ce face | Logica in fisier |
|
||||
|--------|------|---------|------------------|
|
||||
| GET | / | Lista tipuri sursa cu numar utilizari | routes/sources.ts |
|
||||
| GET | /:id | Un tip sursa | routes/sources.ts |
|
||||
| GET | /:id/dependencies | Verifica domain_attribute copii | routes/sources.ts |
|
||||
| POST | / | Creeaza tip sursa | routes/sources.ts |
|
||||
| PUT | /:id | Actualizeaza | routes/sources.ts |
|
||||
| DELETE | /:id | Sterge (blocat daca are copii) | routes/sources.ts |
|
||||
|
||||
### Evaluare surse (`/api/source-assessment`)
|
||||
|
||||
7 sub-resurse, fiecare cu CRUD complet:
|
||||
|
||||
| Sub-resursa | Tabela | Are copii in |
|
||||
|-------------|--------|--------------|
|
||||
| /platform-modifiers | platform_modifier | platform |
|
||||
| /source-credibility | source_credibility | domain_attribute |
|
||||
| /domain-age-scores | domain_age_score | domain_attribute |
|
||||
| /domain-risk-levels | domain_risk_level | domain_attribute |
|
||||
| /domain-red-flags | domain_red_flag | domain_attribute |
|
||||
| /author-classifications | author_classification | author |
|
||||
| /author-credibility | author_credibility | author |
|
||||
|
||||
Fiecare are: GET /, GET /:id, POST /, PUT /:id, DELETE /:id
|
||||
Doar /platform-modifiers are si GET /:id/dependencies (singura sub-resursa cu copii directi in platform).
|
||||
Plus: GET /source-assessment-ranges -- tabel lookup cu range-uri evaluare sursa (leaf, read-only).
|
||||
Plus: GET /source-assessment/all -- combina toate cele 7 tipuri.
|
||||
Logica: routes/source-assessment.ts
|
||||
|
||||
### Claims (`/api/claims`)
|
||||
|
||||
| Metoda | Path | Ce face | Logica in fisier |
|
||||
|--------|------|---------|------------------|
|
||||
| GET | /all | Toate cele 4 tipuri combinate | routes/claims.ts |
|
||||
| GET | /status | Statusuri claim (VT, LT, UV, LF, VF, OP, NV) | routes/claims.ts |
|
||||
| POST,PUT,DELETE | /status/... | CRUD statusuri | routes/claims.ts |
|
||||
| GET | /types | Tipuri claim (EF, VF, RE, SC, QA, CC, PC, OF, VC) | routes/claims.ts |
|
||||
| POST,PUT,DELETE | /types/... | CRUD tipuri | routes/claims.ts |
|
||||
| GET | /confidence | Nivele de incredere | routes/claims.ts |
|
||||
| POST,PUT,DELETE | /confidence/... | CRUD confidence | routes/claims.ts |
|
||||
| GET | /interpretation | Interpretari concordanta surse | routes/claims.ts |
|
||||
| POST,PUT,DELETE | /interpretation/... | CRUD interpretari | routes/claims.ts |
|
||||
|
||||
### Sincronizare Redis (`/api/sync-redis`)
|
||||
|
||||
| Metoda | Path | Ce face | Logica in fisier |
|
||||
|--------|------|---------|------------------|
|
||||
| POST | / | Sincronizeaza tot framework-ul din PG in Redis | routes/sync-redis.ts |
|
||||
| GET | /status | Cand s-a facut ultima sincronizare | routes/sync-redis.ts |
|
||||
| GET | /data/:category | Citeste o categorie din Redis (debug) | routes/sync-redis.ts |
|
||||
|
||||
Categorii sincronizate: techniques (ierarhie completa), sources, claims, verdicts, weights, providers.
|
||||
|
||||
Tabele suplimentare citite la sync (schema bos_parammgmt):
|
||||
- `component_stage_assignment` -- assignment model LLM per componenta + etapa **+ tier** (free/premium); fiecare tier are propriul fallback chain
|
||||
- `component_prompt` -- system prompt + user template per componenta + etapa
|
||||
- `component_config` -- configurari JSONB per componenta (config_key / config_value)
|
||||
|
||||
**Structura tier-nested a `stage_assignments` in Redis** (dupa migration 006 care a adaugat coloana `tier`):
|
||||
```json
|
||||
{
|
||||
"techniques_screening": {
|
||||
"free": { "stage": "...", "description": "...", "models": [{order:1, model_key:"qwen35:Qwen3.5-397B-A17B", ...}, ...] },
|
||||
"premium": { "stage": "...", "description": "...", "models": [{order:1, model_key:"openrouter:google/gemini-3-flash-preview", ...}, ...] }
|
||||
},
|
||||
"techniques_deep": { "free": {...}, "premium": {...} }
|
||||
}
|
||||
```
|
||||
|
||||
sync-redis.ts `fetchStageAssignments()` citeste PG cu `ORDER BY component_code, stage_code, tier, fallback_order` si construieste obiectul nested pe tier. Grouping logic: `result[component_code][stage_code][tier].models.push(...)`.
|
||||
|
||||
Componente acoperite (`VERSION_MAP`): `techniques` (v3), `ai-tampered` (v1), `claims` (v1), `pipeline` (v1), `vision` (v1), `source-assessment` (v1), `verdict` (v1). Fiecare componenta poate avea rows cu `component_code` dedicat (inclusiv `vision` + `verdict` care au fost adaugate in Etapele 4-5).
|
||||
|
||||
Chei Redis scrise:
|
||||
```
|
||||
didi:framework:manifest -- index categorii + timestamp sync
|
||||
didi:framework:techniques -- ierarhie dimensiuni -> subdimensiuni -> tehnici -> indicatori/reguli
|
||||
didi:framework:sources -- evaluare surse
|
||||
didi:framework:claims -- parametri claims
|
||||
didi:framework:verdicts -- categorii verdict + risk mappings
|
||||
didi:framework:weights -- ponderi + scenarii + multiplicatori
|
||||
didi:framework:providers -- configurare LLM (optional)
|
||||
didi:framework:dimensions_compact -- lista compacta dimensiuni (pentru screening)
|
||||
didi:config:<component>:<version>:stage_assignments -- TIER-NESTED assignments per stage per tier
|
||||
didi:config:<component>:<version>:available_models -- modele unice din toate etapele si tierele (union)
|
||||
didi:config:<component>:<version>:prompts:<stage> -- prompturi per etapa
|
||||
didi:config:<component>:<version>:<config_key> -- configurari JSONB
|
||||
didi:config:moderation:v1:settings -- single-row din moderation_config
|
||||
didi:config:moderation:v1:sensitive_topics -- topics active, ordonate
|
||||
didi:config:moderation:v1:roles -- toate rolurile cu permisiuni
|
||||
```
|
||||
|
||||
Total `config_keys` count = **51** (era 48 inainte de phase 1.2). Sync-redis e idempotent: blocul moderation e wrapped in try/catch -- daca migration 011 nu e aplicat, sare silent (graceful degradation).
|
||||
|
||||
### Sincronizare analiza (`/api/sync-analysis`) -- LEGACY
|
||||
|
||||
| Metoda | Path | Ce face | Logica in fisier |
|
||||
|--------|------|---------|------------------|
|
||||
| POST | /:sessionId | Muta o sesiune din Redis in PostgreSQL | routes/sync-analysis.ts |
|
||||
| POST | /batch | Muta mai multe sesiuni | routes/sync-analysis.ts |
|
||||
| GET | /pending | Lista sesiuni completate in Redis | routes/sync-analysis.ts |
|
||||
| GET | /stats | Statistici analize | routes/sync-analysis.ts |
|
||||
|
||||
Nota: agent-v3 scrie acum direct in PG prin PersistService. Aceste rute sunt legacy.
|
||||
|
||||
### Autentificare (`/api/auth`)
|
||||
|
||||
| Metoda | Path | Ce face | Logica in fisier |
|
||||
|--------|------|---------|------------------|
|
||||
| GET | /me | Profil utilizator curent (auto-creeaza daca nu exista) | routes/auth.ts |
|
||||
| POST | /register | Inregistrare utilizator in Keycloak + PG | routes/auth.ts |
|
||||
| GET | /credits | Credite ramase | routes/auth.ts |
|
||||
| POST | /use-credit | Deduce credite (necesita JWT) | routes/auth.ts |
|
||||
| PUT | /profile | Actualizeaza profil | routes/auth.ts |
|
||||
| GET | /verify-email | Pagina confirmare verificare email (HTML, query param: key) | routes/auth.ts |
|
||||
| POST | /verify-email | Executa verificare email in Keycloak (body: key) | routes/auth.ts |
|
||||
|
||||
Auto-inregistrare la GET /me:
|
||||
- Creeaza person + address + persoana_fizica + internet_user + user_credential + subscription + contact
|
||||
- Plan default: Free, 100 credite, 1GB stocare
|
||||
- Creeaza bucket MinIO: user-{internetUserId}
|
||||
|
||||
Schema PG: bos_sysadmin (user_credential, internet_user, subscription) + bos_subscriber (persoana_fizica, contact)
|
||||
|
||||
### Autentificare interna (`/api/auth/internal`)
|
||||
|
||||
| Metoda | Path | Ce face | Logica in fisier |
|
||||
|--------|------|---------|------------------|
|
||||
| POST | /check-credits | Verificare credite + **returneaza planType** pentru tier routing | routes/auth.ts |
|
||||
| POST | /deduct-credits | Deducere credite (body: keycloak_id, media_type, session_id) | routes/auth.ts |
|
||||
| POST | /get-bucket-info | Info bucket MinIO pentru upload (body: keycloak_id, mime_type) | routes/auth.ts |
|
||||
|
||||
**check-credits response** include `planType` (1-6) pe care agent-v3 il foloseste pentru a deriva `searchTier` (`plan_type 1-3 = free`, `plan_type 4-6 = premium`):
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"data": {
|
||||
"hasEnoughCredits": true,
|
||||
"creditsRemained": 600,
|
||||
"creditCost": 1,
|
||||
"planName": "Pro/Protector",
|
||||
"planType": 4,
|
||||
"mediaType": "text"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Acesta e punctul unic de adevar pentru tier: agent-v3 nu determina tier-ul din JWT sau body, ci il primeste de aici. Asta previne escaladarea de privilegii (un utilizator nu poate trimite `plan_type: 6` in request ca sa primeasca modele premium).
|
||||
|
||||
### Admin (`/api/admin`)
|
||||
|
||||
| Metoda | Path | Ce face | Logica in fisier |
|
||||
|--------|------|---------|------------------|
|
||||
| GET | /users | Lista utilizatori paginata (Keycloak + PG) | routes/admin.ts |
|
||||
| GET | /users/:id | Detalii utilizator | routes/admin.ts |
|
||||
| POST | /users/sync | Sincronizeaza utilizator din Keycloak in PG (body: keycloakId) | routes/admin.ts |
|
||||
| PUT | /users/:id | Actualizeaza utilizator | routes/admin.ts |
|
||||
| PUT | /users/:id/email-verified | Seteaza emailVerified in Keycloak (body: emailVerified) | routes/admin.ts |
|
||||
| DELETE | /users/:id | Sterge utilizator (Keycloak + PG + MinIO bucket) | routes/admin.ts |
|
||||
| PUT | /users/:id/subscription | Schimba abonament | routes/admin.ts |
|
||||
| GET | /plans | Lista planuri abonament | routes/admin.ts |
|
||||
| GET | /plans/:id | Detalii plan abonament | routes/admin.ts |
|
||||
| PUT | /plans/:id | Actualizeaza plan abonament | routes/admin.ts |
|
||||
|
||||
### Istoric analize (`/api/history`)
|
||||
|
||||
| Metoda | Path | Ce face | Logica in fisier |
|
||||
|--------|------|---------|------------------|
|
||||
| GET | / | Istoric utilizator paginat (necesita ?user_id=) | routes/history.ts |
|
||||
| GET | /admin | Istoric admin (toti utilizatorii, filtre) | routes/history.ts |
|
||||
| GET | /:sessionId | Detaliu complet analiza (flat canonical types) | routes/history.ts |
|
||||
| DELETE | /:sessionId | Sterge analiza (necesita ?user_id=, valideaza ownership) | routes/history.ts |
|
||||
|
||||
Filtre admin: user_id, search, status, risk_level, from_date, to_date
|
||||
Filtre user: user_id (obligatoriu), page, limit
|
||||
Lista light: fara JSONB-uri grele, doar scoruri sumare
|
||||
Schema PG: bos_analysis
|
||||
|
||||
### Abonamente (`/api/subscriptions`)
|
||||
|
||||
| Metoda | Path | Ce face | Logica in fisier |
|
||||
|--------|------|---------|------------------|
|
||||
| GET | /usage | Credite ramase, plan, statistici utilizare | routes/subscriptions.ts |
|
||||
| GET | /plans | Lista planuri disponibile | routes/subscriptions.ts |
|
||||
|
||||
### Fisiere (`/api/uploads`)
|
||||
|
||||
| Metoda | Path | Ce face | Logica in fisier |
|
||||
|--------|------|---------|------------------|
|
||||
| GET | /health | Health MinIO | routes/uploads.ts |
|
||||
| POST | / | Upload fisier singular (multipart, max 500MB video) | routes/uploads.ts |
|
||||
| POST | /multipart | Upload fisiere multiple (max 10, camp: files) | routes/uploads.ts |
|
||||
| GET | /:fileId | Download fisier | routes/uploads.ts |
|
||||
| GET | /:fileId/url | URL presemnat download | routes/uploads.ts |
|
||||
| DELETE | /:fileId | Sterge fisier | routes/uploads.ts |
|
||||
| GET | / | Lista fisiere | routes/uploads.ts |
|
||||
| GET | /buckets/stats | Statistici bucket-uri | routes/uploads.ts |
|
||||
|
||||
Limite: imagine 20MB, audio 100MB, video 500MB, text 10MB, document 50MB.
|
||||
Bucket-uri: uploads, image-files, audio-files, video-files, text-files, document-files, pipeline-artifacts.
|
||||
|
||||
### Provideri LLM (`/api/providers`)
|
||||
|
||||
| Metoda | Path | Ce face | Logica in fisier |
|
||||
|--------|------|---------|------------------|
|
||||
| GET | /all | Toate cele 4 tipuri combinate | routes/providers.ts |
|
||||
| GET | /configs | Lista provideri (OpenRouter, Groq, Qwen, etc.) | routes/providers.ts |
|
||||
| GET,POST,PUT,DELETE | /configs/... | CRUD provideri | routes/providers.ts |
|
||||
| GET | /models | Lista modele LLM (include input_cost_per_1m, output_cost_per_1m pentru cost calc) | routes/providers.ts |
|
||||
| GET,POST,PUT,DELETE | /models/... | CRUD modele | routes/providers.ts |
|
||||
| GET | /assignments | Assignments componenta -> model, **include tier**; accepta `?tier=free|premium` pentru filtrare | routes/providers.ts |
|
||||
| GET,POST,PUT,DELETE | /assignments/... | CRUD assignments. POST/PUT accepta campul `tier` in body | routes/providers.ts |
|
||||
| GET | /keys | Lista API keys (mascate) | routes/providers.ts |
|
||||
| GET,POST,PUT,DELETE | /keys/... | CRUD API keys | routes/providers.ts |
|
||||
| POST | /test/:providerId | Test conexiune provider | routes/providers.ts |
|
||||
| GET | /prompts | Prompturi componente (filtre: ?component=X&stage=Y) | routes/providers.ts |
|
||||
| GET | /prompts/:id | Detalii prompt | routes/providers.ts |
|
||||
| POST | /prompts | Creeaza prompt (component_code, stage_code, system_prompt, user_template) | routes/providers.ts |
|
||||
| PUT | /prompts/:id | Actualizeaza prompt | routes/providers.ts |
|
||||
| DELETE | /prompts/:id | Sterge prompt | routes/providers.ts |
|
||||
|
||||
**Tier support in providers API**:
|
||||
- `GET /assignments` returneaza TOATE assignments cu campul `tier` (free + premium in acelasi payload), ordonate pe `component_code, stage_code, tier, fallback_order`. Admin dashboard grupeaza per stage in 2 chain-uri si prezinta toggle Free/Premium.
|
||||
- `GET /assignments?tier=premium` filtreaza server-side (folosit pentru debug sau integrari care vor doar o parte).
|
||||
- `POST /assignments` accepta `tier` in body (defaults to 'free' daca lipseste).
|
||||
- `PUT /assignments/:id` poate schimba `tier` via `COALESCE($tier, tier)` — util pentru a muta un assignment dintr-un tier in altul.
|
||||
|
||||
### Chei extensie browser (`/api/extension-keys`)
|
||||
|
||||
| Metoda | Path | Ce face | Logica in fisier |
|
||||
|--------|------|---------|------------------|
|
||||
| POST | / | Creeaza cheie API (format: didi_ext_...) | routes/extension-keys.ts |
|
||||
| GET | / | Lista chei (admin) | routes/extension-keys.ts |
|
||||
| GET | /validate | Valideaza cheie si returneaza user info | routes/extension-keys.ts |
|
||||
| GET | /user/:userId | Chei pentru un utilizator | routes/extension-keys.ts |
|
||||
| PUT | /:id | Toggle activ/inactiv | routes/extension-keys.ts |
|
||||
| DELETE | /:id | Revocare cheie | routes/extension-keys.ts |
|
||||
| POST | /:id/usage | Incrementeaza contor utilizare (by ID) | routes/extension-keys.ts |
|
||||
| POST | /usage-by-key | Incrementeaza contor utilizare (by API key value) | routes/extension-keys.ts |
|
||||
|
||||
Cache Redis: didi:extension:key:*
|
||||
Tabela PG: extension_api_key
|
||||
|
||||
### Prompturi (`/api/prompts`)
|
||||
|
||||
| Metoda | Path | Ce face | Logica in fisier |
|
||||
|--------|------|---------|------------------|
|
||||
| GET | / | Lista fisiere prompt disponibile | routes/prompts.ts |
|
||||
| GET | /:step | Continut prompt pentru un pas (markdown) | routes/prompts.ts |
|
||||
| GET | /:step/sections | Extrage sectiuni (headings) din fisierul prompt | routes/prompts.ts |
|
||||
|
||||
Pasi: intake, techniques, sources, claims, verdict
|
||||
|
||||
### Waitlist (`/api/waitlist`)
|
||||
|
||||
| Metoda | Path | Ce face | Logica in fisier |
|
||||
|--------|------|---------|------------------|
|
||||
| POST | / | Inscriere pe waitlist (public, email + name) | routes/waitlist.ts |
|
||||
| GET | / | Lista inscrisi (admin) | routes/waitlist.ts |
|
||||
| GET | /count | Numar total inscrisi (public) | routes/waitlist.ts |
|
||||
| DELETE | /:id | Sterge inscris (admin) | routes/waitlist.ts |
|
||||
|
||||
Nota: Waitlist foloseste o baza de date separata (staging-dataLayer-postgres:5432/misinformation_db), nu baza principala `didi-postgres:5432/DIDI`.
|
||||
|
||||
### Profiluri verdict per input type (`/api/input-profiles`)
|
||||
|
||||
| Metoda | Path | Ce face | Logica in fisier |
|
||||
|--------|------|---------|------------------|
|
||||
| GET | / | Lista toate profilurile cu override-uri | routes/input-profiles.ts |
|
||||
| GET | /:code | Un profil cu override-uri (text_no_url, image, video etc.) | routes/input-profiles.ts |
|
||||
| PUT | /:code | Update profil (ponderi, reguli INCONCLUSIVE, disclosure multipliers) | routes/input-profiles.ts |
|
||||
| GET | /:code/overrides | Override-uri per profil | routes/input-profiles.ts |
|
||||
| PUT | /:code/overrides | Update override-uri per profil | routes/input-profiles.ts |
|
||||
| GET | /scoring-config/:component | Citeste scoring_config per componenta din component_config PG | routes/input-profiles.ts |
|
||||
| PUT | /scoring-config/:component | Update scoring_config per componenta | routes/input-profiles.ts |
|
||||
|
||||
6 profiluri fixe (nu se adauga/sterg): `text_no_url`, `text_with_url`, `image`, `audio`, `video`, `url`.
|
||||
Fiecare profil defineste: ponderi componente (total=100%), override-uri active, reguli INCONCLUSIVE, AI disclosure multipliers, confidence config.
|
||||
Sync to Redis: `didi:config:pipeline:v1:input_profiles`.
|
||||
|
||||
### Moderation Config (`/api/moderation-config`)
|
||||
|
||||
| Metoda | Path | Ce face | Logica in fisier |
|
||||
|--------|------|---------|------------------|
|
||||
| GET | / | Citeste tot rand-ul de config (single-row, config_id=1) | routes/moderation-config.ts |
|
||||
| PUT | / | Update orice camp whitelisted; seteaza `updated_by` din header `x-user-id` | routes/moderation-config.ts |
|
||||
|
||||
Campuri whitelisted: triage_enabled, confidence_low, risk_grey_min/max, queue_relax_at, queue_strict_at, auto_tune_enabled, brain_enabled, brain_url, brain_lookup_timeout_ms, brain_write_timeout_ms, brain_confidence_min_silver, brain_semantic_threshold, brain_per_component (JSONB).
|
||||
|
||||
### Sensitive Topics (`/api/sensitive-topics`)
|
||||
|
||||
| Metoda | Path | Ce face | Logica in fisier |
|
||||
|--------|------|---------|------------------|
|
||||
| GET | /?active=true\|false\|all | Lista topicuri (default active=true) | routes/sensitive-topics.ts |
|
||||
| POST | / | Creeaza topic (valideaza `topic_code` regex `[a-z0-9_]+`, 409 la duplicat) | routes/sensitive-topics.ts |
|
||||
| PUT | /:id | Update label sau is_active | routes/sensitive-topics.ts |
|
||||
| DELETE | /:id | Soft delete (set is_active=false) | routes/sensitive-topics.ts |
|
||||
|
||||
### Moderation Roles (`/api/moderation-roles`)
|
||||
|
||||
| Metoda | Path | Ce face | Logica in fisier |
|
||||
|--------|------|---------|------------------|
|
||||
| GET | / | Lista toate rolurile cu permisiuni | routes/moderation-roles.ts |
|
||||
| PUT | /:code | Update toggle fields (can_resolve, can_escalate, can_force_gold_brain, is_active) | routes/moderation-roles.ts |
|
||||
|
||||
Fara POST/DELETE -- rolurile sunt fixe (`moderator`, `senior_moderator`).
|
||||
|
||||
### Catalog resurse AI (`/api/skills`) -- Modul 1
|
||||
|
||||
| Metoda | Path | Ce face | Logica in fisier |
|
||||
|--------|------|---------|------------------|
|
||||
| GET | /?health=true | Catalog complet resurse AI (health probe live optional) | routes/skills.ts |
|
||||
|
||||
Read-only registry care raspunde la cerinta de caiet "catalog resurse AI: modele / skills / code-jobs". Trei sectiuni:
|
||||
- `analysis_components` -- nodurile pipeline-ului de analiza (agent-v3), configurabile prin didiFramework (prompturi/modele/ponderi).
|
||||
- `extractor_skills` -- **modulele platformei AI (integrare Lot 1)**: fiecare un serviciu Python izolat pe host-ul platformei AI (`AI_PLATFORM_HOST`, default `<HOST_IP>`), apelabil prin API (llm-inference, embeddings, rerank, audio/Whisper, video/BusterX++, extractors, forensic-features, web, cloak, didi-brain). Cu `?health=true`, didiFramework **probeaza live** serviciile Lot 1 (fail-open). didiFramework nu descrie logica interna a acestor module — doar le cataloghează si le monitorizeaza prin URL din env.
|
||||
- `code_jobs` -- executie cod = aceleasi module Python containerizate (izolare per container); job-uri ad-hoc pe roadmap.
|
||||
|
||||
Plus pointeri: `models_catalog` -> `/api/providers/models`, `pipelines_catalog` -> `/api/pipelines`.
|
||||
Env: `AI_PLATFORM_HOST` (default <HOST_IP>), `AI_PLATFORM_TOKEN` (Bearer optional pentru gateway/catalog-api Lot 1).
|
||||
|
||||
### Pipelines (`/api/pipelines`) -- alias input-profiles, Modul 1
|
||||
|
||||
Acelasi router ca `/api/input-profiles`, expus si sub `/api/pipelines` fiindca `input_type_profile` **este** definitia de pipeline in sensul caietului (Modul 1: creare/editare/clonare/versionare/publicare/activare). Endpoint-uri de lifecycle (pe langa GET/PUT din sectiunea input-profiles):
|
||||
|
||||
| Metoda | Path | Ce face | Logica in fisier |
|
||||
|--------|------|---------|------------------|
|
||||
| POST | /:code/clone | Cloneaza un profil sub cod nou | routes/input-profiles.ts |
|
||||
| POST | /:code/activate | Activeaza profilul | routes/input-profiles.ts |
|
||||
| POST | /:code/deactivate | Dezactiveaza profilul | routes/input-profiles.ts |
|
||||
| GET | /:code/versions | Istoric versiuni (migration 016) | routes/input-profiles.ts |
|
||||
| POST | /:code/versions/:versionId/restore | Restaureaza o versiune anterioara | routes/input-profiles.ts |
|
||||
| POST | /import | Import profil din payload exportat | routes/input-profiles.ts |
|
||||
|
||||
Consumat de pagina "Pipelines" din admin-dashboard (dry-run via agent-v3).
|
||||
|
||||
### Notificari (`/api/notifications`)
|
||||
|
||||
| Metoda | Path | Ce face | Logica in fisier |
|
||||
|--------|------|---------|------------------|
|
||||
| GET | /health | Verifica conexiunea SMTP (nu trimite) | routes/notifications.ts |
|
||||
| POST | /test | Trimite email de test (body: to) | routes/notifications.ts |
|
||||
| POST | /credit-reset | Trigger manual reset credite Free (debug) | routes/notifications.ts |
|
||||
|
||||
---
|
||||
|
||||
## Autentificare JWT globala (Modul 7)
|
||||
|
||||
`src/config/jwt-verify.ts` exporta `jwtVerifyGate()`, montat **global in server.ts inaintea tuturor rutelor** (`app.use(jwtVerifyGate())`). Orice Bearer token care arata a JWT trebuie sa verifice semnatura RS256 + `exp` fata de JWKS-ul realm-ului emitent (`KEYCLOAK_URL/realms/<realm>/protocol/openid-connect/certs`), altfel request-ul primeste 401 (`JWT_INVALID`). Request-urile fara Bearer JWT (sau cu API key opac) trec neatinse — auth per-ruta (requireAdmin etc.) decide mai departe.
|
||||
|
||||
- Realm-uri permise: `JWT_ALLOWED_REALMS` (default `didi-clients,didi-admins`).
|
||||
- Escape hatch dev: `JWT_VERIFY_ENABLED=false` (dezactiveaza gate-ul, cu warning zgomotos).
|
||||
- JWKS cache-uit per realm (`createRemoteJWKSet` din `jose`).
|
||||
|
||||
---
|
||||
|
||||
## openapi.yaml
|
||||
|
||||
Spec OpenAPI la radacina serviciului (`openapi.yaml`), documenteaza suprafata de API livrata (Modulele 5-7 din caiet).
|
||||
|
||||
---
|
||||
|
||||
## Utilitare interne
|
||||
|
||||
### crud-factory.ts
|
||||
|
||||
Exporturi: `createCrudRouter`, `createReadOnlyRouter`
|
||||
|
||||
`createCrudRouter` -- genereaza automat rute CRUD standard pentru orice tabela:
|
||||
- GET / -- lista cu count
|
||||
- GET /:id -- get by ID
|
||||
- GET /:id/dependencies -- verifica copii
|
||||
- POST / -- create cu auto-ID (MAX+1) + parameter entry optional
|
||||
- PUT /:id -- update partial
|
||||
- DELETE /:id -- safe delete cu dependency check
|
||||
- DELETE /:id?force=true -- forteaza stergerea
|
||||
|
||||
`createReadOnlyRouter` -- genereaza rute read-only (pentru tabele lookup):
|
||||
- GET / -- lista cu count
|
||||
- GET /:id -- get by ID
|
||||
|
||||
Configurat prin CrudConfig: tableName, primaryKey, columns[], parameterType, orderBy.
|
||||
Folosit de: verdicts.ts, weights.ts, claims.ts, source-assessment.ts (tabele leaf).
|
||||
|
||||
### dependency-checker.ts
|
||||
|
||||
Verifica dependente intre tabele inainte de stergere:
|
||||
- `checkDependencies(table, idColumn, id)` -- returneaza canDelete + lista copii
|
||||
- `batchCheckDependencies(table, idColumn, ids[])` -- verificare batch pentru mai multe inregistrari
|
||||
- `safeDelete(table, idColumn, id, force)` -- sterge doar daca nu are copii
|
||||
- `isLeafTable(table)` -- daca nu are copii posibili
|
||||
- `hasDependencyRules(table)` -- daca tabela are reguli de dependenta definite
|
||||
- `getDependencyRules(table)` -- returneaza regulile de dependenta
|
||||
- `getDependencySummary(table)` -- sumar complet (hasRules, isLeaf, rules, canHaveChildren)
|
||||
|
||||
Ierarhia definita: dimension->subdimension->technique->indicator/rule, platform_modifier->platform, source_type->domain_attribute, etc.
|
||||
|
||||
---
|
||||
|
||||
## Baza de date PostgreSQL
|
||||
|
||||
### Schema bos_parammgmt (principala)
|
||||
|
||||
| Tabela | Scop | Parinte |
|
||||
|--------|------|---------|
|
||||
| dimension | Dimensiuni analiza (nivel 1) | - |
|
||||
| subdimension | Subdimensiuni (nivel 2) | dimension |
|
||||
| technique | Tehnici manipulare (nivel 3) | subdimension |
|
||||
| technique_indicator | Indicatori per tehnica | technique |
|
||||
| technique_validation_rule | Reguli validare per tehnica | technique |
|
||||
| parameter | Tracking versiuni + tipuri parametri | referit de multe tabele |
|
||||
| verdict_category | Categorii verdict (RELIABLE..DISINFO) | - |
|
||||
| risk_mapping | Mapare scor -> nivel risc | - |
|
||||
| severity_assessment | Nivele severitate | - |
|
||||
| component_weight | Ponderi componente analiza | - |
|
||||
| weight_scenario | Scenarii combinatii ponderi | - |
|
||||
| multiplier | Multiplicatori (topic, temporal, reach) | - |
|
||||
| platform | Platforme social media | platform_modifier |
|
||||
| platform_modifier | Modificatori platforma | - |
|
||||
| source_type | Tipuri sursa | - |
|
||||
| source_credibility | Credibilitate sursa | - |
|
||||
| domain_age_score | Scoruri varsta domeniu | - |
|
||||
| domain_risk_level | Nivele risc domeniu | - |
|
||||
| domain_red_flag | Red flags domeniu | - |
|
||||
| domain_attribute | Atribute domeniu | sursa_type, credibility, age, risk, flag |
|
||||
| author_classification | Clasificari autor | - |
|
||||
| author_credibility | Credibilitate autor | - |
|
||||
| author | Autori | author_classification, author_credibility |
|
||||
| claim | Statusuri claim (VT, LT, UV, LF, VF) | - |
|
||||
| claim_type | Tipuri claim (EF, VF, RE, SC, QA) | - |
|
||||
| confidence | Nivele incredere | - |
|
||||
| interpretation | Interpretari concordanta | - |
|
||||
| llm_provider | Provideri LLM | - |
|
||||
| llm_model | Modele LLM | llm_provider |
|
||||
| component_provider_assignment | Assignment componenta -> model | llm_provider, llm_model |
|
||||
| component_stage_assignment | Assignment model LLM per componenta + etapa (cu fallback order) | llm_provider, llm_model |
|
||||
| component_prompt | Prompturi per componenta + etapa (system_prompt, user_template) | - |
|
||||
| component_config | Configurari JSONB per componenta (config_key / config_value) | - |
|
||||
| provider_api_key | Chei API provider | llm_provider |
|
||||
| extension_api_key | Chei API extensie browser | - |
|
||||
|
||||
Tabele noi in bos_parammgmt (migration 011):
|
||||
|
||||
| Tabela | Scop | Rute CRUD |
|
||||
|--------|------|-----------|
|
||||
| `moderation_config` | Single-row config (CHECK config_id=1) pentru HIL triage + brain client; 14 campuri inclusiv brain_enabled, brain_url, thresholds, brain_per_component (JSONB) | /api/moderation-config |
|
||||
| `sensitive_topic` | Topics care declanseaza HIL review (5 seed: elections, health, war, covid, climate); soft-delete via is_active | /api/sensitive-topics |
|
||||
| `moderation_role` | Keycloak role -> permisiuni (2 seed: moderator, senior_moderator); CHECK constraints pe toggles | /api/moderation-roles |
|
||||
|
||||
### Schema bos_sysadmin (utilizatori)
|
||||
|
||||
| Tabela | Scop |
|
||||
|--------|------|
|
||||
| user_credential | Credentiale utilizator (keycloak_id, parola) |
|
||||
| internet_user | Profil utilizator (email, credite) |
|
||||
| subscription | Abonament activ |
|
||||
| subscription_plan | Planuri disponibile (Free, Pro, Enterprise) |
|
||||
|
||||
### Schema bos_subscriber (date personale)
|
||||
|
||||
| Tabela | Scop |
|
||||
|--------|------|
|
||||
| person | Date persoana (nume) |
|
||||
| persoana_fizica | Persoana fizica + CNP |
|
||||
| address | Adresa |
|
||||
| contact | Contact (email, telefon) |
|
||||
|
||||
### Schema bos_analysis (rezultate analize)
|
||||
|
||||
| Tabela | Scop |
|
||||
|--------|------|
|
||||
| analysis_session | Sesiunea root |
|
||||
| analysis_techniques | Rezultat componenta Techniques |
|
||||
| analysis_ai_tampered | Rezultat componenta AI-Tampered |
|
||||
| analysis_claims | Rezultat componenta Claims |
|
||||
| analysis_domain | Rezultat componenta Domain |
|
||||
| analysis_verdict | Verdict final |
|
||||
| moderation_queue | Coada review HIL (FK la `analysis_session.session_id` UUID); campuri: queue_id BIGSERIAL, priority 1-5, enqueue_reason, status (pending\|in_review\|resolved\|declined\|auto_closed), resolution_action (approved\|corrected\|rejected), assigned_to/resolved_by, time tracking |
|
||||
|
||||
6 coloane noi pe `analysis_session` (migration 011): `review_status`, `human_corrected`, `human_corrections` (JSONB diff), `verified_by`, `verified_at`, `review_notes`.
|
||||
|
||||
Accesat prin: routes/history.ts (citire), routes/sync-analysis.ts (scriere legacy)
|
||||
|
||||
---
|
||||
|
||||
## Servicii externe
|
||||
|
||||
| Serviciu | Scop | Unde in cod |
|
||||
|----------|------|-------------|
|
||||
| PostgreSQL (`didi-postgres:5432`, local) | Stocare permanenta parametri + utilizatori + analize (PG17, DB `DIDI`, user `bos_interface`, schema `bos_parammgmt`) | config/database.ts |
|
||||
| Redis (`didi-cache:6379`, local ACTIV) | Cache framework pentru agent-v3 (via `createRedisConnection`) | routes/sync-redis.ts |
|
||||
| RabbitMQ (`staging-dataLayer-rabbitmq`, local) | Mesagerie analiza | (consumat de agent-v3) |
|
||||
| MinIO (`staging-dataLayer-minio:9000`, local) | Stocare fisiere media | config/minio.ts, routes/uploads.ts |
|
||||
| Keycloak (`didi-keycloak:8080/auth`, local; port extern 28080) | Autentificare OAuth2, management utilizatori, JWKS pentru jwtVerifyGate | routes/auth.ts, routes/admin.ts, config/jwt-verify.ts |
|
||||
| Kong (gateway local `didi-kong`) | Verificare JWT inainte de request | implicit (nu apelat direct) |
|
||||
| Platforma AI (Lot 1, `AI_PLATFORM_HOST`, default <HOST_IP>) | Module extractori/LLM probate de catalogul `/api/skills` | routes/skills.ts |
|
||||
|
||||
---
|
||||
|
||||
## Cum comunica cu agent-v3
|
||||
|
||||
### Framework -> Redis -> agent-v3
|
||||
|
||||
1. Admin modifica parametri in dashboard (CRUD pe didiFramework)
|
||||
2. Admin apasa "Sync Redis" (POST /api/sync-redis)
|
||||
3. didiFramework citeste toata ierarhia din PostgreSQL
|
||||
4. Scrie JSON-uri compacte in Redis (didi:framework:*)
|
||||
5. agent-v3 citeste din Redis la fiecare analiza
|
||||
|
||||
### agent-v3 -> didiFramework (credite)
|
||||
|
||||
1. agent-v3 primeste request de analiza
|
||||
2. Apeleaza POST /api/auth/internal/check-credits (body: keycloak_id, media_type)
|
||||
3. Daca hasEnoughCredits=true, ruleaza analiza
|
||||
4. Apeleaza POST /api/auth/internal/deduct-credits (body: keycloak_id, media_type, session_id)
|
||||
|
||||
### agent-v3 -> didiFramework (extensie browser)
|
||||
|
||||
1. Extensia browser trimite request cu X-API-Key la agent-v3
|
||||
2. agent-v3 apeleaza GET /api/extension-keys/validate?api_key=... (validare cheie)
|
||||
3. didiFramework returneaza user_id, user_email, is_active
|
||||
4. agent-v3 ruleaza analiza cu identitatea validata
|
||||
|
||||
---
|
||||
|
||||
## Pattern-uri importante
|
||||
|
||||
1. **Safe delete** -- orice parinte verifica copiii inainte de stergere, cu raspuns detaliat
|
||||
2. **Parameter table** -- fiecare entitate creata genereaza un record in `parameter` (versionare)
|
||||
3. **CRUD factory** -- tabele simple folosesc crud-factory.ts (un singur fisier configurat)
|
||||
4. **Typo workaround** -- coloana `subdmiension_name` corectata in cod la `subdimension_name`
|
||||
5. **Auto-inregistrare** -- GET /me creeaza utilizatorul daca exista in Keycloak dar nu in PG
|
||||
6. **Bucket per user** -- MinIO creeaza bucket `user-{id}` cu folder-e tipizate la inregistrare
|
||||
7. **Light history** -- listele de istoric nu includ JSONB-uri, doar scoruri sumare
|
||||
8. **Flat canonical types** -- detaliul unei analize returneaza acelasi format ca agent-v3
|
||||
9. **Tier (free/premium)** -- toate stage_assignments au coloana `tier`. sync-redis emite structura nested `{stage_code: {free: {...}, premium: {...}}}`. agent-v3 rezolva tier-ul din `planType` returnat de check-credits si ruleaza chain-ul corespunzator cu fallback automat la 'free' daca 'premium' lipseste.
|
||||
10. **HIL triage in pipeline** -- fiecare run de analiza trece prin `shouldEnqueueForReview()` dupa persist. Wrapped in try/catch -- esecul NU blocheaza analiza. Citeste pragurile din Redis (60s cache).
|
||||
11. **Soft role check** -- in staging (fara JWT) toate endpoint-urile de moderare permit; in productie cu JWT validat de Kong + `realm_access.roles`, check strict.
|
||||
|
||||
---
|
||||
|
||||
## Migration history (SQL)
|
||||
|
||||
Directorul `sql/migrations/` contine migratiile aplicate manual pe cluster (nu rulate automat la startup):
|
||||
|
||||
| Migration | Descriere |
|
||||
|-----------|-----------|
|
||||
| 001_add_explanation_columns.sql | `analysis_verdict.explanation_ro/_en` + view `v_analysis_full` |
|
||||
| 002_add_component_pilot_config.sql | `component_stage_assignment`, `component_prompt`, `component_config` tables |
|
||||
| 003_add_source_assessment.sql | `source_assessment` component rows + source evaluation tables |
|
||||
| 004_add_llm_usage.sql | `analysis_session.llm_usage JSONB` (tokens per componenta) |
|
||||
| 005_add_bilingual_columns.sql | Coloane `_ro` / `_en` pe tehnici/prompts |
|
||||
| **006_add_tier_column.sql** | **`component_stage_assignment.tier varchar(20) DEFAULT 'free'` + unique constraint `(component_code, stage_code, tier, fallback_order)` + CHECK constraint (`free`|`premium`) + index compus** |
|
||||
| **007_seed_premium_assignments.sql** | **Seed 32 rows `tier='premium'` pentru cele 8 stages LLM (techniques/ai-tampered/claims/source-assessment) — Gemini 3 Flash + Claude Sonnet + Grok 4 Fast + Qwen local** |
|
||||
| **008_seed_vision_assignments.sql** | **Seed 7 rows pentru component `vision` stage `image_analysis` (3 free + 4 premium) — citit de agent-v3 `vision.ts`** |
|
||||
| **009_seed_verdict_assignments.sql** | **Seed 8 rows pentru component `verdict` stage `verdict_review` (4 free + 4 premium) — citit de `verdict-explanation.ts`** |
|
||||
| **010_add_user_storage_quota.sql** | **2 coloane pe `bos_sysadmin.internet_user` pentru cota de stocare (arhitectura single-bucket MinIO post-2026-04-25, inlocuieste bucket tags) — acces instant la quota fara listare prefix** |
|
||||
| **011_add_moderation.sql** | **HIL Moderation foundation: 6 coloane pe `analysis_session`, tabela `moderation_queue` (in bos_analysis), 3 tabele config in bos_parammgmt cu seed-uri. Companion `011_rollback.sql`. UUID type pentru FK.** |
|
||||
| **012_add_topic_volatility.sql** | **Phase D1 — extends `bos_parammgmt.sensitive_topic` cu `volatility ('volatile'|'evolving'|'stable')`, `cache_ttl_hours`, `recency_window_days`, `half_life_days`. Seed defaults per topic (war/elections=volatile@24h, health/covid=evolving@168h, climate=stable@720h). Companion `012_rollback.sql`. Brain reads via HTTP poll on /api/sensitive-topics + applies as override on classifier output.** |
|
||||
| **013_user_audit_log.sql** | **Phase U — `bos_sysadmin.user_audit_log` (audit_id bigserial, internet_user_id, target_email/keycloak_id, actor_keycloak_id/email, action, payload jsonb, request_ip, user_agent, created_at). 4 indexes (user, actor, action+time, time). Powers DIDI admin "Audit Log" tab. Companion `013_rollback.sql`.** |
|
||||
| **014_add_atomic_path_prefix.sql** | **Punte `sensitive_topic` -> taxonomie atomic: coloana optionala `atomic_path_prefix` care mapeaza un `topic_code` policy-level (ex. 'health') la prefixul de path din atomic-server. Companion `014_rollback.sql`.** |
|
||||
| **015_social_post.sql** | **DESI 6 — `bos_sysadmin.social_post` (post_id bigserial, session_id UUID optional, platform, continut, autor, timestamp, engagement) pentru postare social media (Facebook) din admin-dashboard + audit PNRR. Companion `015_rollback.sql`.** |
|
||||
| **016_input_profile_versions.sql** | **Modul 1 — `bos_parammgmt.input_type_profile_version`: fiecare PUT pe `/api/input-profiles(/pipelines)/:code` face snapshot al randului anterior inainte de modificare (edit -> version -> restore -> activate/deactivate -> clone). Companion `016_rollback.sql`.** |
|
||||
| **017_model_catalog_attributes.sql** | **Atribute de catalog cerute de caiet pe `llm_model`: `deployment` (local/remote), `compute_target` (gpu/cpu/hybrid), `quantization`, `capabilities` (jsonb). Companion `017_rollback.sql`.** |
|
||||
|
||||
Migrations 006-009 sunt cele care au introdus tier-based routing. Aplicare manual cu `docker exec didi-framework node -e "fs.readFileSync + pool.query"` (sync-redis nu ruleaza migrations automat).
|
||||
|
||||
---
|
||||
|
||||
## Functionalitati suplimentare (in afara celor 9 module de caiet)
|
||||
|
||||
Elemente livrate care nu fac parte din cele 9 module ale caietului de sarcini, dar sunt operationale in serviciu:
|
||||
|
||||
| Functionalitate | Unde in cod | Note |
|
||||
|-----------------|-------------|------|
|
||||
| Plati Stripe | `routes/webhooks/stripe.ts` (montat la `/webhooks/stripe`, raw body inaintea `express.json`) | Test mode; secret in `.env` (`STRIPE_*`) |
|
||||
| Postare social media (DESI 6) | `routes/admin/social.ts` + migration `015_social_post.sql` | Facebook Graph API (`FACEBOOK_*` in `.env`); audit PNRR cine/ce/cand |
|
||||
| HIL moderation (triage + brain client) | `routes/moderation-config.ts`, `routes/sensitive-topics.ts`, `routes/moderation-roles.ts` + migration `011` | Config expus si in dashboard (ModerationSettings) |
|
||||
| Chei API extensie browser | `routes/extension-keys.ts` | Format `didi_ext_...`, cache Redis `didi:extension:key:*` |
|
||||
| Notificari email | `routes/notifications.ts` + `config/email.ts` + credit-reset cron | SMTP `mail.finesynergy.eu` (`SMTP_*` in `.env`) |
|
||||
|
||||
---
|
||||
|
||||
## Phase U — User management endpoints (2026-05-05)
|
||||
|
||||
`src/routes/admin.ts` extins masiv pentru a permite admin-ului să facă tot CRUD-ul de Keycloak fără să intre în consola Keycloak.
|
||||
|
||||
### Helpers Keycloak (in-memory cache 10min)
|
||||
|
||||
- `listRealmRoles()` — lista realm roles (filtrează default-uri Keycloak: `offline_access`, `uma_authorization`, `default-roles-didi-clients`).
|
||||
- `getUserRoles(keycloakId, force?)` — current realm roles per user.
|
||||
- `setUserRoles(keycloakId, desiredNames[])` — diff add/remove pe role-mappings. Returnează `{added, removed, errors}`.
|
||||
- `listKeycloakGroups()` / `getUserGroups()` / `setUserGroup()` — același pattern pentru grupuri (single membership).
|
||||
- `sendResetPasswordEmail(keycloakId, {lifespanSeconds, redirectUri})` — Keycloak `execute-actions-email` cu `UPDATE_PASSWORD`.
|
||||
- `logUserAudit(req, ctx)` — extrage actor din JWT, INSERT în `bos_sysadmin.user_audit_log`. Best-effort (swallow errors). Apelat din toate mutațiile.
|
||||
|
||||
### Endpoint-uri noi
|
||||
|
||||
```
|
||||
GET /api/admin/realm-roles — lista roluri eligibile
|
||||
GET /api/admin/groups — lista grupuri Keycloak
|
||||
GET /api/admin/users/:id/roles — rolurile current
|
||||
PUT /api/admin/users/:id/roles — body {roles: ["name", ...]} → diff add/remove (audit logged)
|
||||
GET /api/admin/users/:id/group — grupul current (single)
|
||||
PUT /api/admin/users/:id/group — body {group: "name" | null} (audit logged)
|
||||
POST /api/admin/users/:id/reset-password — body {lifespanSeconds?, redirectUri?} → email Keycloak (audit logged)
|
||||
GET /api/admin/users/:id/usage-history?limit=100 — citește bos_sysadmin.ai_credit_usage flexibly
|
||||
GET /api/admin/audit-log?action=&actor=&since=&internet_user_id=&limit=&offset= — paginated browser
|
||||
```
|
||||
|
||||
### `GET /api/admin/users` extins
|
||||
|
||||
Răspunsul include acum `storageUsedBytes`, `storageLimitBytes`, `storagePct`, `roles[]`, `groups[]` per user. Acceptă query `?sync_status=all|synced|keycloak_only` și `?include_kc_meta=false` pentru a sări over fan-out-ul Keycloak când nu e nevoie.
|
||||
|
||||
### Endpoint-uri existente — acum loghează audit
|
||||
|
||||
`PUT /:id`, `DELETE /:id`, `POST /sync`, `PUT /:id/email-verified`, `PUT /:id/subscription` apelează `logUserAudit` cu `payload` ce conține diff-ul aplicat.
|
||||
|
||||
### `sensitive-topics.ts` — 4 câmpuri noi în CRUD
|
||||
|
||||
`GET` returnează volatility/cache_ttl_hours/recency_window_days/half_life_days. `POST` și `PUT` acceptă acelea opționale + validare server-side a range-urilor (ttl 1-26280, recency 1-365, half_life > 0). `validateVolatilityFields` helper centralizează regulile alături de CHECK constraints PG.
|
||||
|
||||
### `sync-redis.ts` — key Redis nou
|
||||
|
||||
```
|
||||
didi:config:topics:volatility ← {topics: [{topic_code, topic_label, volatility, cache_ttl_hours,
|
||||
recency_window_days, half_life_days}, ...], synced_at}
|
||||
```
|
||||
|
||||
Brain (`topic_volatility.py`) citește prin HTTP la `didi-framework:3005/api/sensitive-topics` (cache 60s). Cheia legacy `didi:config:moderation:v1:sensitive_topics` e neschimbată ca shape — agent-v3 triage continuă să o citească identic.
|
||||
|
||||
### Compose env
|
||||
|
||||
```
|
||||
KEYCLOAK_URL=http://didi-keycloak:8080/auth ← Keycloak LOCAL pe didi-network, servit sub /auth (KC_HTTP_RELATIVE_PATH)
|
||||
KEYCLOAK_ADMIN=admin
|
||||
KEYCLOAK_ADMIN_PASSWORD=admin123
|
||||
```
|
||||
|
||||
Trafic intern plain HTTP catre containerul local `didi-keycloak` — nu mai exista cert cluster auto-semnat, deci `NODE_TLS_REJECT_UNAUTHORIZED=0` a fost eliminat.
|
||||
Loading…
Add table
Add a link
Reference in a new issue