# Dashboard Module - INDEX

AI platform monitoring/admin dashboard. Browse archived claims, view ingest history, audit trail, runtime config overrides, costs per provider/tier, and live provider quotas. Single FastAPI service serving:
1. **React 19 + MUI 7 SPA** at `/admin-ai/` (default modern UI, added in Phase C 2026-05-02)
2. **Jinja2 templates** at `/`, `/history`, `/cost`, `/providers`, `/archive`, `/audit`, `/config` (legacy, kept side-by-side until full deprecation)
3. **JSON API** at `/api/*` and `/admin-ai/api/*` (dual-mounted) consumed by `web-api` + the SPA itself

## Stack

- Python 3.10+ (FastAPI 0.115+, Uvicorn)
- React 19 + MUI 7 + Vite 7 + react-router 7 + TanStack Query + Recharts (built into `/app/web_dist/`, served as static via SPAStaticFiles with index-fallback)
- Jinja2 templates legacy (HTMX + Alpine.js + Tailwind CDN — kept until React reaches 100% parity)
- SQLAlchemy 2.0 async + asyncpg, Alembic
- pydantic-settings (env prefix `DASHBOARD_`)
- httpx for live provider quota fetching, brain proxy
- python-jose[cryptography] for Keycloak JWT validation

## Coordinates

- **SPA URL**: `https://10.11.10.12:8443/admin-ai/` (via frontend nginx) or `http://10.11.10.12:51300/admin-ai/` (direct)
- **Public URL**: `https://didi365.eu/admin-ai/` (via Cloudflare tunnel + frontend nginx)
- Container: `didiAI-dashboard` (alongside `didiAI-dashboard-db` on `:15432`)
- Compose: `deploy/docker-compose.yml`, profile `dashboard`
- Network: `didi-network` (unified single network for all DIDI + AI platform stacks since 2026-05-04)

## Auth (current state — 2026-05-04)

**Hybrid auth** in `dependencies.py:verify_bearer_token`:
1. `STAGING_MODE=true` → all auth bypassed (default for dev)
2. JWT (3 dot-separated parts) → validated via `keycloak_auth.py` (JWKS cache 10min, signature, issuer, exp, role check)
3. DB-backed bearer tokens (legacy, from CLI `dashboard create-user`)
4. Static `api_tokens` env var (legacy fallback)
5. Otherwise → 401

**Keycloak settings** (`config.py:DashboardSettings`):
- `keycloak_url`, `keycloak_realm` (default `didi-clients`), `keycloak_client_id` (default `ai-platform-dashboard`)
- `keycloak_required_role` (default `admin` — same role as DIDI admin-dashboard for unified access)
- Manual setup: create client + assign role via Keycloak admin or `deploy/setup-keycloak.sh`

**Cutover from staging to prod**:
- Set `DASHBOARD_KEYCLOAK_URL=https://sso.clossers.com`, `DASHBOARD_STAGING_MODE=false`, `VITE_STAGING_MODE=false`
- Rebuild image with build args (Dockerfile bakes Keycloak config into JS bundle at build time)

Plan/runbook: `AI_PLATFORM_RESKIN_PLAN.md` (Phase C.8) + `/home/admin365/didi_mono/UNIFIED_KEYCLOAK_CUTOVER.md`

## Ce face

Sections served as HTML pages (`pages.py`) + JSON-mirror endpoints in `routes/`:

- **Overview** (`/`) - KPIs (totals, error rate, avg duration, total cost) + provider grid (live quotas)
- **Providers** (`/providers`) - detailed provider cards + raw table (live SerpAPI/Tavily/Brave/LinkUp/Exa/OpenRouter/internal stats)
- **History** (`/history`, `/history/{request_id}`) - request log with filters (tier, provider, endpoint, hours) + drill-down with full stages + raw_request/response
- **Cost** (`/cost`) - 24h/7d/30d spend, projected monthly, by-provider, by-tier, top 10 expensive requests, quota-vs-budget bars
- **Archive** (`/archive`, `/archive/{claim_id}`) - browse promoted claims + linked articles (permanent storage seeded via `POST /archive/promote/{request_id}`)
- **Audit** (`/audit`) - audit log entries (config.set, config.reset, config.delete, archive.promote)
- **Config** (`/config`) - runtime overrides table grouped by category (providers/routing/llm/tiers); HTMX in-place edit + reset

## API endpoints

### `routes/health.py`
- `GET /health` - liveness + DB ping
- `GET /ready` - app.state populated check

### `routes/ingest.py` (no auth, service-to-service)
- `POST /api/ingest/event` - receives request events from `web-api` middleware; trims fields, computes cost via `pricing.estimate_cost` if missing, inserts into `request_history`

### `routes/history.py` (no auth, read-only)
- `GET /api/history` - filtered list (tier, provider, endpoint, hours, limit, offset)
- `GET /api/history/{request_id}` - full record (includes stages + raw_request/response)

### `routes/stats.py` (no auth, read-only)
- `GET /api/stats/providers?force=` - live provider stats (cached `provider_stats_cache_seconds`, default 30s)
- `GET /api/stats/summary?hours=` - aggregated counters (totals, by_tier, by_provider, by_endpoint, error_rate, avg_duration, total_cost)
- `GET /api/stats/timeline?hours=` - hourly buckets (`date_trunc('hour', ...)`) for charts

### `routes/archive.py`
- `GET /api/archive/claims` - paginated claims list with optional `q` ilike search
- `GET /api/archive/claims/{claim_id}` - single claim + linked articles
- `POST /api/archive/promote/{request_id}` (auth) - promote a `request_history` row into `claims_archive` + `articles_archive` + `claim_articles`; only `/v1/gather` rows can be promoted

### `routes/config.py`
- `GET /api/config` - all keys with overrides merged on top of `KNOWN_KEYS` defaults (consumed by `web-api`, no auth)
- `GET /api/config/{key}` - single key
- `PUT /api/config/{key}` (auth) - set override, validates against schema (bool/int/enum/csv/string), writes audit log
- `DELETE /api/config/{key}` (auth) - revert to default, writes audit log

`KNOWN_KEYS` in `routes/config.py` enumerates the runtime keys consumed by `web-api`: `web.providers.{serpapi,tavily,brave,linkup,exa}.enabled`, `web.premium.strategy`, `web.premium.priority_order`, `web.openrouter.model`, `web.tier.{free,premium}.max_search_results`.

### `routes/pages.py` (Jinja HTML)
- `GET /` - overview
- `GET /history`, `GET /history/{request_id}`
- `GET /providers`
- `GET /cost`
- `GET /archive`, `GET /archive/{claim_id}`
- `GET /audit`
- `GET /config` + HTMX form handlers `POST /config/{key}` and `POST /config/{key}/reset` (return partial fragments)

## Structura fisiere

```
src/dashboard/
  __init__.py
  cli.py             # admin CLI: create-user, list-users, delete-user (python -m dashboard.cli)
  auth.py            # SHA-256 + hmac.compare_digest, User CRUD
  config.py          # DashboardSettings (pydantic-settings, DASHBOARD_ prefix), SettingsCache
  logging.py         # get_logger helper (JSON or text via DASHBOARD_LOG_JSON)
  pricing.py         # estimate_cost(endpoint, tier, provider) - per-provider USD
  retention.py       # 30-day rolling cleanup of request_history
  api/
    app.py           # FastAPI factory, lifespan (init engine + provider registry), router wiring
    dependencies.py  # get_session, get_registry, verify_bearer_token, get_username
    routes/
      archive.py     # claims archive CRUD + promote
      config.py      # KNOWN_KEYS + runtime override CRUD
      health.py      # /health, /ready
      history.py     # /api/history (read-only)
      ingest.py      # POST /api/ingest/event (service-to-service)
      pages.py       # all Jinja HTML pages + HTMX handlers
      stats.py       # /api/stats/{providers,summary,timeline}
  db/
    models.py        # Base, RequestHistory, ProviderStatsHourly, ConfigOverride, User, AuditLog,
                     # ClaimsArchive, ArticlesArchive, ClaimArticle
    session.py       # init_engine, get_session, get_session_factory, close_engine
  providers/
    base.py          # ProviderStats dataclass + ProviderClient ABC
    registry.py      # ProviderRegistry (cached fan-out across providers)
    serpapi.py       # SerpAPI quota + plan price
    tavily.py        # Tavily quota
    brave.py         # Brave Search quota
    linkup.py        # LinkUp quota
    openrouter.py    # OpenRouter spend
    internal.py      # Internal services (web-api, SearXNG, vLLM, llama.cpp) live health
  templates/         # Jinja
    base.html
    overview.html
    archive.html, archive_detail.html
    audit.html
    config.html
    cost.html
    history.html, history_detail.html
    providers.html
    partials/
      config_row.html      # HTMX swap target after edit/reset
      provider_card.html   # reusable card on overview + providers pages
      quota_bar.html       # quota progress bar
  static/
    css/, js/        # tailwind via CDN, htmx + alpine inline
deploy/
  Dockerfile
  docker-compose.yml # didiAI-dashboard + didiAI-dashboard-db (postgres:16-alpine)
  deploy.sh          # convenience wrapper around `docker compose --profile dashboard`
tests/
  ...
pyproject.toml       # hatchling build, ruff inherited from ../../ruff.toml
uv.lock
```

## Database

`didiAI-dashboard-db` (postgres:16-alpine, host `:15432` -> container `:5432`). Tables:

- `request_history` - 30-day rolling per-request log (BigInt id, request_id unique, JSON stages/raw_request/raw_response, cost_usd Numeric(12,6), indexes on created_at, tier, provider, endpoint)
- `provider_stats_hourly` - rollups by (provider, hour)
- `config_overrides` - runtime config k/v overrides (key PK, JSON value, updated_by)
- `users` - dashboard users (username unique, token_hash SHA-256, role, last_login)
- `audit_log` - mutation history (timestamp, username, action, target, old_value/new_value JSON)
- `claims_archive` - permanent claims storage (claim_hash unique, verdict, confidence Numeric(5,4), summary, entities JSON, tags JSON)
- `articles_archive` - permanent article full-text (url unique, url_hash unique, full_text, publisher, credibility_score)
- `claim_articles` - M2M claims <-> articles (relevance_score, snippet)

Connection string format: `postgresql+asyncpg://USER:PASS@didiAI-dashboard-db:5432/DB` injected via `DASHBOARD_DATABASE_URL`.

## Pages

- `base.html` - layout shell (Tailwind CDN, sidebar nav, HTMX + Alpine includes)
- `overview.html` - KPI tiles + provider grid (uses `partials/provider_card.html` + `partials/quota_bar.html`)
- `providers.html` - full provider cards + raw quota table
- `history.html` / `history_detail.html` - filterable list + drill-down with stages JSON pretty-print
- `cost.html` - cost cards + by-provider/by-tier breakdown + budget bars
- `archive.html` / `archive_detail.html` - claim search + linked articles
- `audit.html` - chronological mutation log
- `config.html` - runtime overrides grouped by category, HTMX inline edit -> `partials/config_row.html`

## Authentication (current state)

- Bearer token in `Authorization: Bearer <token>` header
- `dependencies.verify_bearer_token` reads header, hashes, scans `users`, returns `User` or 401
- `dependencies.get_username(principal)` extracts username for audit log
- Token issuance via CLI inside container:
  ```
  docker exec -it didiAI-dashboard python -m dashboard.cli create-user <name> [--email] [--role admin|viewer]
  ```
- All read endpoints + `POST /api/ingest/event` are auth-free (VPN-internal trust)

## Reskin plan (FUTURE - NOT done yet)

- React 19 + MUI 7 + Keycloak SSO **DONE 2026-05-02** (Phase C in `agent-v3/IMPLEMENTATION_PLAN_HIL_BRAIN.md`, detail in `AI_PLATFORM_RESKIN_PLAN.md`)
- Becomes admin-only (Keycloak realm role)
- Bearer-token table retired; existing `role` column may persist for historical audit-log mapping
- Brain admin UI (atom browse, force-gold, brain stats) added as a new section in this dashboard during Phase C
- Jinja templates + HTMX endpoints in `pages.py` will be replaced by JSON endpoints; existing `routes/*.py` JSON API stays as the contract

## Deployment

```bash
cd /home/admin365/didi_mono/ai_platform/modules/dashboard/deploy
cp ../.env.example .env   # set DASHBOARD_DB_USER/PASSWORD/NAME + provider keys
./deploy.sh up            # docker compose --profile dashboard up -d --build
```

- Healthcheck: `python -c urllib.request.urlopen('http://localhost:51300/health')` every 30s
- Restart policy: `unless-stopped`
- Settings prefix: `DASHBOARD_*` (see `config.py` for full list)

## Ce NU face

- ~~No SSO yet~~ Keycloak SSO wired (DONE 2026-05-02). Bearer tokens kept as legacy fallback.
- No multi-tenant - single shared `users` table, no per-tenant scoping
- No public access - binds to internal `didi-network` network, not exposed via Kong/edge
- ~~No React frontend yet~~ React 19 SPA at `/admin-ai/` (DONE 2026-05-02). Jinja kept side-by-side until parity.
- No real-time push - HTMX polling, no WebSockets/SSE

## Related docs

- AI platform CLAUDE.md: `/home/admin365/didi_mono/ai_platform/CLAUDE.md`
- Module README: `/home/admin365/didi_mono/ai_platform/modules/dashboard/README.md`
- Reskin + Brain admin plan (Phase C): `/home/admin365/didi_mono/backend/services/orchestration-layer/agent-v3/IMPLEMENTATION_PLAN_HIL_BRAIN.md`
- Web-api ingest middleware (the producer for `POST /api/ingest/event`): `/home/admin365/didi_mono/ai_platform/modules/web-api/`

## Recent Changes (2026-05-05)

- **3 pagini noi**:
  - `/admin-ai/operations/live` (Live Status) — KPI 1h cu refresh 5s, Recent Activity feed (latest 20 cu chip warning daca >1h vechi), Provider Health cu cache age, throughput sparkline 6h
  - `/admin-ai/system/settings` (System Settings) — health endpoint, identity card cu roluri, Configuration Surface (98 chei + breakdown per modul + count overrides), Quick Access dynamic links
  - `/admin-ai/system/schema` (Schema Overrides) — CRUD UI pentru `config_schema_override` (Register/Edit/Delete via dialog cu validare type/min/max/options)
- **Pagini eliminate**: `/admin-ai/system/users` (admin backend gestioneaza userii Keycloak)
- **Overview**: card Phase C status chips -> inlocuit cu Quick Links chips clickable (Live Status, History, Cost, Providers, Brain Atoms, Audit Log, Schema Overrides)
- **Modules ModulePage** (toate 8): tab "Live State" functional cu real backend `/api/proxy/{moduleId}/health` (status badge, latency, payload `/v1/info`, refresh 10s, env override DASHBOARD_<MOD>_HEALTH_URL); tab "Actions" reformat ca "Pending Restarts" cu lista overrides cu `restart_required=true` + comanda SSH copy-able
- **Backend endpoint nou**: `routes/proxy.py` cu `GET /api/proxy/{module_id}/health` — proxy catre modul real, hardcoded URLs pe 10.11.10.17 cu fallback la `DASHBOARD_<MODULE>_HEALTH_URL` env override
- **Cost endpoint imbunatatit**: `GET /api/stats/cost` returneaza `projection_basis` (blend 30d=40%/7d=60%, sau 7d_avg, sau 24h_only), `projection_confidence` (stable/moderate/rough), `trend` (increasing/decreasing/stable), `trend_pct`. UI Cost page afiseaza confidence chip + trend arrow (↗↘→).
- **Providers endpoint imbunatatit**: `GET /api/stats/providers` adauga `last_refresh` (ISO), `age_seconds`, `cache_ttl_seconds`. UI Providers + Live Status afiseaza "refreshed Xs ago" chip (warning peste 120s).
- **ProviderRegistry**: tracks `_cache_wall: datetime` separat de `_cache_time` (monotonic), expune properties `last_refresh_iso` + `age_seconds`.
- **Frontend type fix**: `Live.tsx` + `Providers.tsx` foloseau campuri inexistente (`status`, `quota_used_pct`, `plan`) — corectate la `healthy: bool`, `quota_percent_used`, `display_name`, `plan_name` (matching backend response).
- **Schema migration**: 98 chei seed la primul startup in `config_schema_override`, vizibile in Schema Overrides UI cu actiuni Edit/Delete.

---

## Brain admin pages — Phase D2 (2026-05-05)

3 rute noi sub `/admin-ai/brain/` + 1 tab nou pe `system/audit`. Toate consumă brain prin proxy-ul `/api/brain/*`.

### `pages/brain/Facts.tsx`

Browser pe `brain_fact_status` cu filter (entity ILIKE, predicate exact, current_truth, locked_only, topic), DataGrid paginat. Click pe rând → drawer dreapta cu Triple summary (chip volatility + lock + topics), **Truth timeline** Stepper cu toate `brain_fact_version` rows, **Moderator override form** (moderator_user_id + set_truth + confidence + evidence URLs + lock/unlock + notes → PATCH `/v1/fact_status/{id}`).

### `pages/brain/Invalidate.tsx`

Form-driven mass invalidation cu UX 2-step: Build filter (topic_codes, entity_canonicals, claim_pattern, since, invalidate_gold cu warning) → **Preview (dry run)** → **Confirm invalidate** (button enabled doar după Preview). Side panel: ultimele 10 invalidări (auto-refresh 30s).

### `pages/system/AuditLog.tsx` — refactor cu tabs

Wrapper cu Tabs: **Dashboard tab** → `AuditLogDashboard.tsx` (existing code extras intact); **Brain tab** → `AuditLogBrain.tsx` nou, citește `/api/brain/v1/cache/audit_log` cu action presets (judge_*, fact_truth_*, invalidate, promote_gold), action chip color-coded, payload tooltip JSON pretty.

### Backend (`brain_proxy.py`)

Whitelist extins cu: `/v1/fact_status/list`, `/v1/fact_status/`, `/v1/cache/audit_log`, `/v1/cache/invalidate`, `/v1/canonicalize`. GET pass-through; POST/PATCH cer bearer.

### `types/brain.ts` extins

`Volatility`, `FactStatusItem(+List+VersionItem+Versions+Patch)Response`, `AuditLogItem(+Response)`, `CacheInvalidate(Request|Response)`.

### Routes + sidebar

`App.tsx`: `brain/facts`, `brain/invalidate`. `AppShell.tsx` Brain Admin section: Fact Status (FactCheckIcon), Invalidate (DeleteSweepIcon).