LOT 1 - Optimizare script build -Instalare mono comanda
This commit is contained in:
parent
5380c3fc63
commit
42ff22bf85
127 changed files with 16163 additions and 532 deletions
|
|
@ -10,7 +10,7 @@ AI platform monitoring/admin dashboard. Browse archived claims, view ingest hist
|
|||
- Python 3.10+ (FastAPI 0.115+, Uvicorn)
|
||||
- React 19 + MUI 7 + Vite 7 + react-router 7 + TanStack Query + Recharts (built into `/app/web_dist/`, served as static via SPAStaticFiles with index-fallback)
|
||||
- Jinja2 templates legacy (HTMX + Alpine.js + Tailwind CDN — kept until React reaches 100% parity)
|
||||
- SQLAlchemy 2.0 async + asyncpg, Alembic
|
||||
- SQLAlchemy 2.0 async + asyncpg (tables created via `Base.metadata.create_all` at startup; Alembic is only a dependency, not used at runtime — no migrations applied)
|
||||
- pydantic-settings (env prefix `DASHBOARD_`)
|
||||
- httpx for live provider quota fetching, brain proxy
|
||||
- python-jose[cryptography] for Keycloak JWT validation
|
||||
|
|
@ -23,23 +23,28 @@ AI platform monitoring/admin dashboard. Browse archived claims, view ingest hist
|
|||
- Compose: `deploy/docker-compose.yml`, profile `dashboard`
|
||||
- Network: `didi-network` (unified single network for all DIDI + AI platform stacks since 2026-05-04)
|
||||
|
||||
## Auth (current state — 2026-05-04)
|
||||
## Auth (current state)
|
||||
|
||||
**Hybrid auth** in `dependencies.py:verify_bearer_token`:
|
||||
1. `STAGING_MODE=true` → all auth bypassed (default for dev)
|
||||
2. JWT (3 dot-separated parts) → validated via `keycloak_auth.py` (JWKS cache 10min, signature, issuer, exp, role check)
|
||||
**Shipped in STAGING MODE — Keycloak is supported but disabled in this delivery.**
|
||||
With `DASHBOARD_STAGING_MODE=true` (the delivered default) all auth is bypassed
|
||||
and Keycloak is **not** active. JWT validation only engages when
|
||||
`DASHBOARD_KEYCLOAK_URL` is set (empty value = JWT off).
|
||||
|
||||
**Hybrid auth** in `dependencies.py:verify_bearer_token` (in order):
|
||||
1. `STAGING_MODE=true` → all auth bypassed (this is the delivered state)
|
||||
2. JWT (3 dot-separated parts) → validated via `keycloak_auth.py` (JWKS cache 10min, signature, issuer, exp, role check) — only when `DASHBOARD_KEYCLOAK_URL` is set
|
||||
3. DB-backed bearer tokens (legacy, from CLI `dashboard create-user`)
|
||||
4. Static `api_tokens` env var (legacy fallback)
|
||||
5. Otherwise → 401
|
||||
|
||||
**Keycloak settings** (`config.py:DashboardSettings`):
|
||||
- `keycloak_url`, `keycloak_realm` (default `didi-clients`), `keycloak_client_id` (default `ai-platform-dashboard`)
|
||||
**Keycloak settings** (`config.py:DashboardSettings`) — used only once Keycloak is enabled:
|
||||
- `keycloak_url` (empty by default = disabled), `keycloak_realm` (default `didi-clients`), `keycloak_client_id` (default `ai-platform-dashboard`)
|
||||
- `keycloak_required_role` (default `admin` — same role as DIDI admin-dashboard for unified access)
|
||||
- Manual setup: create client + assign role via Keycloak admin or `deploy/setup-keycloak.sh`
|
||||
|
||||
**Cutover from staging to prod**:
|
||||
- Set `DASHBOARD_KEYCLOAK_URL=https://sso.clossers.com`, `DASHBOARD_STAGING_MODE=false`, `VITE_STAGING_MODE=false`
|
||||
- Rebuild image with build args (Dockerfile bakes Keycloak config into JS bundle at build time)
|
||||
**To enable Keycloak (leave staging mode)** — example values:
|
||||
- Set `DASHBOARD_KEYCLOAK_URL=https://sso.clossers.com` (example), `DASHBOARD_STAGING_MODE=false`, `VITE_STAGING_MODE=false`
|
||||
- Rebuild image with build args (Dockerfile bakes Keycloak config into the JS bundle at build time)
|
||||
|
||||
Plan/runbook: `AI_PLATFORM_RESKIN_PLAN.md` (Phase C.8) + `/home/admin365/didi_mono/UNIFIED_KEYCLOAK_CUTOVER.md`
|
||||
|
||||
|
|
@ -57,6 +62,9 @@ Sections served as HTML pages (`pages.py`) + JSON-mirror endpoints in `routes/`:
|
|||
|
||||
## API endpoints
|
||||
|
||||
> All JSON routers below (except `health`) are **dual-mounted** under both `/api/*`
|
||||
> and `/admin-ai/api/*` (see `app.py`: `for prefix in ("/api", "/admin-ai/api")`).
|
||||
|
||||
### `routes/health.py`
|
||||
- `GET /health` - liveness + DB ping
|
||||
- `GET /ready` - app.state populated check
|
||||
|
|
@ -86,7 +94,29 @@ Sections served as HTML pages (`pages.py`) + JSON-mirror endpoints in `routes/`:
|
|||
|
||||
`KNOWN_KEYS` in `routes/config.py` enumerates the runtime keys consumed by `web-api`: `web.providers.{serpapi,tavily,brave,linkup,exa}.enabled`, `web.premium.strategy`, `web.premium.priority_order`, `web.openrouter.model`, `web.tier.{free,premium}.max_search_results`.
|
||||
|
||||
### `routes/pages.py` (Jinja HTML)
|
||||
### `routes/catalog.py` (auth — Model & Extractor catalog, DB-backed CRUD)
|
||||
- `GET /api/catalog` - list catalog entries (models + extractors)
|
||||
- `POST /api/catalog` - create a catalog entry
|
||||
- `PUT /api/catalog/{entry_id}` - update an entry
|
||||
- `DELETE /api/catalog/{entry_id}` - delete an entry
|
||||
|
||||
### `routes/monitoring.py` (auth — AI monitoring)
|
||||
- `GET /api/monitoring/services` - per-service status
|
||||
- `GET /api/monitoring/queues` - queue depths/metrics
|
||||
- `GET /api/monitoring/latency` - latency metrics
|
||||
|
||||
### `routes/proxy.py` (auth — module health proxy)
|
||||
- `GET /api/proxy/{module_id}/health` - proxy to a platform module's health endpoint (URLs with `DASHBOARD_<MODULE>_HEALTH_URL` env override)
|
||||
|
||||
### `routes/brain_proxy.py` (brain proxy, whitelisted paths)
|
||||
- `GET /api/brain/{path}` - read-only pass-through to the brain service
|
||||
- `POST|PATCH|DELETE /api/brain/{path}` (auth) - mutating pass-through
|
||||
- Whitelist includes: `/v1/analysis_atom/{list,stats,...}`, `/v1/verification_cache/{list,...}`, `/v1/fact_status/{list,...}`, `/v1/cache/audit_log`, `/v1/cache/invalidate`, `/v1/canonicalize`, `/v1/taxonomy`, `/v1/taxonomy/reload`
|
||||
|
||||
### `routes/audit.py` (auth — audit log JSON)
|
||||
- `GET /api/audit` - audit log entries (config/archive mutations)
|
||||
|
||||
### `routes/pages.py` (Jinja HTML — legacy, not the primary UI)
|
||||
- `GET /` - overview
|
||||
- `GET /history`, `GET /history/{request_id}`
|
||||
- `GET /providers`
|
||||
|
|
@ -107,16 +137,22 @@ src/dashboard/
|
|||
pricing.py # estimate_cost(endpoint, tier, provider) - per-provider USD
|
||||
retention.py # 30-day rolling cleanup of request_history
|
||||
api/
|
||||
app.py # FastAPI factory, lifespan (init engine + provider registry), router wiring
|
||||
app.py # FastAPI factory, lifespan (init engine + provider registry), router wiring (routes dual-mounted under /api + /admin-ai/api), SPA static mount
|
||||
dependencies.py # get_session, get_registry, verify_bearer_token, get_username
|
||||
keycloak_auth.py # Keycloak JWT validation (JWKS cache, signature/issuer/exp/role); only used when DASHBOARD_KEYCLOAK_URL set
|
||||
routes/
|
||||
archive.py # claims archive CRUD + promote
|
||||
audit.py # GET /api/audit (audit log JSON)
|
||||
brain_proxy.py # /api/brain/* whitelisted proxy to brain service
|
||||
catalog.py # /api/catalog Model & Extractor catalog CRUD (DB-backed)
|
||||
config.py # KNOWN_KEYS + runtime override CRUD
|
||||
health.py # /health, /ready
|
||||
history.py # /api/history (read-only)
|
||||
ingest.py # POST /api/ingest/event (service-to-service)
|
||||
pages.py # all Jinja HTML pages + HTMX handlers
|
||||
stats.py # /api/stats/{providers,summary,timeline}
|
||||
monitoring.py # /api/monitoring/{services,queues,latency} (AI monitoring)
|
||||
pages.py # all Jinja HTML pages + HTMX handlers (legacy)
|
||||
proxy.py # GET /api/proxy/{module_id}/health (module health proxy)
|
||||
stats.py # /api/stats/{providers,summary,timeline,cost}
|
||||
db/
|
||||
models.py # Base, RequestHistory, ProviderStatsHourly, ConfigOverride, User, AuditLog,
|
||||
# ClaimsArchive, ArticlesArchive, ClaimArticle
|
||||
|
|
@ -194,7 +230,7 @@ Connection string format: `postgresql+asyncpg://USER:PASS@didiAI-dashboard-db:54
|
|||
|
||||
## Reskin plan (FUTURE - NOT done yet)
|
||||
|
||||
- React 19 + MUI 7 + Keycloak SSO **DONE 2026-05-02** (Phase C in `agent-v3/IMPLEMENTATION_PLAN_HIL_BRAIN.md`, detail in `AI_PLATFORM_RESKIN_PLAN.md`)
|
||||
- React 19 + MUI 7 SPA **DONE 2026-05-02**; Keycloak SSO **wired but disabled in this delivery** (staging mode — enable by setting `DASHBOARD_KEYCLOAK_URL`) (Phase C in `agent-v3/IMPLEMENTATION_PLAN_HIL_BRAIN.md`, detail in `AI_PLATFORM_RESKIN_PLAN.md`)
|
||||
- Becomes admin-only (Keycloak realm role)
|
||||
- Bearer-token table retired; existing `role` column may persist for historical audit-log mapping
|
||||
- Brain admin UI (atom browse, force-gold, brain stats) added as a new section in this dashboard during Phase C
|
||||
|
|
@ -214,7 +250,7 @@ cp ../.env.example .env # set DASHBOARD_DB_USER/PASSWORD/NAME + provider keys
|
|||
|
||||
## Ce NU face
|
||||
|
||||
- ~~No SSO yet~~ Keycloak SSO wired (DONE 2026-05-02). Bearer tokens kept as legacy fallback.
|
||||
- Keycloak SSO is wired but **disabled in this delivery** (ships in staging mode, auth bypassed; enable via `DASHBOARD_KEYCLOAK_URL`). Bearer tokens kept as legacy fallback.
|
||||
- No multi-tenant - single shared `users` table, no per-tenant scoping
|
||||
- No public access - binds to internal `didi-network` network, not exposed via Kong/edge
|
||||
- ~~No React frontend yet~~ React 19 SPA at `/admin-ai/` (DONE 2026-05-02). Jinja kept side-by-side until parity.
|
||||
|
|
|
|||
|
|
@ -1,20 +1,42 @@
|
|||
# Dashboard
|
||||
|
||||
Admin dashboard for the didiAI platform. Tracks search provider usage, costs, request history, and exposes runtime configuration.
|
||||
Admin dashboard for the didiAI platform. A single FastAPI service that serves a
|
||||
**React 19 + MUI 7 + Vite single-page app** (the admin UI) plus a JSON API for
|
||||
AI monitoring, a DB-backed Model & Extractor catalog, runtime config, RBAC and
|
||||
audit. It also tracks search-provider usage, costs and request history.
|
||||
|
||||
## What it does
|
||||
|
||||
- **Live quota & billing** — pulls real-time data from SerpAPI, Tavily, Brave, OpenRouter
|
||||
- **Health monitoring** — SearXNG, web-api, vLLM, llama.cpp servers
|
||||
- **Request history** — 30-day rolling log of every gather/search/fetch request with drill-down
|
||||
- **AI monitoring** — live health/status of platform modules (proxy to each
|
||||
module's health endpoint), provider quotas (SerpAPI, Tavily, Brave, LinkUp,
|
||||
Exa, OpenRouter), live KPIs and throughput
|
||||
- **Model & Extractor Catalog** — DB-backed CRUD over registered models and
|
||||
extractors, exposed under `/api/catalog`
|
||||
- **Request history** — 30-day rolling log of every gather/search/fetch request
|
||||
with drill-down
|
||||
- **Cost tracking** — per-provider spend, projections, cost per tier
|
||||
- **Future:** runtime config (toggle providers, change strategies, manage tier caps)
|
||||
- **Runtime config** — config override store (`/api/config`) with schema
|
||||
validation and audit trail
|
||||
- **RBAC** — role-based access; in the shipped build the dashboard runs in
|
||||
**staging mode** with auth bypassed (see below)
|
||||
- **Brain admin** — fact-status browse/override and cache invalidation via a
|
||||
brain proxy
|
||||
|
||||
## Authentication / staging mode
|
||||
|
||||
The build ships in **staging mode** (`DASHBOARD_STAGING_MODE=true`), so all auth
|
||||
is bypassed and Keycloak is **not** active. Keycloak (JWT) is supported but
|
||||
disabled in this delivery: JWT validation only turns on when
|
||||
`DASHBOARD_KEYCLOAK_URL` is set (empty = off). To go to authenticated mode, set
|
||||
`DASHBOARD_KEYCLOAK_URL` (e.g. an SSO URL), set `DASHBOARD_STAGING_MODE=false`,
|
||||
and rebuild the image (Keycloak config is baked into the JS bundle at build
|
||||
time). DB-backed bearer tokens remain as a legacy fallback.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Docker 24+ with Compose V2
|
||||
- PostgreSQL 16 (provided by compose)
|
||||
- Internal network access to web-api, SearXNG, LLM servers
|
||||
- PostgreSQL 16 (provided by compose, `didiAI-dashboard-db` on `:15432`)
|
||||
- Internal network access to the platform modules and provider APIs
|
||||
|
||||
## Quick start
|
||||
|
||||
|
|
@ -24,37 +46,49 @@ cp ../.env.example .env # edit with your secrets
|
|||
./deploy.sh up
|
||||
```
|
||||
|
||||
Dashboard is now running at http://localhost:51300
|
||||
The dashboard API listens on `http://localhost:51300`. The admin SPA is served
|
||||
at `http://localhost:51300/admin-ai/`.
|
||||
|
||||
## Endpoints
|
||||
|
||||
### Web UI
|
||||
- `/` — overview with KPIs and provider grid
|
||||
- `/providers` — detailed provider cards + raw table
|
||||
- `/history` — filterable request history
|
||||
- `/history/{request_id}` — full request detail with stages
|
||||
### Admin UI (SPA)
|
||||
|
||||
### JSON API
|
||||
- `GET /health` — liveness
|
||||
- `/admin-ai/` — React 19 + MUI SPA (overview, live status, history, cost,
|
||||
providers, catalog, config/schema, audit, brain admin). All client-side routes
|
||||
under `/admin-ai/` are served by the SPA with index fallback.
|
||||
|
||||
> Legacy Jinja pages (`/`, `/providers`, `/history`, …) still exist server-side
|
||||
> but are not the primary UI; the SPA at `/admin-ai/` is the delivered UI.
|
||||
|
||||
### JSON API (selected)
|
||||
|
||||
- `GET /health` — liveness + DB ping (health JSON at root)
|
||||
- `GET /api/stats/providers` — live provider stats
|
||||
- `GET /api/stats/summary?hours=24` — aggregated counters
|
||||
- `GET /api/stats/timeline?hours=24` — hourly buckets for charts
|
||||
- `GET /api/stats/cost` — cost + projection
|
||||
- `GET /api/history?limit=50&tier=premium` — filtered history
|
||||
- `GET /api/history/{request_id}` — single request with full payload
|
||||
- `GET|POST|PUT|DELETE /api/catalog/...` — Model & Extractor catalog CRUD
|
||||
- `GET /api/config`, `PUT|DELETE /api/config/{key}` — runtime config overrides
|
||||
- `GET /api/proxy/{module_id}/health` — module health proxy (AI monitoring)
|
||||
- `/api/brain/*` — brain proxy (fact status, cache invalidation)
|
||||
- `POST /api/ingest/event` — receives events from web-api middleware
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
web-api ────► POST /api/ingest/event ────► dashboard-api ────► PostgreSQL
|
||||
│
|
||||
browser ────► GET /admin-ai/ (React SPA) ──┘ │
|
||||
├─ reads provider APIs live
|
||||
└─ serves UI via Jinja2+HTMX
|
||||
├─ proxies module health + brain
|
||||
└─ serves SPA static (index fallback)
|
||||
```
|
||||
|
||||
## Tech stack
|
||||
|
||||
- FastAPI + Pydantic
|
||||
- SQLAlchemy 2.0 async + asyncpg
|
||||
- Jinja2 + HTMX + Alpine.js + Tailwind (zero build step)
|
||||
- FastAPI + Pydantic (pydantic-settings, env prefix `DASHBOARD_`)
|
||||
- SQLAlchemy 2.0 async + asyncpg (tables created via `create_all` at startup)
|
||||
- React 19 + MUI 7 + Vite 7 + react-router 7 + TanStack Query + Recharts
|
||||
(built into `web_dist/`, served as static SPA)
|
||||
- PostgreSQL 16
|
||||
|
|
|
|||
|
|
@ -76,9 +76,15 @@ export default function Settings() {
|
|||
});
|
||||
const health = useQuery({
|
||||
queryKey: ['settings', 'health'],
|
||||
// Dashboard liveness probe lives at root /health (next to /ready), not
|
||||
// /api/health — the latter returns 404 and shows "unknown" in the UI.
|
||||
queryFn: () => apiGet<HealthResponse>('/health'),
|
||||
// Dashboard liveness probe lives at the ORIGIN root /health (next to
|
||||
// /ready). apiGet() would prepend BASE_PATH (/admin-ai/health), which the
|
||||
// SPA static fallback answers with index.html → JSON parse yields no
|
||||
// `status` → "unknown". Fetch the absolute root path directly instead.
|
||||
queryFn: async (): Promise<HealthResponse> => {
|
||||
const res = await fetch('/health', { headers: { Accept: 'application/json' } });
|
||||
if (!res.ok) throw new Error(`health ${res.status}`);
|
||||
return res.json();
|
||||
},
|
||||
refetchInterval: 30_000,
|
||||
});
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue