Livrare LOT 1 - Didi

2026-06-25 14:13:25 -07:00 · 2026-06-25 14:13:25 -07:00 · 5380c3fc63
commit 5380c3fc63
990 changed files with 133308 additions and 0 deletions
--- a/ai_platform/modules/catalog-api/INDEX.md
+++ b/ai_platform/modules/catalog-api/INDEX.md
@ -0,0 +1,146 @@
+# catalog-api — INDEX
+
+Catalog API for the DIDI AI platform. It is a **service registry and discovery gateway**: it aggregates `/v1/info` responses from each ML component (LLM, Audio, Video, Web) and re-exposes a unified view (components, models, functions, status) plus a merged OpenAPI 3.1 spec with Swagger UI / ReDoc. Production port `11000`, container `didiAI-catalog-api`. Read-only HTTP aggregator — no database, no writes.
+
+- **Stack:** Python 3.11, FastAPI, httpx (async), pydantic-settings, uv, uvicorn
+- **Container:** `didiAI-catalog-api`
+- **Internal URL:** `http://didiAI-catalog-api:11000`
+- **Production host:** `http://10.11.10.42` (per `CATALOG_EXTERNAL_URL` in `deploy/.env.example`)
+- **Network:** Docker external network `didi-network` (shared with the other ai_platform modules)
+- **Sister CLAUDE.md (platform):** `/home/admin365/didi_mono/ai_platform/CLAUDE.md`
+
+> Note: despite the name, this module is **not** a knowledge-graph / atom catalog. It is a *service catalog* (think "service registry" in the microservices sense). It does not talk to PostgreSQL, Atomic, Redis, or didi-brain.
+
+---
+
+## Ce face
+
+- Aggregates static metadata from each ML component by calling `GET /v1/info` on the configured backends.
+- Returns a single unified response with:
+  - **Resources** — component descriptor (`name`, `slug`, `resource_type`, ...).
+  - **Models** — every model exposed by every backend (LLM, Whisper, vision, etc.).
+  - **Functions** — every endpoint/function each backend advertises.
+- Probes liveness of every backend and reports aggregated health (`healthy` / `degraded` / `unhealthy`).
+- Builds a **merged OpenAPI 3.1.0 document** (paths prefixed with `/{component_id}`, schemas prefixed with `{component_id}_`) and serves it as JSON, Swagger UI, and ReDoc — so a frontend or backend can consume a single contract for all GPU services.
+- Tolerant to partial outages: if a component is unreachable it is recorded under `errors` and skipped, the rest of the catalog still serves.
+
+---
+
+## API endpoints
+
+Source: `src/catalog_api/app.py` and `API.md`.
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/health` | Liveness probe (`{"status":"ok"}`). |
+| GET | `/v1/components` | List all registered components with full metadata (resource + models + functions). |
+| GET | `/v1/components/{component_id}` | Full metadata for one component (404 if unknown). |
+| GET | `/v1/models` | Flattened list of every model across components. |
+| GET | `/v1/functions` | Flattened list of every function/endpoint across components. |
+| GET | `/v1/status` | Aggregated reachability + health of all components (`healthy_count`/`total_count`). |
+| GET | `/v1/openapi` | Merged OpenAPI 3.1 spec (all components, prefixed). |
+| GET | `/v1/docs` | Swagger UI for the merged spec. |
+| GET | `/v1/redoc` | ReDoc for the merged spec. |
+| GET | `/v1/openapi/component/{component_id}` | Raw OpenAPI spec of a single component (passthrough). |
+
+Status values from `/v1/status`: `healthy` (all reachable), `degraded` (some reachable), `unhealthy` (none reachable).
+
+When accessed through the gateway, Bearer-token auth is enforced by nginx (`<host>:11000/catalog/`); direct access is unauthenticated by design.
+
+---
+
+## Architecture
+
+```
+-----------------------------------------------------+
+|  Catalog API (:11000)  — didiAI-catalog-api         |
+|  - Calls /v1/info on each backend                   |
+|  - Merges OpenAPI specs, exposes Swagger UI / ReDoc |
+|  - Stateless, no DB                                 |
+-------+----------+----------+----------+------------+
+        |          |          |          |
+        v          v          v          v
+   LLM 14011  Audio 54300  Video 54600  Web 51100
+   didiAI-llm-api  -audio-api  -video-api  -web-api
+```
+
+- **Pure aggregator** — no persistence, no caching layer; a fresh fan-out happens per request via `httpx.AsyncClient`.
+- **Component list is config-driven** (`CatalogSettings.get_components()` in `settings.py`): a component with an empty URL is silently dropped (used today to disable Video by setting `CATALOG_VIDEO_URL=""`).
+- **External-vs-internal URL split** — internal URLs (`*_URL`) are used for live calls inside the Docker network; the `CATALOG_EXTERNAL_URL` + `*_EXTERNAL_PORT` pair is the public base URL injected into the merged OpenAPI `servers:` so that external clients hit the right hostnames/ports.
+
+### How it is consumed
+
+- **Frontend / API gateway** — fetches `/v1/openapi` to expose Swagger UI for the whole platform; fetches `/v1/status` for a system-health widget.
+- **Backend integrations** — pull `/v1/components` to populate their own catalog tables (`catalog.resources`, `catalog.models`, `catalog.functions`), as illustrated in `README.md` § Use Cases.
+- **Service discovery** — clients query `/v1/models` to find a model by `model_type` (e.g., all `vision` models) without hard-coding hosts.
+
+---
+
+## Structura fișiere
+
+```
+catalog-api/
+├── README.md              Overview, quick-start, configuration, use cases
+├── API.md                 Endpoint reference (request/response shapes)
+├── INDEX.md               This file
+├── pyproject.toml         Hatchling package, FastAPI/httpx/pydantic-settings deps
+└── src/catalog_api/
+    ├── __init__.py        version = "0.1.0"
+    ├── app.py             FastAPI app — all endpoints + OpenAPI merger (~21 KB, single module)
+    └── settings.py        CatalogSettings (env prefix CATALOG_) and Component model
+└── deploy/
+    ├── Dockerfile         Multi-stage build: python:3.11.12-slim + uv 0.10, runs uvicorn
+    ├── docker-compose.yml Defines didiAI-catalog-api on didi-network, expose:11000 only
+    ├── deploy.sh          Wrapper: loads .env, validates CATALOG_EXTERNAL_URL, runs compose
+    ├── .env.example       Documented environment variables
+    └── .env               Local environment (CATALOG_EXTERNAL_URL=...)
+```
+
+Implementation footprint is tiny: one `app.py` (all endpoints + OpenAPI merger live there) plus one `settings.py`.
+
+---
+
+## Configuration
+
+All settings come from environment variables with prefix `CATALOG_` (see `src/catalog_api/settings.py`).
+
+| Variable | Default | Purpose |
+|----------|---------|---------|
+| `CATALOG_HOST` | `0.0.0.0` | Bind address. |
+| `CATALOG_PORT` | `11000` | Bind port. |
+| `CATALOG_LOG_LEVEL` | `INFO` | Python logging level. |
+| `CATALOG_EXTERNAL_URL` | **required** | Public base URL (e.g., `http://10.11.10.42`) injected into the merged OpenAPI `servers:`. `deploy.sh` aborts if missing. |
+| `CATALOG_LLM_URL` | `http://didiAI-llm-api:14011` | LLM Inference internal URL. |
+| `CATALOG_AUDIO_URL` | `http://didiAI-audio-api:54300` | Audio API internal URL. |
+| `CATALOG_VIDEO_URL` | *(empty)* | Video Analysis internal URL — empty string disables Video. |
+| `CATALOG_WEB_URL` | `http://didiAI-web-api:51100` | Web API internal URL. |
+| `CATALOG_LLM_EXTERNAL_PORT` | `14011` | External port advertised in the merged OpenAPI for LLM. |
+| `CATALOG_AUDIO_EXTERNAL_PORT` | `54300` | External port for Audio. |
+| `CATALOG_VIDEO_EXTERNAL_PORT` | `54600` | External port for Video. |
+| `CATALOG_WEB_EXTERNAL_PORT` | `51100` | External port for Web. |
+| `CATALOG_COMPONENT_TIMEOUT` | `10` | Per-call httpx timeout in seconds. |
+
+---
+
+## Deployment
+
+- **Docker compose** (`deploy/docker-compose.yml`): builds `didiai-catalog-api`, attaches to external network `didi-network`, only `expose: 11000` (no host port — traffic comes through the platform gateway). Healthcheck hits `http://localhost:11000/health` every 30 s, restart policy `unless-stopped`.
+- **Dockerfile** (`deploy/Dockerfile`): two-stage build using `ghcr.io/astral-sh/uv:0.10` for dependency install, then a slim `python:3.11.12-slim` runtime that runs `python -m uvicorn catalog_api.app:app`.
+- **Helper script** (`deploy/deploy.sh`): loads `.env`, validates `CATALOG_EXTERNAL_URL`, supports `--detach`, `--down`, `--logs`.
+- **Local dev** (per README.md): `uv sync && uv run python -m uvicorn catalog_api.app:app --host 0.0.0.0 --port 11000` (requires the listed components reachable on the network).
+
+---
+
+## Related modules
+
+This service stands on top of the rest of the `ai_platform/modules/*` family — they are its data sources:
+
+- `llm-inference` (port `14011`, container `didiAI-llm-api`) — chat, embeddings, rerank.
+- `audio` (port `54300`, container `didiAI-audio-api`) — transcription / TTS.
+- `video-analysis` (port `54600`, container `didiAI-video-api`) — vision pipelines (currently disabled by default in the .env example).
+- `web` (port `51100`, container `didiAI-web-api`) — fact-check / web crawler.
+- `dashboard` — primary frontend consumer of the merged OpenAPI / `/v1/status`.
+
+It is **independent** of:
+- `didi-brain`, Atomic / knowledge-graph services, PostgreSQL, Redis — none of these are accessed.
+- The orchestration-layer (`agent-v3`) does not currently consume this catalog; it talks to the GPU services directly.