didi-lot1-ai/ai_platform/modules/gateway/INDEX.md

# Gateway (didiAI-gateway)

API gateway for the AI platform: a thin **Nginx reverse proxy** that acts as the single externally exposed entry point for all didiAI internal services (LLM inference, audio transcription, web fact-checking, catalog, embeddings, rerank). Every request except `/health` requires Bearer token authentication enforced via an `nginx map` block. It currently starts cleanly: only the upstreams whose containers exist on this host (`web`, `catalog`) are active; the upstreams for services not deployed here are commented out in `nginx.conf.template` (see "Current status" below).

- **Stack**: Nginx 1.27-alpine, no Python, no custom code (config-only module).
- **URL** (intended): `http://<host>:11000` — listens on a single port, `11000`, mapped 1:1 to the host. No TLS at this layer.
- **Container**: `didiAI-gateway`
- **Network**: `didi-network` (external, shared across AI platform modules)
- **Module path**: `/home/admin365/didi_mono/ai_platform/modules/gateway/`

## Ce face

Single ingress point for all didiAI services, with three responsibilities:

1. **Bearer-token gatekeeping** — every request must carry `Authorization: Bearer <GATEWAY_API_TOKEN>`; rejected with `401 {"error":"unauthorized",...}` otherwise. The check is done with an `nginx map` from `$http_authorization` to `$auth_status`. The token value is injected at container start with `envsubst` against `nginx.conf.template`.
2. **Path-based reverse proxy** — strips the `/<service>/` prefix and forwards to the matching internal upstream container by Docker DNS name. Adds standard forwarding headers (`Host`, `X-Real-IP`, `X-Forwarded-For`, auto-generated `X-Request-ID`).
3. **Streaming + large upload tuning** — `proxy_buffering off` + HTTP/1.1 + chunked transfer for the LLM SSE route, and `client_max_body_size 500M` + `client_body_buffer_size 10M` for audio uploads.

There is no rate limiting, no request body inspection, no JWT/Keycloak — just a static shared secret. This gateway is internal-only; the public-facing edge is Kong (separate, unrelated component).

## API endpoints / routes

All routes from `deploy/nginx.conf.template`:

| Path prefix | Auth | Upstream | Status | Notes |
|-------------|------|----------|--------|-------|
| `GET /health` | none | nginx direct (returns `{"status":"ok","service":"didiAI-gateway"}`) | active | for healthcheck |
| `/web/*` | Bearer | `didiAI-web-api:51100` | active | plain proxy_pass |
| `/catalog/*` | Bearer | `didiAI-catalog-api:11000` | active | plain proxy_pass; note: catalog also listens on 11000 internally |
| `/llm/*` | Bearer | `didiAI-llm-api:14011` | disabled (upstream + location commented) | SSE streaming: `proxy_buffering off`, `chunked_transfer_encoding on`, HTTP/1.1, `Connection: ''` |
| `/audio/*` | Bearer | `didiAI-audio:54300` | disabled (upstream + location commented) | `client_body_buffer_size 10M` for large uploads |
| `/embeddings/*` | Bearer | `didiAI-embeddings-api:14100` | disabled (upstream + location commented) | OpenAI-compatible API |
| `/rerank/*` | Bearer | `didiAI-rerank-api:14200` | disabled (upstream + location commented) | Cohere/Jina-compatible API |
| `/` (anything else) | none | nginx direct | active | returns `404 {"error":"not_found","routes":["/web/","/catalog/","/health"]}` listing the active prefixes |

The disabled routes are kept commented in `nginx.conf.template`; re-enable the matching `upstream` + `location` blocks once those containers run on this host.

Trailing slash on `proxy_pass http://upstream/;` strips the `/<service>/` prefix when forwarding (so `/llm/v1/chat` becomes `/v1/chat` upstream).

## Current status

**RESOLVED — gateway starts cleanly.** The earlier restart loop (`[emerg] host not found in upstream "didiAI-llm-api:14011"`) was caused by nginx resolving all upstream hostnames at config load (not per-request), so any missing upstream container aborted startup.

Fix applied: in `nginx.conf.template` the `upstream` + `location` blocks for the services not deployed on this host are commented out (option 2 below). The gateway now starts with only the active upstreams:

- Active: `didiAI-web-api` (`/web/`), `didiAI-catalog-api` (`/catalog/`), plus the `/health` and `/` (404) direct locations.
- Commented out (re-enable when their containers run here): `didiAI-llm-api`, `didiAI-audio`, `didiAI-embeddings-api`, `didiAI-rerank-api`.

Other ways the same problem could be addressed if you prefer to keep all blocks listed:

1. Start the missing service modules (`llm-inference`, `audio`, `embeddings`, `rerank`) on this host so the names resolve.
2. (applied) Comment out the `upstream` + `location` blocks for services not deployed locally.
3. Switch the upstream definitions to lazy-resolution form (`set $upstream "didiAI-llm-api:14011"; proxy_pass http://$upstream/;` plus a `resolver` directive) so missing names fail per-request instead of bringing the whole gateway down.

Investigation commands:
```bash
docker logs didiAI-gateway
docker ps --filter "name=didiAI-" --format "table {{.Names}}\t{{.Status}}"
```

## Structura fisiere

```
modules/gateway/
├── README.md                       # User-facing docs (routes, auth, quick start)
├── INDEX.md                        # This file
└── deploy/
    ├── docker-compose.yml          # nginx:1.27-alpine, port 11000:11000, didi-network network
    ├── nginx.conf.template         # ~169 lines: map auth, 2 active upstreams (4 commented), 3 active locations (/web/, /catalog/, /health) + 404 fallback (4 locations commented)
    ├── deploy.sh                   # Bash wrapper: --detach / --down / --logs, fail-fast on missing GATEWAY_API_TOKEN
    ├── .env.example                # Template (only GATEWAY_API_TOKEN)
    └── .env                        # Active config (GATEWAY_API_TOKEN value)
```

No `src/`, no `pyproject.toml` — this module is pure config. There is no application code.

## Configuration

Single env var, declared in `deploy/.env` (or exported before running `deploy.sh`):

| Variable | Required | Description |
|----------|----------|-------------|
| `GATEWAY_API_TOKEN` | yes | Bearer token; clients must send `Authorization: Bearer <value>`. Generate with `python3 -c "import secrets; print(secrets.token_urlsafe(32))"`. |

`docker-compose.yml` injects the token into the container env, then the entrypoint runs:

```sh
envsubst '$GATEWAY_API_TOKEN' < /etc/nginx/nginx.conf.template > /etc/nginx/nginx.conf && nginx -g 'daemon off;'
```

Tunables hard-coded in `nginx.conf.template` (no env vars, edit + redeploy to change):

- `proxy_connect_timeout 60s`
- `proxy_send_timeout 300s`
- `proxy_read_timeout 600s`
- `client_max_body_size 500M`
- `client_body_buffer_size 10M` (audio route only)
- `worker_connections 1024`

## How services connect

The AI platform mixes two patterns; this gateway is **opt-in aggregation**, not platform-wide:

| Service | Port | Routed via gateway? | Notes |
|---------|------|--------------------|-------|
| didi-brain (`brain_api`) | `:8090` | **NO** — direct | Brain is exposed directly on its own port; not registered in gateway upstreams |
| didiAI-dashboard | `:51300` | **NO** — direct | Internal monitoring UI, exposed directly |
| didiAI-web-api | `:51100` | YES (`/web/`) and direct | Both paths work |
| didiAI-llm-api | `:14011` | YES (`/llm/`) — required for SSE | LLM router; gateway adds streaming-friendly proxy settings |
| didiAI-embeddings-api | `:14100` | YES (`/embeddings/`) and direct | OpenAI-compatible |
| didiAI-rerank-api | `:14200` | YES (`/rerank/`) and direct | Cohere/Jina-compatible |
| didiAI-audio | `:54300` | YES (`/audio/`) — recommended for large uploads | gateway sets a 10 MB body buffer |
| didiAI-catalog-api | `:11000` | YES (`/catalog/`) | Catalog also listens on 11000 internally — same number as gateway, different network endpoint |

So the gateway aggregates the **inference/IO services** (LLM, embeddings, rerank, audio, web, catalog) under one host:port, while observability/orchestration components (brain, dashboard) stay on their own ports.

This gateway is **independent from the backend stack's Kong/Keycloak**: backend services on `agent-v3` go through the public Kong cluster (10.11.10.175 → DP1/DP2) with JWT from the SSO cluster; this `didiAI-gateway` lives entirely inside the AI-platform Docker network and uses a static Bearer token. The two systems do not call each other through their gateways — agent-v3 talks to AI-platform services directly by container DNS name on the shared Docker network when co-located, or over the LAN otherwise.

## Deployment

```bash
cd /home/admin365/didi_mono/ai_platform/modules/gateway/deploy
cp .env.example .env
# edit .env, set GATEWAY_API_TOKEN
./deploy.sh -d                    # docker compose up -d
./deploy.sh --logs                # tail logs
./deploy.sh --down                # stop + remove
```

Manual equivalent: `docker compose up -d` from `deploy/`. Network `didi-network` must exist beforehand (it is created by another module's compose, typically `catalog` or `dashboard`).

Healthcheck: `wget --spider http://127.0.0.1:11000/health` every 30 s.

## Related

- AI platform overview: `/home/admin365/didi_mono/ai_platform/CLAUDE.md` (and module-level `README.md` in each sibling: `dashboard/`, `web/`, `embeddings/`, `rerank/`, `didi_brain/brain_api/`).
- Upstream service modules referenced by this gateway:
  - `/home/admin365/didi_mono/ai_platform/modules/web/` (running)
  - `/home/admin365/didi_mono/ai_platform/modules/embeddings/` (not running on this host)
  - `/home/admin365/didi_mono/ai_platform/modules/rerank/` (not running on this host)
  - `llm-inference`, `audio`, `catalog` modules (catalog is running; llm-inference + audio are not)
- **Distinct from backend Kong** — see `backend/services/gateway-auth-layer/didiKong/` and `project_kong_migration_2026_04.md` in the user memory. The backend's Kong cluster is the public edge for `agent-v3`; `didiAI-gateway` is internal-only for AI-platform services.