Gateway (didiAI-gateway)

API gateway for the AI platform: a thin Nginx reverse proxy that acts as the single externally exposed entry point for all didiAI internal services (LLM inference, audio transcription, web fact-checking, catalog, embeddings, rerank). Every request except /health requires Bearer token authentication enforced via an nginx map block. Currently in a restart loop because several upstream containers it references do not exist on this host (see "Current status" below).

Stack: Nginx 1.27-alpine, no Python, no custom code (config-only module).
URL (intended): http://<host>:11000 — listens on a single port, 11000, mapped 1:1 to the host. No TLS at this layer.
Container: didiAI-gateway
Network: didi-network (external, shared across AI platform modules)
Module path: /home/admin365/didi_mono/ai_platform/modules/gateway/

Ce face

Single ingress point for all didiAI services, with three responsibilities:

Bearer-token gatekeeping — every request must carry Authorization: Bearer <GATEWAY_API_TOKEN>; rejected with 401 {"error":"unauthorized",...} otherwise. The check is done with an nginx map from $http_authorization to $auth_status. The token value is injected at container start with envsubst against nginx.conf.template.
Path-based reverse proxy — strips the /<service>/ prefix and forwards to the matching internal upstream container by Docker DNS name. Adds standard forwarding headers (Host, X-Real-IP, X-Forwarded-For, auto-generated X-Request-ID).
Streaming + large upload tuning — proxy_buffering off + HTTP/1.1 + chunked transfer for the LLM SSE route, and client_max_body_size 500M + client_body_buffer_size 10M for audio uploads.

There is no rate limiting, no request body inspection, no JWT/Keycloak — just a static shared secret. This gateway is internal-only; the public-facing edge is Kong (separate, unrelated component).

API endpoints / routes

All routes from deploy/nginx.conf.template:

Path prefix	Auth	Upstream	Notes
`GET /health`	none	nginx direct (returns `{"status":"ok","service":"didiAI-gateway"}`)	for healthcheck
`/llm/*`	Bearer	`didiAI-llm-api:14011`	SSE streaming: `proxy_buffering off`, `chunked_transfer_encoding on`, HTTP/1.1, `Connection: ''`
`/audio/*`	Bearer	`didiAI-audio-api:54300`	`client_body_buffer_size 10M` for large uploads
`/web/*`	Bearer	`didiAI-web-api:51100`	plain proxy_pass
`/catalog/*`	Bearer	`didiAI-catalog-api:11000`	plain proxy_pass; note: catalog also listens on 11000 internally
`/embeddings/*`	Bearer	`didiAI-embeddings-api:14100`	OpenAI-compatible API
`/rerank/*`	Bearer	`didiAI-rerank-api:14200`	Cohere/Jina-compatible API
`/` (anything else)	none	nginx direct	returns `404 {"error":"not_found","routes":[...]}` listing the valid prefixes

Trailing slash on proxy_pass http://upstream/; strips the /<service>/ prefix when forwarding (so /llm/v1/chat becomes /v1/chat upstream).

Current status

BROKEN / restart loop as of 2026-05-01. docker logs didiAI-gateway shows nginx failing to start with:

[emerg] host not found in upstream "didiAI-llm-api:14011" in /etc/nginx/nginx.conf:35

Root cause: the nginx upstreams resolve hostnames at config load (not per-request), so any missing upstream container kills the whole gateway. On this host only 2 of the 6 upstreams are running:

Running: didiAI-catalog-api, didiAI-web-api.
Missing: didiAI-llm-api, didiAI-audio-api, didiAI-embeddings-api, didiAI-rerank-api.

Fix options (pick one before redeploying):

Start the missing service modules (llm-inference, audio, embeddings, rerank) on this host so the names resolve.
Edit nginx.conf.template and remove (or comment out) the upstream blocks + location blocks for services not deployed locally.
Switch the upstream definitions to lazy-resolution form (set $upstream "didiAI-llm-api:14011"; proxy_pass http://$upstream/; plus a resolver directive) so missing names fail per-request instead of bringing the whole gateway down.

Investigation commands:

docker logs didiAI-gateway
docker ps --filter "name=didiAI-" --format "table {{.Names}}\t{{.Status}}"

Structura fisiere

modules/gateway/
├── README.md                       # User-facing docs (routes, auth, quick start)
├── INDEX.md                        # This file
└── deploy/
    ├── docker-compose.yml          # nginx:1.27-alpine, port 11000:11000, didi-network network
    ├── nginx.conf.template         # 175 lines: map auth, 6 upstreams, 7 locations + 404 fallback
    ├── deploy.sh                   # Bash wrapper: --detach / --down / --logs, fail-fast on missing GATEWAY_API_TOKEN
    ├── .env.example                # Template (only GATEWAY_API_TOKEN)
    └── .env                        # Active config (GATEWAY_API_TOKEN value)

No src/, no pyproject.toml — this module is pure config. There is no application code.

Configuration

Single env var, declared in deploy/.env (or exported before running deploy.sh):

Variable	Required	Description
`GATEWAY_API_TOKEN`	yes	Bearer token; clients must send `Authorization: Bearer <value>`. Generate with `python3 -c "import secrets; print(secrets.token_urlsafe(32))"`.

docker-compose.yml injects the token into the container env, then the entrypoint runs:

envsubst '$GATEWAY_API_TOKEN' < /etc/nginx/nginx.conf.template > /etc/nginx/nginx.conf && nginx -g 'daemon off;'

Tunables hard-coded in nginx.conf.template (no env vars, edit + redeploy to change):

proxy_connect_timeout 60s
proxy_send_timeout 300s
proxy_read_timeout 600s
client_max_body_size 500M
client_body_buffer_size 10M (audio route only)
worker_connections 1024

How services connect

The AI platform mixes two patterns; this gateway is opt-in aggregation, not platform-wide:

Service	Port	Routed via gateway?	Notes
didi-brain (`brain_api`)	`:8090`	NO — direct	Brain is exposed directly on its own port; not registered in gateway upstreams
didiAI-dashboard	`:51300`	NO — direct	Internal monitoring UI, exposed directly
didiAI-web-api	`:51100`	YES (`/web/`) and direct	Both paths work
didiAI-llm-api	`:14011`	YES (`/llm/`) — required for SSE	LLM router; gateway adds streaming-friendly proxy settings
didiAI-embeddings-api	`:14100`	YES (`/embeddings/`) and direct	OpenAI-compatible
didiAI-rerank-api	`:14200`	YES (`/rerank/`) and direct	Cohere/Jina-compatible
didiAI-audio-api	`:54300`	YES (`/audio/`) — recommended for large uploads	gateway sets a 10 MB body buffer
didiAI-catalog-api	`:11000`	YES (`/catalog/`)	Catalog also listens on 11000 internally — same number as gateway, different network endpoint

So the gateway aggregates the inference/IO services (LLM, embeddings, rerank, audio, web, catalog) under one host:port, while observability/orchestration components (brain, dashboard) stay on their own ports.

This gateway is independent from the backend stack's Kong/Keycloak: backend services on agent-v3 go through the public Kong cluster (10.11.10.175 → DP1/DP2) with JWT from the SSO cluster; this didiAI-gateway lives entirely inside the AI-platform Docker network and uses a static Bearer token. The two systems do not call each other through their gateways — agent-v3 talks to AI-platform services directly by container DNS name on the shared Docker network when co-located, or over the LAN otherwise.

Deployment

cd /home/admin365/didi_mono/ai_platform/modules/gateway/deploy
cp .env.example .env
# edit .env, set GATEWAY_API_TOKEN
./deploy.sh -d                    # docker compose up -d
./deploy.sh --logs                # tail logs
./deploy.sh --down                # stop + remove

Manual equivalent: docker compose up -d from deploy/. Network didi-network must exist beforehand (it is created by another module's compose, typically catalog or dashboard).

Healthcheck: wget --spider http://127.0.0.1:11000/health every 30 s.

AI platform overview: /home/admin365/didi_mono/ai_platform/CLAUDE.md (and module-level README.md in each sibling: dashboard/, web/, embeddings/, rerank/, didi_brain/brain_api/).
Upstream service modules referenced by this gateway:
- /home/admin365/didi_mono/ai_platform/modules/web/ (running)
- /home/admin365/didi_mono/ai_platform/modules/embeddings/ (not running on this host)
- /home/admin365/didi_mono/ai_platform/modules/rerank/ (not running on this host)
- llm-inference, audio, catalog modules (catalog is running; llm-inference + audio are not)
Distinct from backend Kong — see backend/services/gateway-auth-layer/didiKong/ and project_kong_migration_2026_04.md in the user memory. The backend's Kong cluster is the public edge for agent-v3; didiAI-gateway is internal-only for AI-platform services.

9.1 KiB Raw Blame History