clossers/didi-lot1-ai
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
didi-lot1-ai/ai_platform/modules/gateway/INDEX.md

9.8 KiB
Raw Blame History

Gateway (didiAI-gateway)

API gateway for the AI platform: a thin Nginx reverse proxy that acts as the single externally exposed entry point for all didiAI internal services (LLM inference, audio transcription, web fact-checking, catalog, embeddings, rerank). Every request except /health requires Bearer token authentication enforced via an nginx map block. It currently starts cleanly: only the upstreams whose containers exist on this host (web, catalog) are active; the upstreams for services not deployed here are commented out in nginx.conf.template (see "Current status" below).

  • Stack: Nginx 1.27-alpine, no Python, no custom code (config-only module).
  • URL (intended): http://<host>:11000 — listens on a single port, 11000, mapped 1:1 to the host. No TLS at this layer.
  • Container: didiAI-gateway
  • Network: didi-network (external, shared across AI platform modules)
  • Module path: /home/admin365/didi_mono/ai_platform/modules/gateway/

Ce face

Single ingress point for all didiAI services, with three responsibilities:

  1. Bearer-token gatekeeping — every request must carry Authorization: Bearer <GATEWAY_API_TOKEN>; rejected with 401 {"error":"unauthorized",...} otherwise. The check is done with an nginx map from $http_authorization to $auth_status. The token value is injected at container start with envsubst against nginx.conf.template.
  2. Path-based reverse proxy — strips the /<service>/ prefix and forwards to the matching internal upstream container by Docker DNS name. Adds standard forwarding headers (Host, X-Real-IP, X-Forwarded-For, auto-generated X-Request-ID).
  3. Streaming + large upload tuning — proxy_buffering off + HTTP/1.1 + chunked transfer for the LLM SSE route, and client_max_body_size 500M + client_body_buffer_size 10M for audio uploads.

There is no rate limiting, no request body inspection, no JWT/Keycloak — just a static shared secret. This gateway is internal-only; the public-facing edge is Kong (separate, unrelated component).

API endpoints / routes

All routes from deploy/nginx.conf.template:

Path prefix Auth Upstream Status Notes
GET /health none nginx direct (returns {"status":"ok","service":"didiAI-gateway"}) active for healthcheck
/web/* Bearer didiAI-web-api:51100 active plain proxy_pass
/catalog/* Bearer didiAI-catalog-api:11000 active plain proxy_pass; note: catalog also listens on 11000 internally
/llm/* Bearer didiAI-llm-api:14011 disabled (upstream + location commented) SSE streaming: proxy_buffering off, chunked_transfer_encoding on, HTTP/1.1, Connection: ''
/audio/* Bearer didiAI-audio:54300 disabled (upstream + location commented) client_body_buffer_size 10M for large uploads
/embeddings/* Bearer didiAI-embeddings-api:14100 disabled (upstream + location commented) OpenAI-compatible API
/rerank/* Bearer didiAI-rerank-api:14200 disabled (upstream + location commented) Cohere/Jina-compatible API
/ (anything else) none nginx direct active returns 404 {"error":"not_found","routes":["/web/","/catalog/","/health"]} listing the active prefixes

The disabled routes are kept commented in nginx.conf.template; re-enable the matching upstream + location blocks once those containers run on this host.

Trailing slash on proxy_pass http://upstream/; strips the /<service>/ prefix when forwarding (so /llm/v1/chat becomes /v1/chat upstream).

Current status

RESOLVED — gateway starts cleanly. The earlier restart loop ([emerg] host not found in upstream "didiAI-llm-api:14011") was caused by nginx resolving all upstream hostnames at config load (not per-request), so any missing upstream container aborted startup.

Fix applied: in nginx.conf.template the upstream + location blocks for the services not deployed on this host are commented out (option 2 below). The gateway now starts with only the active upstreams:

  • Active: didiAI-web-api (/web/), didiAI-catalog-api (/catalog/), plus the /health and / (404) direct locations.
  • Commented out (re-enable when their containers run here): didiAI-llm-api, didiAI-audio, didiAI-embeddings-api, didiAI-rerank-api.

Other ways the same problem could be addressed if you prefer to keep all blocks listed:

  1. Start the missing service modules (llm-inference, audio, embeddings, rerank) on this host so the names resolve.
  2. (applied) Comment out the upstream + location blocks for services not deployed locally.
  3. Switch the upstream definitions to lazy-resolution form (set $upstream "didiAI-llm-api:14011"; proxy_pass http://$upstream/; plus a resolver directive) so missing names fail per-request instead of bringing the whole gateway down.

Investigation commands:

docker logs didiAI-gateway
docker ps --filter "name=didiAI-" --format "table {{.Names}}\t{{.Status}}"

Structura fisiere

modules/gateway/
├── README.md                       # User-facing docs (routes, auth, quick start)
├── INDEX.md                        # This file
└── deploy/
    ├── docker-compose.yml          # nginx:1.27-alpine, port 11000:11000, didi-network network
    ├── nginx.conf.template         # ~169 lines: map auth, 2 active upstreams (4 commented), 3 active locations (/web/, /catalog/, /health) + 404 fallback (4 locations commented)
    ├── deploy.sh                   # Bash wrapper: --detach / --down / --logs, fail-fast on missing GATEWAY_API_TOKEN
    ├── .env.example                # Template (only GATEWAY_API_TOKEN)
    └── .env                        # Active config (GATEWAY_API_TOKEN value)

No src/, no pyproject.toml — this module is pure config. There is no application code.

Configuration

Single env var, declared in deploy/.env (or exported before running deploy.sh):

Variable Required Description
GATEWAY_API_TOKEN yes Bearer token; clients must send Authorization: Bearer <value>. Generate with python3 -c "import secrets; print(secrets.token_urlsafe(32))".

docker-compose.yml injects the token into the container env, then the entrypoint runs:

envsubst '$GATEWAY_API_TOKEN' < /etc/nginx/nginx.conf.template > /etc/nginx/nginx.conf && nginx -g 'daemon off;'

Tunables hard-coded in nginx.conf.template (no env vars, edit + redeploy to change):

  • proxy_connect_timeout 60s
  • proxy_send_timeout 300s
  • proxy_read_timeout 600s
  • client_max_body_size 500M
  • client_body_buffer_size 10M (audio route only)
  • worker_connections 1024

How services connect

The AI platform mixes two patterns; this gateway is opt-in aggregation, not platform-wide:

Service Port Routed via gateway? Notes
didi-brain (brain_api) :8090 NO — direct Brain is exposed directly on its own port; not registered in gateway upstreams
didiAI-dashboard :51300 NO — direct Internal monitoring UI, exposed directly
didiAI-web-api :51100 YES (/web/) and direct Both paths work
didiAI-llm-api :14011 YES (/llm/) — required for SSE LLM router; gateway adds streaming-friendly proxy settings
didiAI-embeddings-api :14100 YES (/embeddings/) and direct OpenAI-compatible
didiAI-rerank-api :14200 YES (/rerank/) and direct Cohere/Jina-compatible
didiAI-audio :54300 YES (/audio/) — recommended for large uploads gateway sets a 10 MB body buffer
didiAI-catalog-api :11000 YES (/catalog/) Catalog also listens on 11000 internally — same number as gateway, different network endpoint

So the gateway aggregates the inference/IO services (LLM, embeddings, rerank, audio, web, catalog) under one host:port, while observability/orchestration components (brain, dashboard) stay on their own ports.

This gateway is independent from the backend stack's Kong/Keycloak: backend services on agent-v3 go through the public Kong cluster (10.11.10.175 → DP1/DP2) with JWT from the SSO cluster; this didiAI-gateway lives entirely inside the AI-platform Docker network and uses a static Bearer token. The two systems do not call each other through their gateways — agent-v3 talks to AI-platform services directly by container DNS name on the shared Docker network when co-located, or over the LAN otherwise.

Deployment

cd /home/admin365/didi_mono/ai_platform/modules/gateway/deploy
cp .env.example .env
# edit .env, set GATEWAY_API_TOKEN
./deploy.sh -d                    # docker compose up -d
./deploy.sh --logs                # tail logs
./deploy.sh --down                # stop + remove

Manual equivalent: docker compose up -d from deploy/. Network didi-network must exist beforehand (it is created by another module's compose, typically catalog or dashboard).

Healthcheck: wget --spider http://127.0.0.1:11000/health every 30 s.

Related

  • AI platform overview: /home/admin365/didi_mono/ai_platform/CLAUDE.md (and module-level README.md in each sibling: dashboard/, web/, embeddings/, rerank/, didi_brain/brain_api/).
  • Upstream service modules referenced by this gateway:
    • /home/admin365/didi_mono/ai_platform/modules/web/ (running)
    • /home/admin365/didi_mono/ai_platform/modules/embeddings/ (not running on this host)
    • /home/admin365/didi_mono/ai_platform/modules/rerank/ (not running on this host)
    • llm-inference, audio, catalog modules (catalog is running; llm-inference + audio are not)
  • Distinct from backend Kong — see backend/services/gateway-auth-layer/didiKong/ and project_kong_migration_2026_04.md in the user memory. The backend's Kong cluster is the public edge for agent-v3; didiAI-gateway is internal-only for AI-platform services.