# Deployment Guide > **Note — delivered deployment:** the delivered stack runs **single-host on the GPU machine** via `ai_platform/local_gpu_stack/deploy.sh` (see `local_gpu_stack/README.md`). The guide below describes the **reference multi-host variant** (CPU host + separate GPU host), kept intentionally for portability. This guide covers bootstrapping the **web + dashboard** stack on a fresh CPU-only host. All GPU-dependent services (`llm-inference`, `embeddings`, `rerank`, `audio`, `video-analysis`) remain on a separate GPU machine and are called over HTTP. --- ## Architecture ``` ┌─────────────────────────── CPU-only host (this machine) ──────────────────────────────┐ │ │ │ ┌────────────────────┐ ┌───────────────────────────────────────────────────┐ │ │ │ Backend clients │────▶ │ Web API (:51100) │ │ │ │ (your app / UI) │ │ ├─ free tier → SearXNG │ │ │ └────────────────────┘ │ └─ premium tier → SerpAPI/Tavily/Brave/LinkUp │ │ │ │ + OpenRouter for LLM calls │ │ │ └──────────────┬────────────────────────────────────┘ │ │ │ fires events │ │ ▼ │ │ ┌─────────────────────────────────────────────────────────────────────────────┐ │ │ │ Dashboard (:51300) ◀── PostgreSQL (:15432 internal) │ │ │ │ · overview / providers / history │ │ │ │ · /config (runtime toggles) │ │ │ │ · /archive (promote gathers for future claims-api) │ │ │ │ · /cost (spend + projections) │ │ │ └─────────────────────────────────────────────────────────────────────────────┘ │ │ │ │ ┌───────────────────┐ │ │ │ SearXNG │ (existing — free tier source of truth) │ │ │ :8080 (intern) │ │ │ └───────────────────┘ │ │ │ └──────────┬────────────────────────────────────────────────────────────────────────────┘ │ HTTP calls (over VPN / LAN) ▼ ┌─────────────────────────── GPU host (separate machine) ──────────────────────────────┐ │ │ │ llm-inference (:14011) ◀── Web API calls this for context + evidence LLM │ │ └─ vLLM Qwen3.5-35B (:14001) │ │ └─ llama.cpp servers (optional, on separate GPU nodes) │ │ │ │ embeddings (:14100) ◀── reserved for future semantic search │ │ rerank (:14200) ◀── reserved │ │ audio (:54300) ◀── whisper transcription │ │ video-api (:54600) ◀── deepfake detection │ │ │ └──────────────────────────────────────────────────────────────────────────────────────┘ ``` ### What's deployed where | Service | This host (CPU) | GPU host | |-----------------|:---------------:|:--------------:| | SearXNG | ✓ | | | web-api | ✓ | | | dashboard | ✓ | | | dashboard-db | ✓ | | | llm-inference | | ✓ | | embeddings | | ✓ | | rerank | | ✓ | | audio | | ✓ | | video-analysis | | ✓ | --- ## Prerequisites - **OS:** Linux (Ubuntu 22.04+ or similar) - **Docker:** 24.0+ with Docker Compose V2 (`docker compose` subcommand) - **Disk:** ~2 GB free for images + DB volume - **RAM:** 2 GB+ available for containers - **Network:** - Port 51100 (web-api) + 51300 (dashboard) free on the host - 15432 free if you want the dashboard DB exposed for debugging - Outbound HTTPS to SerpAPI, Tavily, Brave, LinkUp, OpenRouter (if using premium) - LAN/VPN reachability to your GPU host for `WEB_LLM_BASE_URL` Not required on this host: NVIDIA driver, CUDA, nvidia-container-toolkit. --- ## Quickstart (one command) ```bash git clone https://git.finesynergy.eu/ml/ml-projects.git cd ml-projects ./bootstrap.sh ``` The script will: 1. Verify Docker + Compose. 2. Create the `didi-network` network. 3. Prompt for: - GPU host address (for LLM URL) - SearXNG container name / URL - Paid provider keys (SerpAPI, Tavily, Brave, LinkUp, OpenRouter — leave blank to skip) - Generate a random Postgres password 4. Render `modules/web/deploy/.env` and `modules/dashboard/deploy/.env`. 5. Deploy dashboard + web-api. 6. Create an admin user and print the Bearer token. 7. Run smoke tests. ### Script flags | Flag | Purpose | |------|---------| | _(none)_ | Interactive — prompts for everything | | `--non-interactive` | Use existing `.env` files without prompting | | `--deploy-searxng` | Also deploy the bundled SearXNG stack from `modules/web/deploy/metasearch` | | `--skip-smoke` | Skip post-deploy smoke tests | | `--help` | Show usage | --- ## Manual deployment (step by step) If the bootstrap script fails or you prefer manual control: ### 1. Create the network ```bash docker network inspect didi-network >/dev/null 2>&1 \ || docker network create didi-network ``` ### 2. Ensure SearXNG is reachable If SearXNG is already running on this host, connect it to the network: ```bash docker network connect didi-network ``` Otherwise, deploy the bundled SearXNG: ```bash cd modules/web/deploy/metasearch docker compose up -d cd - ``` ### 3. Configure and deploy the dashboard ```bash cp modules/dashboard/.env.example modules/dashboard/deploy/.env # Edit modules/dashboard/deploy/.env — set: # DASHBOARD_DB_PASSWORD= # DASHBOARD_LLM_API_URL=http://:14011 # DASHBOARD_VLLM_QWEN_URL=http://:14001 # (paid provider keys if you want live quota readouts) cd modules/dashboard/deploy ./deploy.sh up # Wait for health curl http://localhost:51300/health ``` ### 4. Configure and deploy web-api ```bash cp modules/web/.env.example modules/web/deploy/.env # Edit modules/web/deploy/.env — set: # WEB_SEARXNG_BASE_URL=http://:8080 # WEB_LLM_BASE_URL=http://:14011 # WEB_VISION_BASE_URL=http://:14011 # WEB_DASHBOARD_URL=http://didiAI-dashboard:51300 # (paid provider keys for premium tier) # (WEB_OPENROUTER_API_KEY for premium LLM) cd modules/web/deploy docker compose --profile api up -d --build # Wait for health curl http://localhost:51100/health ``` ### 5. Create an admin user on the dashboard ```bash docker exec -it didiAI-dashboard python -m dashboard.cli \ create-user \ --email \ --role admin ``` Save the Bearer token it prints — it's not stored in plain text and can't be recovered. ### 6. Verify everything works ```bash # Dashboard curl http://localhost:51300/health curl http://localhost:51300/api/stats/providers | jq '.providers | length' # Free tier search (uses SearXNG) curl -X POST http://localhost:51100/v1/search \ -H "Content-Type: application/json" \ -d '{"queries":["hello world"],"max_results":3}' # Premium tier search (uses paid rotation) curl -X POST http://localhost:51100/v1/search \ -H "Content-Type: application/json" \ -H "X-Search-Tier: premium" \ -d '{"queries":["hello world"],"max_results":3}' ``` Then browse to for the UI. --- ## Configuration reference ### Required values for this host | File | Variable | Purpose | |------|----------|---------| | `modules/web/deploy/.env` | `WEB_SEARXNG_BASE_URL` | SearXNG internal URL | | `modules/web/deploy/.env` | `WEB_LLM_BASE_URL` | Remote LLM inference endpoint | | `modules/web/deploy/.env` | `WEB_VISION_BASE_URL` | Usually same as LLM URL | | `modules/web/deploy/.env` | `WEB_EXTERNAL_URL` | Public-facing web-api URL | | `modules/dashboard/deploy/.env` | `DASHBOARD_DB_PASSWORD` | Random strong password | | `modules/dashboard/deploy/.env` | `DASHBOARD_EXTERNAL_URL` | Public-facing dashboard URL | | `modules/dashboard/deploy/.env` | `DASHBOARD_LLM_API_URL` | Remote LLM (for health check card) | ### Optional — enable premium tier Set **at least one** of these in `modules/web/deploy/.env`. If all are blank, premium tier returns empty results. ``` WEB_SERPAPI_API_KEY=... WEB_TAVILY_API_KEY=... WEB_BRAVE_API_KEY=... WEB_LINKUP_API_KEY=... WEB_OPENROUTER_API_KEY=... WEB_OPENROUTER_MODEL=google/gemini-3.1-flash-lite-preview ``` Also mirror these into `modules/dashboard/deploy/.env` (`DASHBOARD_*` prefix) so the dashboard can show live quota. ### Dashboard ↔ Web wiring The web-api polls the dashboard for runtime config every 30s and fires events back. This requires: ``` # modules/web/deploy/.env WEB_DASHBOARD_URL=http://didiAI-dashboard:51300 ``` Both containers must be on the `didi-network` network for internal hostname resolution. --- ## Sending traffic Send `X-Search-Tier` to select the pipeline: ```bash # Free — SearXNG only + remote Qwen LLM (cheap / slow / open) curl -X POST http://localhost:51100/v1/gather \ -H "Content-Type: application/json" \ -d '{"claim":"...","max_search_results":10}' # Premium — paid rotation + OpenRouter (fast / higher quality) curl -X POST http://localhost:51100/v1/gather \ -H "Content-Type: application/json" \ -H "X-Search-Tier: premium" \ -d '{"claim":"...","max_search_results":10}' ``` Response schema is identical between tiers. Only the upstream sources differ. --- ## Operations ### Tailing logs ```bash docker logs -f didiAI-web-api docker logs -f didiAI-dashboard docker logs -f didiAI-dashboard-db ``` ### Restarting a single container ```bash docker restart didiAI-web-api # or for a full rebuild: cd modules/web/deploy && docker compose --profile api up -d --build ``` ### Managing users ```bash # Create docker exec didiAI-dashboard python -m dashboard.cli create-user alice \ --email alice@example.com --role admin # List docker exec didiAI-dashboard python -m dashboard.cli list-users # Delete docker exec didiAI-dashboard python -m dashboard.cli delete-user alice ``` ### Runtime configuration (no restart needed) Open `http://localhost:51300/config` for the UI, or use the API: ```bash TOKEN="" # Disable a provider curl -X PUT http://localhost:51300/api/config/web.providers.serpapi.enabled \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ -d '{"value": false}' # Switch premium rotation strategy curl -X PUT http://localhost:51300/api/config/web.premium.strategy \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ -d '{"value": "parallel"}' # Swap OpenRouter model curl -X PUT http://localhost:51300/api/config/web.openrouter.model \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ -d '{"value": "anthropic/claude-haiku-4.5"}' ``` Changes propagate within 30 seconds (next poll cycle) — or faster with `docker restart didiAI-web-api`. Audit trail at `http://localhost:51300/audit`. --- ## Backup & restore ### Dashboard database The dashboard stores request history, runtime config, audit log, users, and archived claims in PostgreSQL volume `deploy_dashboard_db_data`. Nightly backup: ```bash docker exec didiAI-dashboard-db pg_dump -U dashboard -d dashboard \ | gzip > "backup-$(date +%Y%m%d).sql.gz" ``` Restore on a fresh install: ```bash # Make sure the dashboard DB container is running but the app is down docker stop didiAI-dashboard gunzip < backup-YYYYMMDD.sql.gz | \ docker exec -i didiAI-dashboard-db psql -U dashboard -d dashboard docker start didiAI-dashboard ``` ### Config files `.env` files under `modules/*/deploy/.env` are gitignored — they are the only stateful config on the host outside PostgreSQL. Back them up securely (they contain provider API keys and the DB password). --- ## Troubleshooting ### `docker network didi-network not found` ```bash docker network create didi-network ``` ### SearXNG unreachable from web-api Check that SearXNG is on the shared network: ```bash docker network inspect didi-network | grep -A2 searxng # If missing: docker network connect didi-network ``` Then verify the hostname/port in `WEB_SEARXNG_BASE_URL`. ### Dashboard shows "LLM unreachable" This host has no GPU. `WEB_LLM_BASE_URL` + `DASHBOARD_LLM_API_URL` must point at your GPU machine: ``` WEB_LLM_BASE_URL=http://10.x.x.x:14011 ``` Verify from inside the web-api container: ```bash docker exec didiAI-web-api curl -sf http://10.x.x.x:14011/v1/models ``` ### Premium tier returns empty results Open `http://localhost:51300/providers` and check each paid provider's status: - **Down / 401** → invalid API key in `.env`. Fix and redeploy web-api. - **429 Too Many Requests** → monthly quota exhausted. Either disable the provider runtime on `/config` or top up the plan. ### `pg_isready` failing on dashboard-db First start takes ~15 s to initialize the database cluster. If it's still failing after 60 s: ```bash docker logs didiAI-dashboard-db --tail 50 ``` Common cause: stale volume with different credentials. To wipe and restart: ```bash cd modules/dashboard/deploy docker compose --profile dashboard down docker volume rm deploy_dashboard_db_data docker compose --profile dashboard up -d ``` **Warning:** this deletes all request history, config overrides, archive, and users. Back up first if needed. ### Port already in use Edit the exposed port mapping in `modules//deploy/docker-compose.yml` and update `*_EXTERNAL_URL` in the matching `.env`. ### "Invalid token" on `/api/config` PUT Use a user token created via `docker exec ... create-user`, not an env-based token. Legacy static tokens in `DASHBOARD_API_TOKENS` still work but only if no users exist in the DB. --- ## Upgrade Pull latest code and redeploy affected modules: ```bash git pull # Dashboard changes: cd modules/dashboard/deploy && docker compose --profile dashboard up -d --build # Web changes: cd modules/web/deploy && docker compose --profile api up -d --build ``` Schema migrations happen automatically on startup (the dashboard creates missing tables idempotently). For destructive schema changes, see release notes. --- ## Security notes - All services assume they run on a trusted internal network (VPN / LAN). The dashboard's read endpoints are unauthenticated by design. - Only mutations (`PUT /api/config/*`, `POST /api/archive/promote/*`) require a Bearer token. - If you need to expose any of these on the public internet, put them behind an authenticated reverse proxy and set `WEB_API_TOKENS` / `DASHBOARD_API_TOKENS` to enforce auth on read endpoints too. - Provider API keys live in `.env` files (mode 0644 on disk). Rotate them periodically via each provider's dashboard. - The admin Bearer token is printed once. Store it in a password manager; rotate with `delete-user` + `create-user`. --- ## Related docs - `CLAUDE.md` — stable coding conventions - `ENDPOINTS.md` — endpoint catalog across modules - `INDEX.md` — module directory - `STATUS.md` — living notes on the active work stream - `modules/web/README.md` — web module details - `modules/dashboard/README.md` — dashboard module details