Livrare LOT 1 - Didi
This commit is contained in:
commit
5380c3fc63
990 changed files with 133308 additions and 0 deletions
472
ai_platform/DEPLOYMENT.md
Normal file
472
ai_platform/DEPLOYMENT.md
Normal file
|
|
@ -0,0 +1,472 @@
|
|||
# Deployment Guide
|
||||
|
||||
This guide covers bootstrapping the **web + dashboard** stack on a fresh CPU-only host. All GPU-dependent services (`llm-inference`, `embeddings`, `rerank`, `audio`, `video-analysis`) remain on a separate GPU machine and are called over HTTP.
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────── CPU-only host (this machine) ──────────────────────────────┐
|
||||
│ │
|
||||
│ ┌────────────────────┐ ┌───────────────────────────────────────────────────┐ │
|
||||
│ │ Backend clients │────▶ │ Web API (:51100) │ │
|
||||
│ │ (your app / UI) │ │ ├─ free tier → SearXNG │ │
|
||||
│ └────────────────────┘ │ └─ premium tier → SerpAPI/Tavily/Brave/LinkUp │ │
|
||||
│ │ + OpenRouter for LLM calls │ │
|
||||
│ └──────────────┬────────────────────────────────────┘ │
|
||||
│ │ fires events │
|
||||
│ ▼ │
|
||||
│ ┌─────────────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Dashboard (:51300) ◀── PostgreSQL (:15432 internal) │ │
|
||||
│ │ · overview / providers / history │ │
|
||||
│ │ · /config (runtime toggles) │ │
|
||||
│ │ · /archive (promote gathers for future claims-api) │ │
|
||||
│ │ · /cost (spend + projections) │ │
|
||||
│ └─────────────────────────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ ┌───────────────────┐ │
|
||||
│ │ SearXNG │ (existing — free tier source of truth) │
|
||||
│ │ :55100/8080 │ │
|
||||
│ └───────────────────┘ │
|
||||
│ │
|
||||
└──────────┬────────────────────────────────────────────────────────────────────────────┘
|
||||
│ HTTP calls (over VPN / LAN)
|
||||
▼
|
||||
┌─────────────────────────── GPU host (separate machine) ──────────────────────────────┐
|
||||
│ │
|
||||
│ llm-inference (:14011) ◀── Web API calls this for context + evidence LLM │
|
||||
│ └─ vLLM Qwen3.5-35B (:14001) │
|
||||
│ └─ llama.cpp servers (optional, on separate GPU nodes) │
|
||||
│ │
|
||||
│ embeddings (:14100) ◀── reserved for future semantic search │
|
||||
│ rerank (:14200) ◀── reserved │
|
||||
│ audio (:54300) ◀── whisper transcription │
|
||||
│ video-api (:54600) ◀── deepfake detection │
|
||||
│ │
|
||||
└──────────────────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### What's deployed where
|
||||
|
||||
| Service | This host (CPU) | GPU host |
|
||||
|-----------------|:---------------:|:--------------:|
|
||||
| SearXNG | ✓ | |
|
||||
| web-api | ✓ | |
|
||||
| dashboard | ✓ | |
|
||||
| dashboard-db | ✓ | |
|
||||
| llm-inference | | ✓ |
|
||||
| embeddings | | ✓ |
|
||||
| rerank | | ✓ |
|
||||
| audio | | ✓ |
|
||||
| video-analysis | | ✓ |
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- **OS:** Linux (Ubuntu 22.04+ or similar)
|
||||
- **Docker:** 24.0+ with Docker Compose V2 (`docker compose` subcommand)
|
||||
- **Disk:** ~2 GB free for images + DB volume
|
||||
- **RAM:** 2 GB+ available for containers
|
||||
- **Network:**
|
||||
- Port 51100 (web-api) + 51300 (dashboard) free on the host
|
||||
- 15432 free if you want the dashboard DB exposed for debugging
|
||||
- Outbound HTTPS to SerpAPI, Tavily, Brave, LinkUp, OpenRouter (if using premium)
|
||||
- LAN/VPN reachability to your GPU host for `WEB_LLM_BASE_URL`
|
||||
|
||||
Not required on this host: NVIDIA driver, CUDA, nvidia-container-toolkit.
|
||||
|
||||
---
|
||||
|
||||
## Quickstart (one command)
|
||||
|
||||
```bash
|
||||
git clone https://git.finesynergy.eu/ml/ml-projects.git
|
||||
cd ml-projects
|
||||
./bootstrap.sh
|
||||
```
|
||||
|
||||
The script will:
|
||||
|
||||
1. Verify Docker + Compose.
|
||||
2. Create the `deploy_default` network.
|
||||
3. Prompt for:
|
||||
- GPU host address (for LLM URL)
|
||||
- SearXNG container name / URL
|
||||
- Paid provider keys (SerpAPI, Tavily, Brave, LinkUp, OpenRouter — leave blank to skip)
|
||||
- Generate a random Postgres password
|
||||
4. Render `modules/web/deploy/.env` and `modules/dashboard/deploy/.env`.
|
||||
5. Deploy dashboard + web-api.
|
||||
6. Create an admin user and print the Bearer token.
|
||||
7. Run smoke tests.
|
||||
|
||||
### Script flags
|
||||
|
||||
| Flag | Purpose |
|
||||
|------|---------|
|
||||
| _(none)_ | Interactive — prompts for everything |
|
||||
| `--non-interactive` | Use existing `.env` files without prompting |
|
||||
| `--deploy-searxng` | Also deploy the bundled SearXNG stack from `modules/web/deploy/metasearch` |
|
||||
| `--skip-smoke` | Skip post-deploy smoke tests |
|
||||
| `--help` | Show usage |
|
||||
|
||||
---
|
||||
|
||||
## Manual deployment (step by step)
|
||||
|
||||
If the bootstrap script fails or you prefer manual control:
|
||||
|
||||
### 1. Create the network
|
||||
|
||||
```bash
|
||||
docker network inspect deploy_default >/dev/null 2>&1 \
|
||||
|| docker network create deploy_default
|
||||
```
|
||||
|
||||
### 2. Ensure SearXNG is reachable
|
||||
|
||||
If SearXNG is already running on this host, connect it to the network:
|
||||
|
||||
```bash
|
||||
docker network connect deploy_default <searxng-container-name>
|
||||
```
|
||||
|
||||
Otherwise, deploy the bundled SearXNG:
|
||||
|
||||
```bash
|
||||
cd modules/web/deploy/metasearch
|
||||
docker compose up -d
|
||||
cd -
|
||||
```
|
||||
|
||||
### 3. Configure and deploy the dashboard
|
||||
|
||||
```bash
|
||||
cp modules/dashboard/.env.example modules/dashboard/deploy/.env
|
||||
# Edit modules/dashboard/deploy/.env — set:
|
||||
# DASHBOARD_DB_PASSWORD=<strong-random-password>
|
||||
# DASHBOARD_LLM_API_URL=http://<your-gpu-host>:14011
|
||||
# DASHBOARD_VLLM_QWEN_URL=http://<your-gpu-host>:14001
|
||||
# (paid provider keys if you want live quota readouts)
|
||||
|
||||
cd modules/dashboard/deploy
|
||||
./deploy.sh up
|
||||
|
||||
# Wait for health
|
||||
curl http://localhost:51300/health
|
||||
```
|
||||
|
||||
### 4. Configure and deploy web-api
|
||||
|
||||
```bash
|
||||
cp modules/web/.env.example modules/web/deploy/.env
|
||||
# Edit modules/web/deploy/.env — set:
|
||||
# WEB_SEARXNG_BASE_URL=http://<searxng-container>:8080
|
||||
# WEB_LLM_BASE_URL=http://<your-gpu-host>:14011
|
||||
# WEB_VISION_BASE_URL=http://<your-gpu-host>:14011
|
||||
# WEB_DASHBOARD_URL=http://didiAI-dashboard:51300
|
||||
# (paid provider keys for premium tier)
|
||||
# (WEB_OPENROUTER_API_KEY for premium LLM)
|
||||
|
||||
cd modules/web/deploy
|
||||
docker compose --profile api up -d --build
|
||||
|
||||
# Wait for health
|
||||
curl http://localhost:51100/health
|
||||
```
|
||||
|
||||
### 5. Create an admin user on the dashboard
|
||||
|
||||
```bash
|
||||
docker exec -it didiAI-dashboard python -m dashboard.cli \
|
||||
create-user <your-username> \
|
||||
--email <your-email> \
|
||||
--role admin
|
||||
```
|
||||
|
||||
Save the Bearer token it prints — it's not stored in plain text and can't be recovered.
|
||||
|
||||
### 6. Verify everything works
|
||||
|
||||
```bash
|
||||
# Dashboard
|
||||
curl http://localhost:51300/health
|
||||
curl http://localhost:51300/api/stats/providers | jq '.providers | length'
|
||||
|
||||
# Free tier search (uses SearXNG)
|
||||
curl -X POST http://localhost:51100/v1/search \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"queries":["hello world"],"max_results":3}'
|
||||
|
||||
# Premium tier search (uses paid rotation)
|
||||
curl -X POST http://localhost:51100/v1/search \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "X-Search-Tier: premium" \
|
||||
-d '{"queries":["hello world"],"max_results":3}'
|
||||
```
|
||||
|
||||
Then browse to <http://localhost:51300> for the UI.
|
||||
|
||||
---
|
||||
|
||||
## Configuration reference
|
||||
|
||||
### Required values for this host
|
||||
|
||||
| File | Variable | Purpose |
|
||||
|------|----------|---------|
|
||||
| `modules/web/deploy/.env` | `WEB_SEARXNG_BASE_URL` | SearXNG internal URL |
|
||||
| `modules/web/deploy/.env` | `WEB_LLM_BASE_URL` | Remote LLM inference endpoint |
|
||||
| `modules/web/deploy/.env` | `WEB_VISION_BASE_URL` | Usually same as LLM URL |
|
||||
| `modules/web/deploy/.env` | `WEB_EXTERNAL_URL` | Public-facing web-api URL |
|
||||
| `modules/dashboard/deploy/.env` | `DASHBOARD_DB_PASSWORD` | Random strong password |
|
||||
| `modules/dashboard/deploy/.env` | `DASHBOARD_EXTERNAL_URL` | Public-facing dashboard URL |
|
||||
| `modules/dashboard/deploy/.env` | `DASHBOARD_LLM_API_URL` | Remote LLM (for health check card) |
|
||||
|
||||
### Optional — enable premium tier
|
||||
|
||||
Set **at least one** of these in `modules/web/deploy/.env`. If all are blank, premium tier returns empty results.
|
||||
|
||||
```
|
||||
WEB_SERPAPI_API_KEY=...
|
||||
WEB_TAVILY_API_KEY=...
|
||||
WEB_BRAVE_API_KEY=...
|
||||
WEB_LINKUP_API_KEY=...
|
||||
WEB_OPENROUTER_API_KEY=...
|
||||
WEB_OPENROUTER_MODEL=google/gemini-3.1-flash-lite-preview
|
||||
```
|
||||
|
||||
Also mirror these into `modules/dashboard/deploy/.env` (`DASHBOARD_*` prefix) so the dashboard can show live quota.
|
||||
|
||||
### Dashboard ↔ Web wiring
|
||||
|
||||
The web-api polls the dashboard for runtime config every 30s and fires events back. This requires:
|
||||
|
||||
```
|
||||
# modules/web/deploy/.env
|
||||
WEB_DASHBOARD_URL=http://didiAI-dashboard:51300
|
||||
```
|
||||
|
||||
Both containers must be on the `deploy_default` network for internal hostname resolution.
|
||||
|
||||
---
|
||||
|
||||
## Sending traffic
|
||||
|
||||
Send `X-Search-Tier` to select the pipeline:
|
||||
|
||||
```bash
|
||||
# Free — SearXNG only + remote Qwen LLM (cheap / slow / open)
|
||||
curl -X POST http://localhost:51100/v1/gather \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"claim":"...","max_search_results":10}'
|
||||
|
||||
# Premium — paid rotation + OpenRouter (fast / higher quality)
|
||||
curl -X POST http://localhost:51100/v1/gather \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "X-Search-Tier: premium" \
|
||||
-d '{"claim":"...","max_search_results":10}'
|
||||
```
|
||||
|
||||
Response schema is identical between tiers. Only the upstream sources differ.
|
||||
|
||||
---
|
||||
|
||||
## Operations
|
||||
|
||||
### Tailing logs
|
||||
|
||||
```bash
|
||||
docker logs -f didiAI-web-api
|
||||
docker logs -f didiAI-dashboard
|
||||
docker logs -f didiAI-dashboard-db
|
||||
```
|
||||
|
||||
### Restarting a single container
|
||||
|
||||
```bash
|
||||
docker restart didiAI-web-api
|
||||
# or for a full rebuild:
|
||||
cd modules/web/deploy && docker compose --profile api up -d --build
|
||||
```
|
||||
|
||||
### Managing users
|
||||
|
||||
```bash
|
||||
# Create
|
||||
docker exec didiAI-dashboard python -m dashboard.cli create-user alice \
|
||||
--email alice@example.com --role admin
|
||||
|
||||
# List
|
||||
docker exec didiAI-dashboard python -m dashboard.cli list-users
|
||||
|
||||
# Delete
|
||||
docker exec didiAI-dashboard python -m dashboard.cli delete-user alice
|
||||
```
|
||||
|
||||
### Runtime configuration (no restart needed)
|
||||
|
||||
Open `http://localhost:51300/config` for the UI, or use the API:
|
||||
|
||||
```bash
|
||||
TOKEN="<your admin token>"
|
||||
|
||||
# Disable a provider
|
||||
curl -X PUT http://localhost:51300/api/config/web.providers.serpapi.enabled \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"value": false}'
|
||||
|
||||
# Switch premium rotation strategy
|
||||
curl -X PUT http://localhost:51300/api/config/web.premium.strategy \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"value": "parallel"}'
|
||||
|
||||
# Swap OpenRouter model
|
||||
curl -X PUT http://localhost:51300/api/config/web.openrouter.model \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"value": "anthropic/claude-haiku-4.5"}'
|
||||
```
|
||||
|
||||
Changes propagate within 30 seconds (next poll cycle) — or faster with `docker restart didiAI-web-api`.
|
||||
|
||||
Audit trail at `http://localhost:51300/audit`.
|
||||
|
||||
---
|
||||
|
||||
## Backup & restore
|
||||
|
||||
### Dashboard database
|
||||
|
||||
The dashboard stores request history, runtime config, audit log, users, and archived claims in PostgreSQL volume `deploy_dashboard_db_data`.
|
||||
|
||||
Nightly backup:
|
||||
|
||||
```bash
|
||||
docker exec didiAI-dashboard-db pg_dump -U dashboard -d dashboard \
|
||||
| gzip > "backup-$(date +%Y%m%d).sql.gz"
|
||||
```
|
||||
|
||||
Restore on a fresh install:
|
||||
|
||||
```bash
|
||||
# Make sure the dashboard DB container is running but the app is down
|
||||
docker stop didiAI-dashboard
|
||||
gunzip < backup-YYYYMMDD.sql.gz | \
|
||||
docker exec -i didiAI-dashboard-db psql -U dashboard -d dashboard
|
||||
docker start didiAI-dashboard
|
||||
```
|
||||
|
||||
### Config files
|
||||
|
||||
`.env` files under `modules/*/deploy/.env` are gitignored — they are the only stateful config on the host outside PostgreSQL. Back them up securely (they contain provider API keys and the DB password).
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### `docker network deploy_default not found`
|
||||
|
||||
```bash
|
||||
docker network create deploy_default
|
||||
```
|
||||
|
||||
### SearXNG unreachable from web-api
|
||||
|
||||
Check that SearXNG is on the shared network:
|
||||
|
||||
```bash
|
||||
docker network inspect deploy_default | grep -A2 searxng
|
||||
# If missing:
|
||||
docker network connect deploy_default <searxng-container>
|
||||
```
|
||||
|
||||
Then verify the hostname/port in `WEB_SEARXNG_BASE_URL`.
|
||||
|
||||
### Dashboard shows "LLM unreachable"
|
||||
|
||||
This host has no GPU. `WEB_LLM_BASE_URL` + `DASHBOARD_LLM_API_URL` must point at your GPU machine:
|
||||
|
||||
```
|
||||
WEB_LLM_BASE_URL=http://10.x.x.x:14011
|
||||
```
|
||||
|
||||
Verify from inside the web-api container:
|
||||
|
||||
```bash
|
||||
docker exec didiAI-web-api curl -sf http://10.x.x.x:14011/v1/models
|
||||
```
|
||||
|
||||
### Premium tier returns empty results
|
||||
|
||||
Open `http://localhost:51300/providers` and check each paid provider's status:
|
||||
|
||||
- **Down / 401** → invalid API key in `.env`. Fix and redeploy web-api.
|
||||
- **429 Too Many Requests** → monthly quota exhausted. Either disable the provider runtime on `/config` or top up the plan.
|
||||
|
||||
### `pg_isready` failing on dashboard-db
|
||||
|
||||
First start takes ~15 s to initialize the database cluster. If it's still failing after 60 s:
|
||||
|
||||
```bash
|
||||
docker logs didiAI-dashboard-db --tail 50
|
||||
```
|
||||
|
||||
Common cause: stale volume with different credentials. To wipe and restart:
|
||||
|
||||
```bash
|
||||
cd modules/dashboard/deploy
|
||||
docker compose --profile dashboard down
|
||||
docker volume rm deploy_dashboard_db_data
|
||||
docker compose --profile dashboard up -d
|
||||
```
|
||||
|
||||
**Warning:** this deletes all request history, config overrides, archive, and users. Back up first if needed.
|
||||
|
||||
### Port already in use
|
||||
|
||||
Edit the exposed port mapping in `modules/<module>/deploy/docker-compose.yml` and update `*_EXTERNAL_URL` in the matching `.env`.
|
||||
|
||||
### "Invalid token" on `/api/config` PUT
|
||||
|
||||
Use a user token created via `docker exec ... create-user`, not an env-based token. Legacy static tokens in `DASHBOARD_API_TOKENS` still work but only if no users exist in the DB.
|
||||
|
||||
---
|
||||
|
||||
## Upgrade
|
||||
|
||||
Pull latest code and redeploy affected modules:
|
||||
|
||||
```bash
|
||||
git pull
|
||||
# Dashboard changes:
|
||||
cd modules/dashboard/deploy && docker compose --profile dashboard up -d --build
|
||||
# Web changes:
|
||||
cd modules/web/deploy && docker compose --profile api up -d --build
|
||||
```
|
||||
|
||||
Schema migrations happen automatically on startup (the dashboard creates missing tables idempotently). For destructive schema changes, see release notes.
|
||||
|
||||
---
|
||||
|
||||
## Security notes
|
||||
|
||||
- All services assume they run on a trusted internal network (VPN / LAN). The dashboard's read endpoints are unauthenticated by design.
|
||||
- Only mutations (`PUT /api/config/*`, `POST /api/archive/promote/*`) require a Bearer token.
|
||||
- If you need to expose any of these on the public internet, put them behind an authenticated reverse proxy and set `WEB_API_TOKENS` / `DASHBOARD_API_TOKENS` to enforce auth on read endpoints too.
|
||||
- Provider API keys live in `.env` files (mode 0644 on disk). Rotate them periodically via each provider's dashboard.
|
||||
- The admin Bearer token is printed once. Store it in a password manager; rotate with `delete-user` + `create-user`.
|
||||
|
||||
---
|
||||
|
||||
## Related docs
|
||||
|
||||
- `CLAUDE.md` — stable coding conventions
|
||||
- `ENDPOINTS.md` — endpoint catalog across modules
|
||||
- `INDEX.md` — module directory
|
||||
- `STATUS.md` — living notes on the active work stream
|
||||
- `modules/web/README.md` — web module details
|
||||
- `modules/dashboard/README.md` — dashboard module details
|
||||
Loading…
Add table
Add a link
Reference in a new issue