clossers/didi-lot1-ai
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

LOT 1 - Optimizare script build -Instalare mono comanda

Browse source
This commit is contained in:
Dezvoltari Evotech 2026-06-27 06:42:02 -07:00
parent 5380c3fc63
commit 42ff22bf85
127 changed files with 16163 additions and 532 deletions
Download patch file Download diff file Expand all files Collapse all files

15
ai_platform/modules/didi_brain/INDEX.md
View file

@ -2,7 +2,7 @@
Knowledge atom storage + claim verification cache + analysis atom cache pentru platforma DIDI. FastAPI service care vorbeste contractul `web-gathering` 1:1 (drop-in replacement / cache layer pentru `/v1/gather`) si in plus expune doua cache-uri proprii: `brain_verification_cache` (claims, din 2026-04-23) si `brain_analysis_atom` (techniques + ai_tampered + claims, NEW v2 din 2026-05-01).
**Stack**: Python 3.11+, FastAPI, Pydantic v2, asyncpg, BGE-M3 1024-dim embeddings (vLLM), BGE-reranker-v2-m3 (vLLM cross-encoder), Qwen3.5-397B-A17B (llama.cpp via OpenAI-compat router)
**Stack**: Python 3.11+, FastAPI, Pydantic v2, asyncpg, BGE-M3 1024-dim embeddings (vLLM), BGE-reranker-v2-m3 (vLLM cross-encoder), qwen3.5 (vLLM via OpenAI-compat router)
**Storage**: Atomic (Rust knowledge graph) on Postgres 16 + pgvector. Brain owns its own tables prefixed `brain_*` alongside Atomic's tables.
**URL**: `http://10.11.10.12:8090` (production-local, since brain cutover 2026-05-01 — anterior pe `10.11.10.13:8090`)
**Container**: `didibrain-api` (alongside `didibrain-atomic` and `didibrain-postgres`)
@ -163,9 +163,10 @@ UNIQUE: `(content_hash, component, prompt_hash)` — `tier` OUT of unique key (w
## Configuration
- Settings via `shared/config.py` cu `pydantic-settings`
- Env vars: `postgres_dsn`, `atomic_url`, `llm_router_url`, `embedding_url`, `reranker_url`, `verification_cache_ttl_days` (default 30), `verification_cache_max_payload_kb` (default 64)
- Internal Atomic URL: `http://didibrain-atomic:8080` (Docker DNS)
- Upstream LLM/embed/rerank pe VPN 10.11.10.x (10.11.10.17:14011 router, 10.11.10.15:8200 BGE-M3, 10.11.10.15:8100 reranker)
- Env vars: `postgres_dsn`, `atomic_url`, `atomic_token`, `llm_router_url`, `embedding_url`, `reranker_url`, `verification_cache_ttl_days` (default 30), `verification_cache_max_payload_kb` (default 64)
- `ATOMIC_TOKEN`: required for atomic-server API calls, auto-generated/auto-populated by `scripts/02_bootstrap_atomic.py` (written back to `.env`)
- Internal Atomic URL: `http://atomic-server:8080` (Docker DNS)
- Upstream LLM/embed/rerank reached over Docker DNS on `didi-network` (`didiAI-llm-api` → 10.11.10.17:14011 router, `didiAI-embeddings-api` → 10.11.10.15:14100 BGE-M3, `didiAI-rerank-api` → 10.11.10.15:14200 reranker)
- **No auth in v1** (binds to internal Docker network)
---
@ -173,10 +174,10 @@ UNIQUE: `(content_hash, component, prompt_hash)` — `tier` OUT of unique key (w
## Deployment
- Docker compose la `infra/docker-compose.yml`
- 3 containere: `didibrain-api` (FastAPI), `didibrain-atomic` (Rust KG), `didibrain-postgres` (PG 16 + pgvector)
- 4 containere: `didibrain-api` (FastAPI), `didibrain-atomic` (Rust KG), `didibrain-postgres` (PG 16 + pgvector), `didibrain-scheduler` (feeder + auditor + watcher)
- Schema bootstrap on startup (idempotent — `_SCHEMA_SQL` ruleaza la fiecare connect, all DDL is `IF NOT EXISTS` + ALTER guards)
- TTL cleanup: cron job in `scripts/` sterge expired silver/bronze atoms
- Resource footprint: ~200 MB RAM total cross 3 containere; brain-api idle ~6% CPU, spike ~20% during `/v1/gather` cu NLI
- Resource footprint: ~200 MB RAM total cross 4 containere; brain-api idle ~6% CPU, spike ~20% during `/v1/gather` cu NLI
**Rebuild**:
```bash
@ -194,7 +195,7 @@ docker compose -f infra/docker-compose.yml --env-file .env up -d --build
- `scripts/04_seed_taxonomy.py` — seed canonical tag taxonomy (79 tags, 7 namespaces)
- `scripts/05_import_wikipedia_seed.py` — seed-list Wikipedia import
- `scripts/06_validate_queries.py` — smoke test doc-level retrieval
- `scripts/07_run_extraction.py` — claim extraction batch (Qwen 397B)
- `scripts/07_run_extraction.py` — claim extraction batch (qwen3.5)
- `scripts/08_validate_claims.py` — smoke test claim-level retrieval
- `scripts/09_brain_api_demo.py` — brain_api contract test (all endpoints)
- `scripts/10_run_lint.py` — Lint pass runner (contradiction detection)