clossers/didi-lot1-ai
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

LOT 1 - Optimizare script build -Instalare mono comanda

Browse source
This commit is contained in:
Dezvoltari Evotech 2026-06-27 06:42:02 -07:00
parent 5380c3fc63
commit 42ff22bf85
127 changed files with 16163 additions and 532 deletions
Download patch file Download diff file Expand all files Collapse all files

46
ai_platform/modules/rerank/INDEX.md
View file

@ -132,7 +132,6 @@ rerank/
├── API.md # Full HTTP API reference + curl/Python examples
├── INDEX.md # This file
├── pyproject.toml # Package metadata, deps (fastapi, httpx, openai SDK)
├── uv.lock # uv lockfile
├── .env.example # All RERANK_* env vars documented
├── deploy/
│ ├── Dockerfile # API wrapper image
@ -196,30 +195,35 @@ All env vars use the `RERANK_` prefix and are read via Pydantic Settings (`confi
| `RERANK_LOG_LEVEL` | `INFO` | |
| `RERANK_LOG_JSON` | `false` | |
**vLLM backend:**
**Backend base URLs (app settings — fields on `RerankSettings`):**
| Var | Default |
|-----|---------|
| `RERANK_VLLM_BASE_URL` | `http://localhost:54201` |
| `RERANK_VLLM_MODEL` | `BAAI/bge-reranker-v2-m3` |
| `RERANK_VLLM_PORT` | `14201` |
| `RERANK_VLLM_GPU` | `0` |
| `RERANK_VLLM_GPU_UTIL` | `0.50` |
| `RERANK_VLLM_MAX_LEN` | `8192` |
| Var | Default | Notes |
|-----|---------|-------|
| `RERANK_VLLM_BASE_URL` | `http://localhost:54201` | Where the API wrapper reaches vLLM |
| `RERANK_VLLM_API_KEY` | (empty) | Optional API key for the vLLM server |
| `RERANK_LLAMACPP_BASE_URL` | `http://localhost:54210` | Where the API wrapper reaches llama.cpp |
**llama.cpp backend:**
> Only `*_BASE_URL` / `*_API_KEY` are read by the API app (`config.py`). The model id, port, GPU and context knobs below are **not** `RerankSettings` fields.
| Var | Default |
|-----|---------|
| `RERANK_LLAMACPP_BASE_URL` | `http://localhost:54210` |
| `RERANK_LLAMACPP_MODEL` | `bge-reranker-v2-m3-q4_k_m.gguf` |
| `RERANK_LLAMACPP_PORT` | `14210` |
| `RERANK_LLAMACPP_CTX` | `8192` |
| `RERANK_LLAMACPP_THREADS` | `4` |
| `RERANK_LLAMACPP_PARALLEL` | `4` |
| `MODELS_DIR` | `/cai2_ds_storage/models` |
**Backend service variables (docker-compose only):**
**Shared HF:** `HF_CACHE_DIR` (default `/cai2_ds_storage/hf_cache`), `HF_TOKEN`.
These are consumed by `deploy/docker-compose.yml` to launch the vLLM / llama.cpp containers — they configure the backend server, not the API app.
| Var | Default | Description |
|-----|---------|-------------|
| `RERANK_VLLM_MODEL` | `BAAI/bge-reranker-v2-m3` | HF model id loaded by the vLLM container |
| `RERANK_VLLM_PORT` | `14201` | Host port mapped to the vLLM container |
| `RERANK_VLLM_GPU` | `0` | `CUDA_VISIBLE_DEVICES` for the vLLM container |
| `RERANK_VLLM_GPU_UTIL` | `0.50` | vLLM `--gpu-memory-utilization` |
| `RERANK_VLLM_MAX_LEN` | `8192` | vLLM `--max-model-len` |
| `RERANK_LLAMACPP_MODEL` | `bge-reranker-v2-m3-q4_k_m.gguf` | GGUF filename inside `MODELS_DIR` |
| `RERANK_LLAMACPP_PORT` | `14210` | Host port mapped to the llama.cpp container |
| `RERANK_LLAMACPP_CTX` | `8192` | llama.cpp context size |
| `RERANK_LLAMACPP_THREADS` | `4` | llama.cpp threads |
| `RERANK_LLAMACPP_PARALLEL` | `4` | llama.cpp parallel slots |
| `MODELS_DIR` | `/cai2_ds_storage/models` | GGUF model directory for llama.cpp |
| `HF_CACHE_DIR` | `/cai2_ds_storage/hf_cache` | Mounted into the vLLM container |
| `HF_TOKEN` | (unset) | Forwarded as `HUGGING_FACE_HUB_TOKEN` |
Full list with comments in `.env.example`.