Livrare LOT 1 - Didi

ai_platform/modules/embeddings/README.md

@@ -0,0 +1,183 @@
+# Embeddings Module
+
+OpenAI-compatible embeddings API with support for vLLM and llama.cpp backends.
+
+## Features
+
+- **OpenAI-compatible API**: Drop-in replacement for OpenAI's `/v1/embeddings` endpoint
+- **Multiple backends**: Support for vLLM and llama.cpp
+- **High performance**: Built on FastAPI with async support
+- **Production-ready**: Rate limiting, authentication, health checks
+
+## Prerequisites
+
+### Required
+
+- **Linux** - Ubuntu 22.04+ or similar
+- **Python 3.10+** - Managed via `uv`
+- **uv** - Fast Python package manager
+
+### Optional (for backends)
+
+- **vLLM** - Requires NVIDIA GPU with CUDA 12.x
+- **llama.cpp** - Can run on CPU or GPU
+
+## Quick Start
+
+### 1. Install dependencies
+
+```bash
+cd modules/embeddings
+uv sync --all-extras
+```
+
+### 2. Configure environment
+
+```bash
+cp .env.example .env
+# Edit .env with your settings
+```
+
+### 3. Start backend server
+
+**vLLM (GPU):**
+```bash
+python -m vllm.entrypoints.openai.api_server \
+    --model BAAI/bge-m3 \
+    --host 0.0.0.0 --port 54101 \
+    --task embed
+```
+
+**llama.cpp (CPU):**
+```bash
+llama-server \
+    --model /models/bge-m3-q4_k_m.gguf \
+    --host 0.0.0.0 --port 54110 \
+    --embedding
+```
+
+### 4. Start API server
+
+```bash
+# Set required environment variables
+export EMB_DEFAULT_BACKEND=vllm
+export EMB_ENABLE_VLLM=true
+export EMB_ENABLE_LLAMACPP=false
+export EMB_EXTERNAL_URL=http://localhost:54100
+
+# Run the server
+uv run python -m embeddings.cli --port 54100
+```
+
+### 5. Test the API
+
+```bash
+curl -X POST http://localhost:54100/v1/embeddings \
+  -H "Content-Type: application/json" \
+  -d '{
+    "input": "Hello, world!",
+    "model": "BAAI/bge-m3"
+  }'
+```
+
+## Docker Deployment
+
+```bash
+cd modules/embeddings/deploy
+
+# Copy and configure .env
+cp ../.env.example .env
+# Edit .env with your settings
+
+# Start with vLLM backend
+./deploy.sh --profile vllm -d
+
+# Or start with llama.cpp backend
+./deploy.sh --profile llamacpp -d
+
+# View logs
+./deploy.sh --profile vllm --logs
+
+# Stop
+./deploy.sh --profile vllm --down
+```
+
+## Python Library Usage
+
+```python
+from embeddings import EmbeddingClient
+
+# Initialize client (reads config from environment)
+client = EmbeddingClient()
+
+# Generate embeddings
+response = await client.embed(
+    texts=["Hello, world!", "How are you?"],
+    model="BAAI/bge-m3",
+)
+
+# Access embeddings
+for item in response.data:
+    print(f"Index {item.index}: {len(item.embedding)} dimensions")
+
+# List available models
+models = await client.list_models()
+for model in models:
+    print(f"{model.id} on {model.backend}")
+```
+
+## Configuration
+
+All configuration is via environment variables with the `EMB_` prefix:
+
+| Variable | Required | Default | Description |
+|----------|----------|---------|-------------|
+| `EMB_DEFAULT_BACKEND` | Yes | - | Default backend: `vllm` or `llamacpp` |
+| `EMB_ENABLE_VLLM` | Yes | - | Enable vLLM backend |
+| `EMB_ENABLE_LLAMACPP` | Yes | - | Enable llama.cpp backend |
+| `EMB_EXTERNAL_URL` | Yes | - | External URL for OpenAPI spec |
+| `EMB_PORT` | No | 54100 | API server port |
+| `EMB_VLLM_BASE_URL` | No | http://localhost:54101 | vLLM server URL |
+| `EMB_LLAMACPP_BASE_URL` | No | http://localhost:54110 | llama.cpp server URL |
+| `EMB_API_TOKENS` | No | - | Comma-separated API tokens |
+| `EMB_RATE_LIMIT_RPS` | No | 20.0 | Requests per second limit |
+| `EMB_MAX_CONCURRENT_REQUESTS` | No | 20 | Max concurrent requests |
+
+See `.env.example` for the complete list.
+
+## Port Allocation
+
+Following the datacenter port schema (x41xx = Embeddings):
+
+| Port | Service | Environment |
+|------|---------|-------------|
+| 14100 | Embeddings API | Production |
+| 54100 | Embeddings API | Development |
+| 14101 | vLLM Embed Server | Production |
+| 54101 | vLLM Embed Server | Development |
+| 14110 | llama.cpp Embed Server | Production |
+| 54110 | llama.cpp Embed Server | Development |
+
+## Development
+
+```bash
+# Install dev dependencies
+uv sync --all-extras
+
+# Run tests
+uv run pytest
+
+# Run tests with coverage
+uv run pytest --cov=src/embeddings --cov-report=term-missing
+
+# Lint and format
+uv run ruff check .
+uv run ruff format .
+
+# Type check
+uv run mypy src/
+```
+
+## API Reference
+
+See [API.md](API.md) for the complete API documentation.