didi-lot1-ai/ai_platform/modules/rerank/API.md

# Rerank API Reference

Cohere/Jina-compatible reranking API with multiple backend support.

## Base URL

```
{BASE_URL}
```

- **Local development:** `http://localhost:54200`
- **Docker (internal):** `http://didiAI-rerank-api:14200`
- **Production:** Use your configured hostname

## Authentication

Authentication is **optional**. If `RERANK_API_TOKENS` is set, requests require a Bearer token:

```
Authorization: Bearer <token>
```

Health endpoints (`/health`, `/ready`) are always public.

## Endpoints

### Rerank Documents

Rerank documents against a query based on relevance.

**Endpoints:**
- `POST /v1/rerank`
- `POST /v2/rerank` (alias)

**Request Headers:**
| Header | Required | Description |
|--------|----------|-------------|
| `Content-Type` | Yes | Must be `application/json` |
| `Authorization` | If auth enabled | `Bearer <token>` |

**Request Body:**
```json
{
  "model": "BAAI/bge-reranker-v2-m3",
  "query": "What is machine learning?",
  "documents": [
    "Machine learning is a subset of AI",
    "Cats are pets",
    "Deep learning uses neural networks"
  ],
  "top_n": 3,
  "return_documents": false,
  "backend": null
}
```

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `model` | string | Yes | Model identifier for reranking |
| `query` | string | Yes | The search query |
| `documents` | string[] | Yes | Documents to rerank (1-1000) |
| `top_n` | integer | No | Number of results to return (default: all) |
| `return_documents` | boolean | No | Include document text in response |
| `backend` | string | No | Override default backend: `"vllm"` or `"llamacpp"` |

**Response:**
```json
{
  "id": "rerank-abc123def456",
  "model": "BAAI/bge-reranker-v2-m3",
  "results": [
    {
      "index": 2,
      "relevance_score": 0.9523,
      "document": null
    },
    {
      "index": 0,
      "relevance_score": 0.8876,
      "document": null
    },
    {
      "index": 1,
      "relevance_score": 0.0234,
      "document": null
    }
  ],
  "usage": {
    "total_tokens": 150
  },
  "backend": "vllm"
}
```

**Example:**
```bash
curl -X POST http://localhost:54200/v1/rerank \
  -H "Content-Type: application/json" \
  -d '{
    "model": "BAAI/bge-reranker-v2-m3",
    "query": "What is machine learning?",
    "documents": [
      "Machine learning is a subset of AI",
      "Cats are pets",
      "Deep learning uses neural networks"
    ]
  }'
```

**Get top 2 results:**
```bash
curl -X POST http://localhost:54200/v1/rerank \
  -H "Content-Type: application/json" \
  -d '{
    "model": "BAAI/bge-reranker-v2-m3",
    "query": "programming languages",
    "documents": ["Python", "Java", "Cooking", "C++", "Hiking"],
    "top_n": 2
  }'
```

**Include documents in response:**
```bash
curl -X POST http://localhost:54200/v1/rerank \
  -H "Content-Type: application/json" \
  -d '{
    "model": "BAAI/bge-reranker-v2-m3",
    "query": "AI",
    "documents": ["Machine learning", "Deep learning", "Recipes"],
    "return_documents": true
  }'
```

---

### List Models

List available reranking models.

**Endpoint:** `GET /v1/models`

**Query Parameters:**
| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `backend` | string | No | Filter by backend: `"vllm"` or `"llamacpp"` |

**Response:**
```json
{
  "object": "list",
  "data": [
    {
      "id": "BAAI/bge-reranker-v2-m3",
      "backend": "vllm",
      "loaded": true,
      "max_input_tokens": null
    }
  ]
}
```

**Example:**
```bash
# List all models
curl http://localhost:54200/v1/models

# List models from specific backend
curl "http://localhost:54200/v1/models?backend=vllm"
```

---

### List Backends

List available backends.

**Endpoint:** `GET /v1/backends`

**Response:**
```json
{
  "backends": ["vllm", "llamacpp"]
}
```

**Example:**
```bash
curl http://localhost:54200/v1/backends
```

---

### Health Check

Detailed health status including per-backend health.

**Endpoint:** `GET /health`

**Response:**
```json
{
  "status": "healthy",
  "backends": [
    {
      "name": "vllm",
      "healthy": true,
      "message": null
    }
  ]
}
```

Status values:
- `"healthy"` - All backends are healthy
- `"degraded"` - Some backends are unhealthy
- `"unhealthy"` - All backends are unhealthy

**Example:**
```bash
curl http://localhost:54200/health
```

---

### Readiness Probe

Simple readiness check for Kubernetes.

**Endpoint:** `GET /ready`

**Response:**
```json
{
  "ready": true
}
```

**Example:**
```bash
curl http://localhost:54200/ready
```

---

## Error Responses

All errors follow this format:

```json
{
  "detail": "Error message describing what went wrong"
}
```

### HTTP Status Codes

| Code | Meaning |
|------|---------|
| 200 | Success |
| 400 | Bad request (invalid parameters, backend not enabled) |
| 401 | Authentication required or failed |
| 429 | Rate limit exceeded |
| 503 | Service unavailable (backend connection failed) |
| 504 | Gateway timeout (backend request timed out) |

### Rate Limit Response

```json
{
  "detail": "Too many requests",
  "retry_after": 1.5
}
```

Headers include: `Retry-After: 2`

---

## Request Headers

| Header | Required | Description |
|--------|----------|-------------|
| `Content-Type` | Yes (POST) | Must be `application/json` |
| `Authorization` | If auth enabled | `Bearer <token>` |
| `X-Request-ID` | No | Request tracking ID (generated if not provided) |

Response always includes `X-Request-ID` header for tracking.

---

## SDK Examples

### Python (httpx)

```python
import httpx

async def rerank_documents(
    query: str,
    documents: list[str],
    top_n: int | None = None,
) -> list[tuple[int, float]]:
    async with httpx.AsyncClient() as client:
        response = await client.post(
            "http://localhost:54200/v1/rerank",
            json={
                "model": "BAAI/bge-reranker-v2-m3",
                "query": query,
                "documents": documents,
                "top_n": top_n,
            },
            headers={"Authorization": "Bearer your-token"},
        )
        response.raise_for_status()
        data = response.json()
        return [(r["index"], r["relevance_score"]) for r in data["results"]]
```

### Python (requests)

```python
import requests

def rerank_documents(query: str, documents: list[str]) -> dict:
    response = requests.post(
        "http://localhost:54200/v1/rerank",
        json={
            "model": "BAAI/bge-reranker-v2-m3",
            "query": query,
            "documents": documents,
        },
    )
    response.raise_for_status()
    return response.json()

# Usage
result = rerank_documents(
    "What is AI?",
    ["Machine learning", "Deep learning", "Cooking recipes"]
)
for r in result["results"]:
    print(f"Doc {r['index']}: {r['relevance_score']:.4f}")
```

### curl

```bash
# Basic reranking
curl -X POST http://localhost:54200/v1/rerank \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer your-token" \
  -d '{
    "model": "BAAI/bge-reranker-v2-m3",
    "query": "programming",
    "documents": ["Python", "Java", "Cooking"]
  }'

# With specific backend
curl -X POST http://localhost:54200/v1/rerank \
  -H "Content-Type: application/json" \
  -d '{
    "model": "BAAI/bge-reranker-v2-m3",
    "query": "programming",
    "documents": ["Python", "Java"],
    "backend": "vllm"
  }'

# v2 endpoint (identical)
curl -X POST http://localhost:54200/v2/rerank \
  -H "Content-Type: application/json" \
  -d '{
    "model": "BAAI/bge-reranker-v2-m3",
    "query": "test",
    "documents": ["doc1", "doc2"]
  }'
```