didi-lot1-ai/ai_platform/modules/rerank/API.md

Rerank API Reference

Cohere/Jina-compatible reranking API with multiple backend support.

Base URL

{BASE_URL}

• Local development: http://localhost:54200
• Docker (internal): http://didiAI-rerank-api:14200
• Production: Use your configured hostname

Authentication

Authentication is optional. If RERANK_API_TOKENS is set, requests require a Bearer token:

Authorization: Bearer <token>

Health endpoints (/health, /ready) are always public.

Endpoints

Rerank Documents

Rerank documents against a query based on relevance.

Endpoints:

• POST /v1/rerank
• POST /v2/rerank (alias)

Request Headers:

Header	Required	Description
Content-Type	Yes	Must be application/json
Authorization	If auth enabled	Bearer <token>

Request Body:

{
  "model": "BAAI/bge-reranker-v2-m3",
  "query": "What is machine learning?",
  "documents": [
    "Machine learning is a subset of AI",
    "Cats are pets",
    "Deep learning uses neural networks"
  ],
  "top_n": 3,
  "return_documents": false,
  "backend": null
}

Field	Type	Required	Description
model	string	Yes	Model identifier for reranking
query	string	Yes	The search query
documents	string[]	Yes	Documents to rerank (1-1000)
top_n	integer	No	Number of results to return (default: all)
return_documents	boolean	No	Include document text in response
backend	string	No	Override default backend: "vllm" or "llamacpp"

Response:

{
  "id": "rerank-abc123def456",
  "model": "BAAI/bge-reranker-v2-m3",
  "results": [
    {
      "index": 2,
      "relevance_score": 0.9523,
      "document": null
    },
    {
      "index": 0,
      "relevance_score": 0.8876,
      "document": null
    },
    {
      "index": 1,
      "relevance_score": 0.0234,
      "document": null
    }
  ],
  "usage": {
    "total_tokens": 150
  },
  "backend": "vllm"
}

Example:

curl -X POST http://localhost:54200/v1/rerank \
  -H "Content-Type: application/json" \
  -d '{
    "model": "BAAI/bge-reranker-v2-m3",
    "query": "What is machine learning?",
    "documents": [
      "Machine learning is a subset of AI",
      "Cats are pets",
      "Deep learning uses neural networks"
    ]
  }'

Get top 2 results:

curl -X POST http://localhost:54200/v1/rerank \
  -H "Content-Type: application/json" \
  -d '{
    "model": "BAAI/bge-reranker-v2-m3",
    "query": "programming languages",
    "documents": ["Python", "Java", "Cooking", "C++", "Hiking"],
    "top_n": 2
  }'

Include documents in response:

curl -X POST http://localhost:54200/v1/rerank \
  -H "Content-Type: application/json" \
  -d '{
    "model": "BAAI/bge-reranker-v2-m3",
    "query": "AI",
    "documents": ["Machine learning", "Deep learning", "Recipes"],
    "return_documents": true
  }'

---

List Models

List available reranking models.

Endpoint: GET /v1/models

Query Parameters:

Parameter	Type	Required	Description
backend	string	No	Filter by backend: "vllm" or "llamacpp"

Response:

{
  "object": "list",
  "data": [
    {
      "id": "BAAI/bge-reranker-v2-m3",
      "backend": "vllm",
      "loaded": true,
      "max_input_tokens": null
    }
  ]
}

Example:

# List all models
curl http://localhost:54200/v1/models

# List models from specific backend
curl "http://localhost:54200/v1/models?backend=vllm"

---

List Backends

List available backends.

Endpoint: GET /v1/backends

Response:

{
  "backends": ["vllm", "llamacpp"]
}

Example:

curl http://localhost:54200/v1/backends

---

Health Check

Detailed health status including per-backend health.

Endpoint: GET /health

Response:

{
  "status": "healthy",
  "backends": [
    {
      "name": "vllm",
      "healthy": true,
      "message": null
    }
  ]
}

Status values:

• "healthy" - All backends are healthy
• "degraded" - Some backends are unhealthy
• "unhealthy" - All backends are unhealthy

Example:

curl http://localhost:54200/health

---

Readiness Probe

Simple readiness check for Kubernetes.

Endpoint: GET /ready

Response:

{
  "ready": true
}

Example:

curl http://localhost:54200/ready

---

Error Responses

All errors follow this format:

{
  "detail": "Error message describing what went wrong"
}

HTTP Status Codes

Code	Meaning
200	Success
400	Bad request (invalid parameters, backend not enabled)
401	Authentication required or failed
429	Rate limit exceeded
503	Service unavailable (backend connection failed)
504	Gateway timeout (backend request timed out)

Rate Limit Response

{
  "detail": "Too many requests",
  "retry_after": 1.5
}

Headers include: Retry-After: 2

---

Request Headers

Header	Required	Description
Content-Type	Yes (POST)	Must be application/json
Authorization	If auth enabled	Bearer <token>
X-Request-ID	No	Request tracking ID (generated if not provided)

Response always includes X-Request-ID header for tracking.

---

SDK Examples

Python (httpx)

import httpx

async def rerank_documents(
    query: str,
    documents: list[str],
    top_n: int | None = None,
) -> list[tuple[int, float]]:
    async with httpx.AsyncClient() as client:
        response = await client.post(
            "http://localhost:54200/v1/rerank",
            json={
                "model": "BAAI/bge-reranker-v2-m3",
                "query": query,
                "documents": documents,
                "top_n": top_n,
            },
            headers={"Authorization": "Bearer your-token"},
        )
        response.raise_for_status()
        data = response.json()
        return [(r["index"], r["relevance_score"]) for r in data["results"]]

Python (requests)

import requests

def rerank_documents(query: str, documents: list[str]) -> dict:
    response = requests.post(
        "http://localhost:54200/v1/rerank",
        json={
            "model": "BAAI/bge-reranker-v2-m3",
            "query": query,
            "documents": documents,
        },
    )
    response.raise_for_status()
    return response.json()

# Usage
result = rerank_documents(
    "What is AI?",
    ["Machine learning", "Deep learning", "Cooking recipes"]
)
for r in result["results"]:
    print(f"Doc {r['index']}: {r['relevance_score']:.4f}")

curl

# Basic reranking
curl -X POST http://localhost:54200/v1/rerank \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer your-token" \
  -d '{
    "model": "BAAI/bge-reranker-v2-m3",
    "query": "programming",
    "documents": ["Python", "Java", "Cooking"]
  }'

# With specific backend
curl -X POST http://localhost:54200/v1/rerank \
  -H "Content-Type: application/json" \
  -d '{
    "model": "BAAI/bge-reranker-v2-m3",
    "query": "programming",
    "documents": ["Python", "Java"],
    "backend": "vllm"
  }'

# v2 endpoint (identical)
curl -X POST http://localhost:54200/v2/rerank \
  -H "Content-Type: application/json" \
  -d '{
    "model": "BAAI/bge-reranker-v2-m3",
    "query": "test",
    "documents": ["doc1", "doc2"]
  }'