didi-lot1-ai/ai_platform/modules/web/API.md

11 KiB

Web API Documentation

Base URL

{BASE_URL}
  • Local development: http://localhost:51100
  • Docker (internal): http://web-api:51100
  • Production: Use your configured hostname

Authentication

Authentication is optional. If WEB_API_TOKENS is set, Bearer token authentication is required.

Authorization: Bearer <token>

If authentication is disabled (default), no Authorization header is needed.

Endpoints

Gather (Main Endpoint)

Execute the full evidence-gathering pipeline: search → fetch → evidence.

POST /v1/gather

Request Body

{
  "claim": "The claim to verify",
  "search_queries": ["optional custom queries"],
  "max_search_results": 10,
  "site_allowlist": ["reuters.com"],
  "site_blocklist": ["spam-site.com"],
  "fetch_method": "auto",
  "auto_fallback": true,
  "extract_snippets": false,
  "max_evidence_items": 15,
  "dedupe": true,
  "timeout_seconds": 90.0
}
Field Type Required Default Description
claim string Yes - Claim to gather evidence for (10-1000 chars)
search_queries array[string] No null Custom search queries (auto-generated if not provided)
max_search_results integer No 10 Max search results (5-50)
site_allowlist array[string] No null Only search these domains
site_blocklist array[string] No null Exclude these domains
fetch_method string No "auto" "auto", "http", "browse", "vision"
auto_fallback boolean No true Escalate on fetch failure
extract_snippets boolean No False Deprecated — LLM snippet extraction (no longer used by default)
max_evidence_items integer No 15 Max items in final pack (1-50)
dedupe boolean No true Deduplicate evidence
parallel_fetches integer No 5 Concurrent fetch operations (1-10)
timeout_seconds number No 90.0 Total pipeline timeout (10-300)

Response

{
  "request_id": "uuid",
  "claim": "The claim",
  "evidence": [
    {
      "url": "https://example.com/article",
      "title": "Article Title",
      "publisher": "example.com",
      "snippet": "Relevant excerpt...",
      "relevance_score": 0.8,
      "credibility_score": null,
      "published_at": "2024-01-15",
      "retrieved_at": "2025-01-20T10:00:00Z",
      "full_text_hash": "sha256...",
      "provenance": {
        "extraction_method": "http",
        "fallback_chain": []
      }
    }
  ],
  "evidence_stats": {
    "input_items": 10,
    "after_dedup": 8,
    "output_items": 8,
    "duplicates_removed": 2,
    "tokens_used": 1500
  },
  "stages": [
    {"stage": "search", "success": true, "items_processed": 10, "items_failed": 0, "duration_ms": 500.0},
    {"stage": "fetch", "success": true, "items_processed": 8, "items_failed": 2, "duration_ms": 5000.0},
    {"stage": "evidence", "success": true, "items_processed": 8, "items_failed": 0, "duration_ms": 3000.0}
  ],
  "total_urls_found": 10,
  "total_pages_fetched": 8,
  "total_evidence_items": 8,
  "execution_time_ms": 8500.0
}

Example

curl -X POST http://localhost:51100/v1/gather \
  -H "Content-Type: application/json" \
  -d '{
    "claim": "Romania had the highest economic growth in the EU in 2024",
    "max_search_results": 5,
    "extract_snippets": true,
    "max_evidence_items": 5
  }'

Execute web search queries.

POST /v1/search

Request Body

{
  "queries": ["string"],
  "max_results": 10,
  "site_allowlist": ["string"],
  "site_blocklist": ["string"],
  "language": "auto",
  "country": "US",
  "freshness": "month",
  "safe_search": "moderate"
}
Field Type Required Default Description
queries array[string] Yes - Search queries (1-10)
max_results integer No 10 Results per query (1-100)
site_allowlist array[string] No null Only include these domains
site_blocklist array[string] No null Exclude these domains
language string No "auto" Search language (ISO 639-1, or "auto")
country string No "US" Search country (ISO 3166-1)
freshness string No null Filter by age: day, week, month, year
safe_search string No "moderate" off, moderate, strict

Response

{
  "request_id": "uuid",
  "results": [
    {
      "query": "string",
      "url": "string",
      "title": "string",
      "snippet": "string",
      "rank": 1,
      "site": "string",
      "published_at": "string|null"
    }
  ],
  "total_results": 10,
  "execution_time_ms": 150.5,
  "queries_processed": 1
}

Example

curl -X POST http://localhost:51100/v1/search \
  -H "Content-Type: application/json" \
  -d '{
    "queries": ["climate change effects", "renewable energy"],
    "max_results": 5,
    "site_allowlist": ["reuters.com", "bbc.com"],
    "freshness": "month"
  }'

Search for images using SearXNG image search.

POST /v1/image-search

Request Body

{
  "queries": ["solar eclipse"],
  "max_results": 50,
  "language": "en",
  "country": "US",
  "safe_search": "strict",
  "spellcheck": true
}
Field Type Required Default Description
queries array[string] Yes - Search queries (1-10)
max_results integer No 50 Results per query (1-200)
language string No "en" Search language (ISO 639-1)
country string No "US" Search country (ISO 3166-1)
safe_search string No "strict" off or strict
spellcheck boolean No true Enable spellcheck

Response

{
  "request_id": "uuid",
  "results": [
    {
      "query": "solar eclipse",
      "image_url": "https://example.com/eclipse.jpg",
      "thumbnail_url": "https://example.com/eclipse_thumb.jpg",
      "source_url": "https://example.com/article",
      "title": "Solar Eclipse Photo",
      "description": "A total solar eclipse captured in 2024",
      "width": 1920,
      "height": 1080,
      "publisher": "example.com",
      "rank": 1
    }
  ],
  "total_results": 50,
  "execution_time_ms": 200.5,
  "queries_processed": 1
}

Example

curl -X POST http://localhost:51100/v1/image-search \
  -H "Content-Type: application/json" \
  -d '{
    "queries": ["solar eclipse", "northern lights"],
    "max_results": 5,
    "safe_search": "strict"
  }'

Fetch

Fetch and extract content from URLs with automatic fallback (HTTP → Playwright → Vision LLM).

POST /v1/fetch

Request Body

{
  "urls": ["https://example.com/article"],
  "extract_text": true,
  "include_html": false,
  "extract_metadata": true,
  "auto_fallback": true,
  "method": "auto",
  "timeout_seconds": 30.0,
  "min_text_length": 200,
  "parallel_fetches": 5
}
Field Type Required Default Description
urls array[string] Yes - URLs to fetch (1-50)
extract_text boolean No true Extract main text content
include_html boolean No false Include raw HTML
extract_metadata boolean No true Extract metadata
auto_fallback boolean No true Auto-escalate on failure
method string No "auto" "auto", "http", "browse", "vision"
timeout_seconds number No 30.0 Timeout per URL (1-120)
min_text_length integer No 200 Min text before fallback
parallel_fetches integer No 5 Concurrent fetches (1-20)

Response

{
  "request_id": "uuid",
  "pages": [
    {
      "url": "https://example.com/article",
      "title": "Article Title",
      "text": "Extracted text content...",
      "text_hash": "sha256...",
      "extraction_method": "http",
      "fallback_chain": [],
      "retrieved_at": "2025-01-20T10:00:00Z",
      "extraction_time_ms": 500.0,
      "status_code": 200,
      "content_type": "text/html"
    }
  ],
  "total_fetched": 1,
  "total_failed": 0,
  "execution_time_ms": 600.0,
  "failed_urls": []
}

Example

curl -X POST http://localhost:51100/v1/fetch \
  -H "Content-Type: application/json" \
  -d '{
    "urls": ["https://en.wikipedia.org/wiki/Climate_change"],
    "method": "auto",
    "auto_fallback": true
  }'

Info

Get component information for service catalog.

GET /v1/info

Response

Returns complete metadata about this service including resource information, available functions, and their schemas. Used by the catalog-api for service discovery.

Example

curl http://localhost:51100/v1/info

Health Check

Check service health and provider status.

GET /health

Response

{
  "status": "healthy",
  "providers": [
    {
      "name": "searxng",
      "healthy": true,
      "message": null
    }
  ]
}
Status Description
healthy All providers operational
degraded Some providers unavailable
unhealthy All providers down

Readiness Probe

Check if service is ready to accept requests.

GET /ready

Response

{
  "ready": true
}

Error Responses

Error Format

{
  "detail": {
    "error": "error_code",
    "message": "Human readable message",
    "provider": "searxng"
  }
}

HTTP Status Codes

Code Error Description
400 validation_error Invalid request body
401 authentication_required Missing or invalid auth token
422 validation_error Request validation failed
429 rate_limit_exceeded Too many requests
502 provider_error Search provider error
503 service_unavailable Concurrency limit reached
504 timeout Request timeout

Request Headers

Header Required Description
Content-Type Yes Must be application/json
Authorization Conditional Bearer token (if auth enabled)
X-Request-ID No Custom request ID for tracing

Rate Limiting

The API implements token bucket rate limiting:

  • Default: 10 requests/second with burst of 20
  • 429 Response: Includes Retry-After header

Note: Rate limiting is per-process. In multi-replica deployments, use external rate limiting.


SDK Example (Python)

import httpx

async def search(query: str, base_url: str = "http://localhost:51100"):
    async with httpx.AsyncClient() as client:
        response = await client.post(
            f"{base_url}/v1/search",
            json={
                "queries": [query],
                "max_results": 10,
            },
        )
        response.raise_for_status()
        return response.json()

# Usage
results = await search("climate change")
for r in results["results"]:
    print(f"{r['title']}: {r['url']}")