11 KiB
Web API Documentation
Base URL
{BASE_URL}
- Local development:
http://localhost:51100 - Docker (internal):
http://web-api:51100 - Production: Use your configured hostname
Authentication
Authentication is optional. If WEB_API_TOKENS is set, Bearer token authentication is required.
Authorization: Bearer <token>
If authentication is disabled (default), no Authorization header is needed.
Endpoints
Gather (Main Endpoint)
Execute the full evidence-gathering pipeline: search → fetch → evidence.
POST /v1/gather
Request Body
{
"claim": "The claim to verify",
"search_queries": ["optional custom queries"],
"max_search_results": 10,
"site_allowlist": ["reuters.com"],
"site_blocklist": ["spam-site.com"],
"fetch_method": "auto",
"auto_fallback": true,
"extract_snippets": false,
"max_evidence_items": 15,
"dedupe": true,
"timeout_seconds": 90.0
}
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
claim |
string | Yes | - | Claim to gather evidence for (10-1000 chars) |
search_queries |
array[string] | No | null | Custom search queries (auto-generated if not provided) |
max_search_results |
integer | No | 10 | Max search results (5-50) |
site_allowlist |
array[string] | No | null | Only search these domains |
site_blocklist |
array[string] | No | null | Exclude these domains |
fetch_method |
string | No | "auto" | "auto", "http", "browse", "vision" |
auto_fallback |
boolean | No | true | Escalate on fetch failure |
extract_snippets |
boolean | No | False | Deprecated — LLM snippet extraction (no longer used by default) |
max_evidence_items |
integer | No | 15 | Max items in final pack (1-50) |
dedupe |
boolean | No | true | Deduplicate evidence |
parallel_fetches |
integer | No | 5 | Concurrent fetch operations (1-10) |
timeout_seconds |
number | No | 90.0 | Total pipeline timeout (10-300) |
Response
{
"request_id": "uuid",
"claim": "The claim",
"evidence": [
{
"url": "https://example.com/article",
"title": "Article Title",
"publisher": "example.com",
"snippet": "Relevant excerpt...",
"relevance_score": 0.8,
"credibility_score": null,
"published_at": "2024-01-15",
"retrieved_at": "2025-01-20T10:00:00Z",
"full_text_hash": "sha256...",
"provenance": {
"extraction_method": "http",
"fallback_chain": []
}
}
],
"evidence_stats": {
"input_items": 10,
"after_dedup": 8,
"output_items": 8,
"duplicates_removed": 2,
"tokens_used": 1500
},
"stages": [
{"stage": "search", "success": true, "items_processed": 10, "items_failed": 0, "duration_ms": 500.0},
{"stage": "fetch", "success": true, "items_processed": 8, "items_failed": 2, "duration_ms": 5000.0},
{"stage": "evidence", "success": true, "items_processed": 8, "items_failed": 0, "duration_ms": 3000.0}
],
"total_urls_found": 10,
"total_pages_fetched": 8,
"total_evidence_items": 8,
"execution_time_ms": 8500.0
}
Example
curl -X POST http://localhost:51100/v1/gather \
-H "Content-Type: application/json" \
-d '{
"claim": "Romania had the highest economic growth in the EU in 2024",
"max_search_results": 5,
"extract_snippets": true,
"max_evidence_items": 5
}'
Search
Execute web search queries.
POST /v1/search
Request Body
{
"queries": ["string"],
"max_results": 10,
"site_allowlist": ["string"],
"site_blocklist": ["string"],
"language": "auto",
"country": "US",
"freshness": "month",
"safe_search": "moderate"
}
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
queries |
array[string] | Yes | - | Search queries (1-10) |
max_results |
integer | No | 10 | Results per query (1-100) |
site_allowlist |
array[string] | No | null | Only include these domains |
site_blocklist |
array[string] | No | null | Exclude these domains |
language |
string | No | "auto" | Search language (ISO 639-1, or "auto") |
country |
string | No | "US" | Search country (ISO 3166-1) |
freshness |
string | No | null | Filter by age: day, week, month, year |
safe_search |
string | No | "moderate" | off, moderate, strict |
Response
{
"request_id": "uuid",
"results": [
{
"query": "string",
"url": "string",
"title": "string",
"snippet": "string",
"rank": 1,
"site": "string",
"published_at": "string|null"
}
],
"total_results": 10,
"execution_time_ms": 150.5,
"queries_processed": 1
}
Example
curl -X POST http://localhost:51100/v1/search \
-H "Content-Type: application/json" \
-d '{
"queries": ["climate change effects", "renewable energy"],
"max_results": 5,
"site_allowlist": ["reuters.com", "bbc.com"],
"freshness": "month"
}'
Image Search
Search for images using SearXNG image search.
POST /v1/image-search
Request Body
{
"queries": ["solar eclipse"],
"max_results": 50,
"language": "en",
"country": "US",
"safe_search": "strict",
"spellcheck": true
}
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
queries |
array[string] | Yes | - | Search queries (1-10) |
max_results |
integer | No | 50 | Results per query (1-200) |
language |
string | No | "en" | Search language (ISO 639-1) |
country |
string | No | "US" | Search country (ISO 3166-1) |
safe_search |
string | No | "strict" | off or strict |
spellcheck |
boolean | No | true | Enable spellcheck |
Response
{
"request_id": "uuid",
"results": [
{
"query": "solar eclipse",
"image_url": "https://example.com/eclipse.jpg",
"thumbnail_url": "https://example.com/eclipse_thumb.jpg",
"source_url": "https://example.com/article",
"title": "Solar Eclipse Photo",
"description": "A total solar eclipse captured in 2024",
"width": 1920,
"height": 1080,
"publisher": "example.com",
"rank": 1
}
],
"total_results": 50,
"execution_time_ms": 200.5,
"queries_processed": 1
}
Example
curl -X POST http://localhost:51100/v1/image-search \
-H "Content-Type: application/json" \
-d '{
"queries": ["solar eclipse", "northern lights"],
"max_results": 5,
"safe_search": "strict"
}'
Fetch
Fetch and extract content from URLs with automatic fallback (HTTP → Playwright → Vision LLM).
POST /v1/fetch
Request Body
{
"urls": ["https://example.com/article"],
"extract_text": true,
"include_html": false,
"extract_metadata": true,
"auto_fallback": true,
"method": "auto",
"timeout_seconds": 30.0,
"min_text_length": 200,
"parallel_fetches": 5
}
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
urls |
array[string] | Yes | - | URLs to fetch (1-50) |
extract_text |
boolean | No | true | Extract main text content |
include_html |
boolean | No | false | Include raw HTML |
extract_metadata |
boolean | No | true | Extract metadata |
auto_fallback |
boolean | No | true | Auto-escalate on failure |
method |
string | No | "auto" | "auto", "http", "browse", "vision" |
timeout_seconds |
number | No | 30.0 | Timeout per URL (1-120) |
min_text_length |
integer | No | 200 | Min text before fallback |
parallel_fetches |
integer | No | 5 | Concurrent fetches (1-20) |
Response
{
"request_id": "uuid",
"pages": [
{
"url": "https://example.com/article",
"title": "Article Title",
"text": "Extracted text content...",
"text_hash": "sha256...",
"extraction_method": "http",
"fallback_chain": [],
"retrieved_at": "2025-01-20T10:00:00Z",
"extraction_time_ms": 500.0,
"status_code": 200,
"content_type": "text/html"
}
],
"total_fetched": 1,
"total_failed": 0,
"execution_time_ms": 600.0,
"failed_urls": []
}
Example
curl -X POST http://localhost:51100/v1/fetch \
-H "Content-Type: application/json" \
-d '{
"urls": ["https://en.wikipedia.org/wiki/Climate_change"],
"method": "auto",
"auto_fallback": true
}'
Info
Get component information for service catalog.
GET /v1/info
Response
Returns complete metadata about this service including resource information, available functions, and their schemas. Used by the catalog-api for service discovery.
Example
curl http://localhost:51100/v1/info
Health Check
Check service health and provider status.
GET /health
Response
{
"status": "healthy",
"providers": [
{
"name": "searxng",
"healthy": true,
"message": null
}
]
}
| Status | Description |
|---|---|
healthy |
All providers operational |
degraded |
Some providers unavailable |
unhealthy |
All providers down |
Readiness Probe
Check if service is ready to accept requests.
GET /ready
Response
{
"ready": true
}
Error Responses
Error Format
{
"detail": {
"error": "error_code",
"message": "Human readable message",
"provider": "searxng"
}
}
HTTP Status Codes
| Code | Error | Description |
|---|---|---|
| 400 | validation_error |
Invalid request body |
| 401 | authentication_required |
Missing or invalid auth token |
| 422 | validation_error |
Request validation failed |
| 429 | rate_limit_exceeded |
Too many requests |
| 502 | provider_error |
Search provider error |
| 503 | service_unavailable |
Concurrency limit reached |
| 504 | timeout |
Request timeout |
Request Headers
| Header | Required | Description |
|---|---|---|
Content-Type |
Yes | Must be application/json |
Authorization |
Conditional | Bearer token (if auth enabled) |
X-Request-ID |
No | Custom request ID for tracing |
Rate Limiting
The API implements token bucket rate limiting:
- Default: 10 requests/second with burst of 20
- 429 Response: Includes
Retry-Afterheader
Note: Rate limiting is per-process. In multi-replica deployments, use external rate limiting.
SDK Example (Python)
import httpx
async def search(query: str, base_url: str = "http://localhost:51100"):
async with httpx.AsyncClient() as client:
response = await client.post(
f"{base_url}/v1/search",
json={
"queries": [query],
"max_results": 10,
},
)
response.raise_for_status()
return response.json()
# Usage
results = await search("climate change")
for r in results["results"]:
print(f"{r['title']}: {r['url']}")