didi-lot1-ai/ai_platform/modules/web/benchmark/RESULTS.md

Web API Benchmark Results

Date: 2026-02-08 Base URL: http://localhost:51100 Iterations: 3 (+ 1 warmup)

Summary

Endpoint	Scenarios	Avg Latency	Status
Health	1	~10ms	OK
Search (/v1/search)	8	1181ms	OK
Fetch (/v1/fetch)	7	939ms	OK
Image Search (/v1/image-search)	2	1469ms	3 errors
Gather (/v1/gather)	9	6165ms	OK

---

Health

health

Metric	Min	Mean	Median	P95	Max
curl_ttfb_s	0.01	0.01	0.01	0.01	0.01
curl_total_s	0.01	0.01	0.01	0.01	0.01

---

Search (/v1/search)

search-single

Config: 1 query, max_results=10

Metric	Min	Mean	Median	P95	Max
total_ms	574	661	650	749	760
curl_ttfb_s	0.58	0.67	0.66	0.75	0.76
curl_total_s	0.58	0.67	0.66	0.75	0.76

Results: 10 items returned

search-multi-query

Config: 5 queries, max_results=10

Metric	Min	Mean	Median	P95	Max
total_ms	1775	1936	1972	2053	2062
curl_ttfb_s	1.78	1.94	1.98	2.06	2.07
curl_total_s	1.78	1.94	1.98	2.06	2.07

Results: 50 items returned

search-max-queries

Config: 10 queries, max_results=5

Metric	Min	Mean	Median	P95	Max
total_ms	2776	2945	3012	3045	3049
curl_ttfb_s	2.78	2.95	3.02	3.05	3.05
curl_total_s	2.78	2.95	3.02	3.05	3.05

Results: 50 items returned

search-large-results

Config: 2 queries, max_results=50

Metric	Min	Mean	Median	P95	Max
total_ms	930	1087	1105	1213	1225
curl_ttfb_s	0.94	1.09	1.11	1.22	1.23
curl_total_s	0.94	1.09	1.11	1.22	1.23

Results: 37 items returned

search-freshness-day

Config: 1 query, max_results=10, freshness=day

Metric	Min	Mean	Median	P95	Max
total_ms	662	690	704	705	706
curl_ttfb_s	0.67	0.70	0.71	0.71	0.71
curl_total_s	0.67	0.70	0.71	0.71	0.71

Results: 10 items returned

search-freshness-week

Config: 1 query, max_results=10, freshness=week

Metric	Min	Mean	Median	P95	Max
total_ms	573	623	614	675	681
curl_ttfb_s	0.58	0.63	0.62	0.68	0.69
curl_total_s	0.58	0.63	0.62	0.68	0.69

Results: 10 items returned

search-site-filter

Config: 1 query, max_results=20, site_allowlist=[wikipedia.org, python.org, realpython.com]

Metric	Min	Mean	Median	P95	Max
total_ms	586	701	612	876	905
curl_ttfb_s	0.59	0.71	0.62	0.88	0.91
curl_total_s	0.59	0.71	0.62	0.88	0.91

Results: 9 items returned

search-non-english

Config: 1 query (French), max_results=10, language=fr, country=FR

Metric	Min	Mean	Median	P95	Max
total_ms	570	805	779	1036	1065
curl_ttfb_s	0.58	0.81	0.78	1.04	1.07
curl_total_s	0.58	0.81	0.79	1.04	1.07

Results: 10 items returned

---

Fetch (/v1/fetch)

fetch-single

Config: 1 URL

Metric	Min	Mean	Median	P95	Max
total_ms	138	142	144	145	146
curl_ttfb_s	0.14	0.15	0.15	0.15	0.15
curl_total_s	0.14	0.15	0.15	0.15	0.15

Results: 1 item returned

fetch-multi

Config: 5 URLs, parallel_fetches=5

Metric	Min	Mean	Median	P95	Max
total_ms	481	512	496	552	558
curl_ttfb_s	0.49	0.52	0.50	0.56	0.57
curl_total_s	0.49	0.52	0.51	0.56	0.57

Results: 5 items returned

fetch-large

Config: 10 URLs, parallel_fetches=10

Metric	Min	Mean	Median	P95	Max
total_ms	3342	3521	3552	3658	3670
curl_ttfb_s	3.36	3.54	3.57	3.68	3.69
curl_total_s	3.36	3.54	3.57	3.68	3.69

Results: 9 items (1 failed — reuters.com timeout)

fetch-serial

Config: 5 URLs, parallel_fetches=1

Metric	Min	Mean	Median	P95	Max
total_ms	791	895	894	990	1001
curl_ttfb_s	0.80	0.90	0.90	1.00	1.01
curl_total_s	0.80	0.90	0.90	1.00	1.01

Results: 5 items returned

fetch-no-text

Config: 3 URLs, extract_text=false, include_html=true

Metric	Min	Mean	Median	P95	Max
total_ms	492	501	502	508	509
curl_ttfb_s	0.52	0.53	0.53	0.53	0.53
curl_total_s	0.52	0.53	0.53	0.54	0.54

Results: 3 items returned

fetch-no-fallback

Config: 3 URLs, auto_fallback=false, method=http

Metric	Min	Mean	Median	P95	Max
total_ms	497	501	501	505	505
curl_ttfb_s	0.51	0.51	0.51	0.51	0.51
curl_total_s	0.51	0.51	0.51	0.51	0.52

Results: 3 items returned

fetch-short-timeout

Config: 3 URLs, timeout_seconds=5

Metric	Min	Mean	Median	P95	Max
total_ms	485	498	496	511	512
curl_ttfb_s	0.49	0.51	0.50	0.52	0.52
curl_total_s	0.50	0.51	0.51	0.52	0.52

Results: 3 items returned

---

Image Search (/v1/image-search)

image-search-small

Config: 1 query, max_results=5

Metric	Min	Mean	Median	P95	Max
total_ms	1365	1469	1487	1549	1555
curl_ttfb_s	1.37	1.47	1.49	1.55	1.56
curl_total_s	1.37	1.47	1.49	1.55	1.56

Results: 5 items returned

image-search-large

Config: 2 queries, max_results=100

Status: 3/3 FAILED

Error: internal_error: An unexpected error occurred

Note: Large image search result counts (100+) trigger internal errors. Needs investigation.

---

Gather (/v1/gather)

gather-minimal

Config: max_search_results=5, max_evidence_items=3

Metric	Min	Mean	Median	P95	Max
total_ms	2499	2843	2641	3315	3390
search_ms	855	1172	1008	1588	1653
fetch_ms	1537	1575	1553	1626	1635
evidence_ms	84	84	84	85	85

Results: 3 evidence items

gather-default

Config: max_search_results=10, max_evidence_items=8

Metric	Min	Mean	Median	P95	Max
total_ms	4604	6402	5055	9099	9548
search_ms	901	1493	1120	2324	2458
fetch_ms	3183	4516	3762	6319	6603
evidence_ms	206	208	208	211	211

Results: 8 evidence items

gather-snippets

Config: max_search_results=5, max_evidence_items=5, extract_snippets=true

Metric	Min	Mean	Median	P95	Max
total_ms	8000	8074	8087	8130	8134
search_ms	595	908	1024	1098	1106
fetch_ms	1742	1764	1773	1777	1777
evidence_ms	5107	5370	5355	5619	5648

Results: 5 evidence items

Note: LLM snippet extraction adds ~5s to evidence processing.

gather-high-parallel

Config: max_search_results=20, max_evidence_items=10, parallel_fetches=10

Metric	Min	Mean	Median	P95	Max
total_ms	3043	3704	3663	4333	4407
search_ms	797	1227	1389	1485	1496
fetch_ms	2223	2357	2260	2555	2588

Results: 0 evidence items (no matching content)

gather-serial-fetch

Config: max_search_results=10, max_evidence_items=5, parallel_fetches=1

Metric	Min	Mean	Median	P95	Max
total_ms	6100	9994	10926	12754	12957
search_ms	775	1478	982	2509	2678
fetch_ms	4209	5771	5171	7656	7932

Results: 0 evidence items

Note: Serial fetching is 2-3x slower than parallel.

gather-no-fallback

Config: max_search_results=10, max_evidence_items=5, fetch_method=http, auto_fallback=false

Metric	Min	Mean	Median	P95	Max
total_ms	2475	3091	3339	3446	3458
search_ms	460	1055	895	1718	1809
fetch_ms	1061	1467	1593	1732	1747
evidence_ms	30	79	37	158	171

Results: 5 evidence items

gather-site-restricted

Config: max_search_results=15, max_evidence_items=8, site_allowlist=[wikipedia.org, bbc.com, reuters.com]

Metric	Min	Mean	Median	P95	Max
total_ms	5941	6925	6357	8266	8478
search_ms	3411	4141	3767	5098	5246
fetch_ms	2338	2530	2372	2828	2879
evidence_ms	184	206	209	223	224

Results: 8 evidence items

Note: Site filtering increases search latency (SearXNG site: prefix).

gather-max-evidence

Config: max_search_results=30, max_evidence_items=25

Metric	Min	Mean	Median	P95	Max
total_ms	5780	6238	6123	6743	6812
search_ms	875	1005	921	1191	1221
fetch_ms	4796	5199	4885	5812	5916

Results: 0 evidence items

gather-large

Config: max_search_results=20, max_evidence_items=15

Metric	Min	Mean	Median	P95	Max
total_ms	6053	8215	8421	9996	10171
search_ms	792	987	1047	1115	1122
fetch_ms	4603	6648	6789	8376	8552
evidence_ms	134	167	148	213	220

Results: 15 evidence items

---

Key Findings

Performance Rankings

Fastest scenarios:

1. fetch-single — 142ms
2. fetch-short-timeout — 498ms
3. fetch-no-text — 501ms
4. fetch-no-fallback — 501ms
5. fetch-multi — 512ms

Slowest scenarios:

1. gather-serial-fetch — 9994ms
2. gather-large — 8215ms
3. gather-snippets — 8074ms
4. gather-site-restricted — 6925ms
5. gather-default — 6402ms

Issues Detected

• image-search-large: Internal error when max_results=100 — needs investigation

Observations

Finding	Impact
Search scaling	~300ms per additional query (linear)
Fetch parallelism	parallel_fetches=5-10 is 2x faster than serial
LLM snippets	extract_snippets=true adds ~5s to evidence stage
Site filtering	site_allowlist adds 2-3s to search (SearXNG overhead)
Image search limits	max_results > ~50 causes internal errors

Recommendations

1. Use parallel fetches — Default of 5 is good, 10 for large workloads
2. Avoid snippets for speed — Only enable when LLM extraction is needed
3. Site filters are expensive — Use sparingly, prefer post-fetch filtering
4. Image search cap — Keep max_results ≤ 50 until bug is fixed