clossers/didi-lot1-ai
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Livrare LOT 1 - Didi

Browse source
This commit is contained in:
Dezvoltari Evotech 2026-06-25 14:13:25 -07:00
commit 5380c3fc63
990 changed files with 133308 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

2
artefacte_lot1/output_teste/cloak/pytest_output.txt Normal file
View file

@ -0,0 +1,2 @@
.... [100%]
4 passed in 0.06s

1
artefacte_lot1/output_teste/cloak/test-results.xml Normal file
View file

@ -0,0 +1 @@
<?xml version="1.0" encoding="utf-8"?><testsuites name="pytest tests"><testsuite name="pytest" errors="0" failures="0" skipped="0" tests="4" time="0.066" timestamp="2026-06-25T15:04:48.254758+00:00" hostname="ebd2d6bfd7a9"><testcase classname="tests.test_schemas" name="test_search_request_minimal" time="0.001" /><testcase classname="tests.test_schemas" name="test_search_request_rejects_unknown_engine" time="0.000" /><testcase classname="tests.test_schemas" name="test_search_request_rejects_empty_queries" time="0.000" /><testcase classname="tests.test_schemas" name="test_search_response_round_trip" time="0.001" /></testsuite></testsuites>

2
artefacte_lot1/output_teste/dashboard/pytest_output.txt Normal file
View file

@ -0,0 +1,2 @@
............ [100%]
12 passed in 2.79s

1
artefacte_lot1/output_teste/dashboard/test-results.xml Normal file
View file

@ -0,0 +1 @@
<?xml version="1.0" encoding="utf-8"?><testsuites name="pytest tests"><testsuite name="pytest" errors="0" failures="0" skipped="0" tests="12" time="2.791" timestamp="2026-06-25T15:19:42.196450+00:00" hostname="f17c6dfbf315"><testcase classname="tests.test_catalog" name="test_crud_lifecycle" time="0.613" /><testcase classname="tests.test_catalog" name="test_duplicate_rejected" time="0.561" /><testcase classname="tests.test_catalog" name="test_invalid_kind_rejected" time="0.573" /><testcase classname="tests.test_catalog" name="test_update_missing_404" time="0.525" /><testcase classname="tests.test_monitoring" name="test_services_summary" time="0.012" /><testcase classname="tests.test_monitoring" name="test_queues_disabled_when_unset" time="0.002" /><testcase classname="tests.test_monitoring" name="test_queues_parsed_and_sorted" time="0.010" /><testcase classname="tests.test_monitoring" name="test_queues_failopen_on_error" time="0.008" /><testcase classname="tests.test_monitoring" name="test_latency_disabled_when_unset" time="0.002" /><testcase classname="tests.test_monitoring" name="test_latency_parsed_per_job" time="0.011" /><testcase classname="tests.test_rbac" name="test_human_read_routers_are_protected" time="0.000" /><testcase classname="tests.test_rbac" name="test_service_routers_stay_open" time="0.000" /></testsuite></testsuites>

21
artefacte_lot1/output_teste/embeddings/pytest_output.txt Normal file
View file

@ -0,0 +1,21 @@
============================= test session starts ==============================
platform linux -- Python 3.11.15, pytest-9.1.1, pluggy-1.6.0
rootdir: /app
configfile: pyproject.toml
testpaths: tests
plugins: anyio-4.14.1, respx-0.23.1, asyncio-1.4.0, cov-7.1.0
asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collected 36 items
tests/test_config.py ............ [ 33%]
tests/test_schemas.py ............. [ 69%]
tests/test_types.py ........... [100%]
=============================== warnings summary ===============================
../usr/local/lib/python3.11/site-packages/fastapi/testclient.py:1
/usr/local/lib/python3.11/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
------------------ generated xml file: /app/test-results.xml -------------------
======================== 36 passed, 1 warning in 0.04s =========================

1
artefacte_lot1/output_teste/embeddings/test-results.xml Normal file
View file

@ -0,0 +1 @@
<?xml version="1.0" encoding="utf-8"?><testsuites name="pytest tests"><testsuite name="pytest" errors="0" failures="0" skipped="0" tests="36" time="0.038" timestamp="2026-06-25T15:03:53.358675+00:00" hostname="70a963c0ed5d"><testcase classname="tests.test_config.TestEmbeddingSettings" name="test_required_fields" time="0.001" /><testcase classname="tests.test_config.TestEmbeddingSettings" name="test_with_required_fields" time="0.001" /><testcase classname="tests.test_config.TestEmbeddingSettings" name="test_optional_defaults" time="0.001" /><testcase classname="tests.test_config.TestEmbeddingSettings" name="test_env_override" time="0.001" /><testcase classname="tests.test_config.TestEmbeddingSettings" name="test_vllm_url_default" time="0.001" /><testcase classname="tests.test_config.TestEmbeddingSettings" name="test_llamacpp_url_default" time="0.001" /><testcase classname="tests.test_config.TestEmbeddingSettings" name="test_api_tokens_parsing" time="0.001" /><testcase classname="tests.test_config.TestEmbeddingSettings" name="test_auth_disabled_by_default" time="0.001" /><testcase classname="tests.test_config.TestSettingsCache" name="test_set_and_get" time="0.001" /><testcase classname="tests.test_config.TestSettingsCache" name="test_get_without_set_creates_instance" time="0.001" /><testcase classname="tests.test_config.TestSettingsCache" name="test_cached_returns_same_instance" time="0.001" /><testcase classname="tests.test_config.TestSettingsCache" name="test_clear_removes_cached_instance" time="0.001" /><testcase classname="tests.test_schemas.TestEmbeddingRequest" name="test_single_string_input" time="0.000" /><testcase classname="tests.test_schemas.TestEmbeddingRequest" name="test_list_input" time="0.000" /><testcase classname="tests.test_schemas.TestEmbeddingRequest" name="test_empty_input_rejected" time="0.000" /><testcase classname="tests.test_schemas.TestEmbeddingRequest" name="test_empty_string_in_list_rejected" time="0.000" /><testcase classname="tests.test_schemas.TestEmbeddingRequest" name="test_default_encoding_format" time="0.000" /><testcase classname="tests.test_schemas.TestEmbeddingRequest" name="test_base64_encoding_format" time="0.000" /><testcase classname="tests.test_schemas.TestEmbeddingRequest" name="test_backend_override" time="0.000" /><testcase classname="tests.test_schemas.TestEmbeddingRequest" name="test_dimensions_parameter" time="0.000" /><testcase classname="tests.test_schemas.TestEmbeddingRequest" name="test_invalid_dimensions" time="0.000" /><testcase classname="tests.test_schemas.TestEmbeddingResponse" name="test_creation" time="0.000" /><testcase classname="tests.test_schemas.TestEmbeddingResponse" name="test_multiple_embeddings" time="0.000" /><testcase classname="tests.test_schemas.TestEncodeEmbeddingBase64" name="test_encode_simple" time="0.000" /><testcase classname="tests.test_schemas.TestEncodeEmbeddingBase64" name="test_encode_decode_roundtrip" time="0.000" /><testcase classname="tests.test_types.TestBackendType" name="test_vllm_value" time="0.000" /><testcase classname="tests.test_types.TestBackendType" name="test_llamacpp_value" time="0.000" /><testcase classname="tests.test_types.TestBackendType" name="test_from_string" time="0.000" /><testcase classname="tests.test_types.TestBackendType" name="test_invalid_backend" time="0.001" /><testcase classname="tests.test_types.TestEmbeddingUsage" name="test_default_values" time="0.000" /><testcase classname="tests.test_types.TestEmbeddingUsage" name="test_with_values" time="0.000" /><testcase classname="tests.test_types.TestEmbeddingData" name="test_creation" time="0.000" /><testcase classname="tests.test_types.TestEmbeddingData" name="test_forbids_extra_fields" time="0.000" /><testcase classname="tests.test_types.TestModelInfo" name="test_required_fields" time="0.000" /><testcase classname="tests.test_types.TestModelInfo" name="test_optional_fields" time="0.000" /><testcase classname="tests.test_types.TestModelInfo" name="test_with_all_fields" time="0.000" /></testsuite></testsuites>

2
artefacte_lot1/output_teste/extractors/pytest_output.txt Normal file
View file

@ -0,0 +1,2 @@
....................................... [100%]
39 passed in 0.40s

1
artefacte_lot1/output_teste/extractors/test-results.xml Normal file
View file

@ -0,0 +1 @@
<?xml version="1.0" encoding="utf-8"?><testsuites name="pytest tests"><testsuite name="pytest" errors="0" failures="0" skipped="0" tests="39" time="0.399" timestamp="2026-06-25T15:05:11.587575+00:00" hostname="32b0a295c2c9"><testcase classname="tests.test_api" name="test_health" time="0.011" /><testcase classname="tests.test_api" name="test_ready_reports_tooling" time="0.006" /><testcase classname="tests.test_api" name="test_metadata_image_runs_image_extractors" time="0.020" /><testcase classname="tests.test_api" name="test_metadata_rejects_empty" time="0.004" /><testcase classname="tests.test_detect" name="test_summarize_counts" time="0.000" /><testcase classname="tests.test_detect" name="test_analyze_parses_and_sorts" time="0.000" /><testcase classname="tests.test_detect" name="test_analyze_empty_image" time="0.000" /><testcase classname="tests.test_detect" name="test_detect_endpoint_503_without_ml_extra" time="0.004" /><testcase classname="tests.test_detect" name="test_detect_endpoint_with_monkeypatched_model" time="0.004" /><testcase classname="tests.test_ela" name="test_ela_returns_stats" time="0.007" /><testcase classname="tests.test_ela" name="test_ela_on_bad_input" time="0.016" /><testcase classname="tests.test_exif" name="test_no_exif_is_reported" time="0.001" /><testcase classname="tests.test_exif" name="test_editing_software_flagged_as_anomaly" time="0.001" /><testcase classname="tests.test_exif" name="test_non_image_returns_error" time="0.001" /><testcase classname="tests.test_integrity" name="test_identifies_jpeg_and_hashes" time="0.001" /><testcase classname="tests.test_integrity" name="test_identifies_png" time="0.000" /><testcase classname="tests.test_integrity" name="test_detects_trailing_data_after_jpeg_eoi" time="0.001" /><testcase classname="tests.test_integrity" name="test_detects_mp4_brand" time="0.000" /><testcase classname="tests.test_llm_features" name="test_extract_json_plain" time="0.000" /><testcase classname="tests.test_llm_features" name="test_extract_json_fenced_with_prose" time="0.000" /><testcase classname="tests.test_llm_features" name="test_extract_json_missing_raises" time="0.000" /><testcase classname="tests.test_llm_features" name="test_sentiment_parses_model_json" time="0.001" /><testcase classname="tests.test_llm_features" name="test_sentiment_clamps_and_defaults" time="0.001" /><testcase classname="tests.test_llm_features" name="test_sentiment_empty_text" time="0.001" /><testcase classname="tests.test_llm_features" name="test_ocr_parses_text" time="0.001" /><testcase classname="tests.test_llm_features" name="test_ocr_accepts_plain_text_fallback" time="0.001" /><testcase classname="tests.test_llm_features" name="test_sentiment_endpoint" time="0.005" /><testcase classname="tests.test_llm_features" name="test_ocr_endpoint" time="0.004" /><testcase classname="tests.test_llm_features" name="test_sentiment_endpoint_503_without_gateway" time="0.003" /><testcase classname="tests.test_ner" name="test_ner_parses_and_sorts" time="0.000" /><testcase classname="tests.test_ner" name="test_ner_empty_entities" time="0.000" /><testcase classname="tests.test_ner" name="test_ner_endpoint_503_without_ml_extra" time="0.004" /><testcase classname="tests.test_ner" name="test_ner_endpoint_with_monkeypatched_model" time="0.004" /><testcase classname="tests.test_spectrogram" name="test_compute_features_on_tone" time="0.003" /><testcase classname="tests.test_spectrogram" name="test_detects_energy_discontinuity" time="0.001" /><testcase classname="tests.test_spectrogram" name="test_empty_signal" time="0.000" /><testcase classname="tests.test_video_meta" name="test_parse_fps_fractional" time="0.000" /><testcase classname="tests.test_video_meta" name="test_parse_ffprobe_full" time="0.000" /><testcase classname="tests.test_video_meta" name="test_parse_ffprobe_audio_only" time="0.000" /></testsuite></testsuites>

30
artefacte_lot1/output_teste/llm-inference/pytest_output.txt Normal file
View file

@ -0,0 +1,30 @@
============================= test session starts ==============================
platform linux -- Python 3.11.15, pytest-9.1.1, pluggy-1.6.0
rootdir: /app
configfile: pyproject.toml
testpaths: tests
plugins: anyio-4.14.1, respx-0.23.1, asyncio-1.4.0, cov-7.1.0
asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collected 123 items
tests/test_api.py ............. [ 10%]
tests/test_auth.py .................... [ 26%]
tests/test_client.py ............. [ 37%]
tests/test_concurrency.py ........ [ 43%]
tests/test_config.py ............ [ 53%]
tests/test_e2e_real.py sssssssssssssssssssssss [ 72%]
tests/test_retry.py ......... [ 79%]
tests/test_text_completions.py ........ [ 86%]
tests/test_types.py ............ [ 95%]
tests/test_utils.py ..... [100%]
=============================== warnings summary ===============================
tests/test_auth.py: 6 warnings
tests/test_client.py: 1 warning
tests/test_config.py: 10 warnings
/usr/local/lib/python3.11/site-packages/pydantic/main.py:263: UserWarning: No API keys configured for litellm backend. Set at least one of: OPENROUTER_API_KEY, OPENAI_API_KEY, ANTHROPIC_API_KEY
validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
------------------ generated xml file: /app/test-results.xml -------------------
================= 100 passed, 23 skipped, 17 warnings in 4.96s =================

1
artefacte_lot1/output_teste/llm-inference/test-results.xml Normal file
View file

File diff suppressed because one or more lines are too long

20
artefacte_lot1/output_teste/rerank/pytest_output.txt Normal file
View file

@ -0,0 +1,20 @@
============================= test session starts ==============================
platform linux -- Python 3.11.15, pytest-9.1.1, pluggy-1.6.0
rootdir: /app
configfile: pyproject.toml
testpaths: tests
plugins: anyio-4.14.1, respx-0.23.1, asyncio-1.4.0, cov-7.1.0
asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collected 24 items
tests/test_config.py ............ [ 50%]
tests/test_schemas.py ............ [100%]
=============================== warnings summary ===============================
../usr/local/lib/python3.11/site-packages/fastapi/testclient.py:1
/usr/local/lib/python3.11/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
------------------ generated xml file: /app/test-results.xml -------------------
======================== 24 passed, 1 warning in 0.05s =========================

1
artefacte_lot1/output_teste/rerank/test-results.xml Normal file
View file

@ -0,0 +1 @@
<?xml version="1.0" encoding="utf-8"?><testsuites name="pytest tests"><testsuite name="pytest" errors="0" failures="0" skipped="0" tests="24" time="0.048" timestamp="2026-06-25T15:04:24.017081+00:00" hostname="1d416f4566a3"><testcase classname="tests.test_config.TestRerankSettings" name="test_required_fields" time="0.002" /><testcase classname="tests.test_config.TestRerankSettings" name="test_with_required_fields" time="0.001" /><testcase classname="tests.test_config.TestRerankSettings" name="test_optional_defaults" time="0.001" /><testcase classname="tests.test_config.TestRerankSettings" name="test_env_override" time="0.001" /><testcase classname="tests.test_config.TestRerankSettings" name="test_vllm_url_default" time="0.001" /><testcase classname="tests.test_config.TestRerankSettings" name="test_llamacpp_url_default" time="0.001" /><testcase classname="tests.test_config.TestRerankSettings" name="test_api_tokens_parsing" time="0.001" /><testcase classname="tests.test_config.TestRerankSettings" name="test_auth_disabled_by_default" time="0.001" /><testcase classname="tests.test_config.TestSettingsCache" name="test_set_and_get" time="0.001" /><testcase classname="tests.test_config.TestSettingsCache" name="test_get_without_set_creates_instance" time="0.001" /><testcase classname="tests.test_config.TestSettingsCache" name="test_cached_returns_same_instance" time="0.001" /><testcase classname="tests.test_config.TestSettingsCache" name="test_clear_removes_cached_instance" time="0.001" /><testcase classname="tests.test_schemas.TestRerankRequest" name="test_basic_request" time="0.000" /><testcase classname="tests.test_schemas.TestRerankRequest" name="test_empty_documents_rejected" time="0.000" /><testcase classname="tests.test_schemas.TestRerankRequest" name="test_empty_string_in_documents_rejected" time="0.000" /><testcase classname="tests.test_schemas.TestRerankRequest" name="test_empty_query_rejected" time="0.000" /><testcase classname="tests.test_schemas.TestRerankRequest" name="test_top_n_parameter" time="0.000" /><testcase classname="tests.test_schemas.TestRerankRequest" name="test_return_documents_parameter" time="0.000" /><testcase classname="tests.test_schemas.TestRerankRequest" name="test_backend_override" time="0.000" /><testcase classname="tests.test_schemas.TestRerankRequest" name="test_invalid_top_n" time="0.000" /><testcase classname="tests.test_schemas.TestRerankResponse" name="test_creation" time="0.000" /><testcase classname="tests.test_schemas.TestRerankResponse" name="test_results_ordering" time="0.000" /><testcase classname="tests.test_schemas.TestRerankResult" name="test_without_document" time="0.000" /><testcase classname="tests.test_schemas.TestRerankResult" name="test_with_document" time="0.000" /></testsuite></testsuites>

252
artefacte_lot1/output_teste/video-analysis/TESTING.md Normal file
View file

@ -0,0 +1,252 @@
# Video Analysis - Testing Checklist
## ⚠️ HIGH PRIORITY: Model Comparison for Semantic Analysis
### Background
Currently both deepfake detection and semantic analysis use **BusterX** (Qwen2.5-VL-7B, 7B parameters). However, we have access to a much larger model **Qwen3-VL-30B** (30B parameters) that could provide significantly better semantic understanding.
### Hypothesis
Semantic analysis (content understanding, scene description, narrative) would benefit from the larger Qwen3-VL-30B model, while deepfake detection should continue using the specialized BusterX model.
---
## Test Plan: Qwen3-VL-30B for Semantic Analysis
### Current Configuration
```bash
# modules/video-analysis/deploy/.env
VIDEO_ANALYSIS_VLLM_BASE_URL=http://didiAI-video-vllm-buster:54500 # Port 54500
VIDEO_ANALYSIS_VLLM_MODEL=busterx # 7B parameters
```
### Test Configuration
```bash
# modules/video-analysis/deploy/.env
VIDEO_ANALYSIS_VLLM_BASE_URL=http://didiAI-llm-vllm-vision:14002 # Port 14002
VIDEO_ANALYSIS_VLLM_MODEL=qwen3-vl # 30B parameters
```
---
## Testing Procedure
### Step 1: Baseline Test (BusterX 7B)
```bash
# Current configuration - no changes needed
cd /home/vasi/ml-projects/modules/video-analysis/deploy
# Test semantic analysis with BusterX
curl -X POST http://localhost:8007/analyze/video/semantic \
-F "file=@test_video_60s.mp4" \
-F "chunk_duration_s=10.0" \
-F "frames_per_chunk=24" \
-o baseline_busterx.json
# Review results
jq '{
model: "BusterX-7B",
num_chunks: .num_chunks,
chunk_descriptions: [.chunk_results[].description],
final_summary: .final_summary,
total_latency_s: .total_latency_s
}' baseline_busterx.json
```
### Step 2: Test with Qwen3-VL-30B
```bash
cd /home/vasi/ml-projects/modules/video-analysis/deploy
# Backup current config
cp .env .env.backup
# Update to use Qwen3-VL
cat > .env << 'ENVFILE'
HF_TOKEN=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
HF_CACHE_DIR=/cai2_ds_storage/hf_cache
VIDEO_ANALYSIS_RUNS_DIR=/app/runs
# Switch to Qwen3-VL for testing
VIDEO_ANALYSIS_VLLM_BASE_URL=http://didiAI-llm-vllm-vision:14002
VIDEO_ANALYSIS_VLLM_MODEL=qwen3-vl
# Semantic analysis settings
VIDEO_ANALYSIS_SEMANTIC_CHUNK_DURATION_S=10.0
VIDEO_ANALYSIS_SEMANTIC_FRAMES_PER_CHUNK=24
VIDEO_ANALYSIS_SEMANTIC_ENABLE_AGGREGATION=true
VIDEO_ANALYSIS_SEMANTIC_AGGREGATION_MODEL=gpt-oss-120b
VIDEO_ANALYSIS_SEMANTIC_LLM_BASE_URL=http://deploy-llm-api-1:8100
ENVFILE
# Rebuild and restart
docker compose build video-analysis-api
docker compose up -d video-analysis-api
# Wait for startup
sleep 10
# Test with same video
curl -X POST http://localhost:8007/analyze/video/semantic \
-F "file=@test_video_60s.mp4" \
-F "chunk_duration_s=10.0" \
-F "frames_per_chunk=24" \
-o test_qwen3vl.json
# Review results
jq '{
model: "Qwen3-VL-30B",
num_chunks: .num_chunks,
chunk_descriptions: [.chunk_results[].description],
final_summary: .final_summary,
total_latency_s: .total_latency_s
}' test_qwen3vl.json
```
### Step 3: Compare Results
```bash
# Side-by-side comparison
echo "=== BusterX 7B ==="
jq -r '.chunk_results[0].description' baseline_busterx.json
echo ""
echo "=== Qwen3-VL 30B ==="
jq -r '.chunk_results[0].description' test_qwen3vl.json
echo ""
echo "=== Final Summaries ==="
echo "BusterX: $(jq -r '.final_summary' baseline_busterx.json)"
echo ""
echo "Qwen3-VL: $(jq -r '.final_summary' test_qwen3vl.json)"
```
---
## Evaluation Criteria
### Quality Metrics
1. **Description Detail**
- [ ] More specific object identification
- [ ] Better action recognition
- [ ] More context understanding
2. **Narrative Coherence**
- [ ] Logical flow between segments
- [ ] Temporal consistency
- [ ] Better story understanding
3. **Accuracy**
- [ ] Correct identification of people/objects
- [ ] Accurate scene descriptions
- [ ] Proper action sequences
### Performance Metrics
| Metric | BusterX 7B | Qwen3-VL 30B | Difference |
|--------|-----------|--------------|------------|
| Latency per chunk | ~12s | ~??s | ?? |
| Total latency (60s video) | ~77s | ~??s | ?? |
| Token usage per chunk | ~5200 | ~????? | ?? |
---
## Expected Outcomes
### If Qwen3-VL is Better:
**Action:** Update default configuration to use Qwen3-VL for semantic analysis
```bash
# Keep two separate configs:
# 1. Deepfake endpoint → BusterX (specialized)
# 2. Semantic endpoint → Qwen3-VL (better understanding)
```
**Implementation:**
- Add `VIDEO_ANALYSIS_DEEPFAKE_MODEL` and `VIDEO_ANALYSIS_SEMANTIC_MODEL` settings
- Configure different models per endpoint
### If BusterX is Sufficient:
**Action:** Keep current configuration, document findings
**Rationale:**
- Latency advantage (30B model is slower)
- VRAM savings
- BusterX might be sufficient for semantic tasks
---
## Test Videos
Suggested test scenarios:
1. **Meeting/Conference** (60s)
- Multiple people
- Complex interactions
- Scene changes
2. **Action Sequence** (60s)
- Fast movements
- Object manipulation
- Environmental changes
3. **Indoor/Outdoor Transition** (60s)
- Lighting changes
- Multiple scenes
- Context shifts
---
## Rollback Procedure
If issues occur:
```bash
cd /home/vasi/ml-projects/modules/video-analysis/deploy
# Restore original config
cp .env.backup .env
# Rebuild
docker compose build video-analysis-api
docker compose up -d video-analysis-api
```
---
## Status
- [ ] Baseline test completed (BusterX)
- [ ] Test with Qwen3-VL completed
- [ ] Results compared
- [ ] Decision made
- [ ] Configuration updated (if needed)
- [ ] Documentation updated
---
## Notes
Add observations here during testing:
```
Date: ___________
Tester: _________
Observations:
Recommendation:
```

3
artefacte_lot1/output_teste/video-analysis/pytest_output.txt Normal file
View file

@ -0,0 +1,3 @@
....... [100%]
------------------ generated xml file: /app/test-results.xml -------------------
7 passed in 0.23s

1
artefacte_lot1/output_teste/video-analysis/test-results.xml Normal file
View file

@ -0,0 +1 @@
<?xml version="1.0" encoding="utf-8"?><testsuites name="pytest tests"><testsuite name="pytest" errors="0" failures="0" skipped="0" tests="7" time="0.235" timestamp="2026-06-25T15:16:23.358556+00:00" hostname="2b93193c8419"><testcase classname="tests.test_verdict" name="test_parse_real" time="0.000" /><testcase classname="tests.test_verdict" name="test_parse_fake" time="0.000" /><testcase classname="tests.test_verdict" name="test_parse_unclear_is_uncertain" time="0.000" /><testcase classname="tests.test_verdict" name="test_parse_empty_is_uncertain" time="0.000" /><testcase classname="tests.test_verdict" name="test_build_frame_evidence_aligns_indices_and_timestamps" time="0.000" /><testcase classname="tests.test_verdict" name="test_build_frame_evidence_handles_missing_timestamps" time="0.000" /><testcase classname="tests.test_verdict" name="test_build_frame_evidence_empty" time="0.000" /></testsuite></testsuites>

405
artefacte_lot1/output_teste/web/RESULTS.md Normal file
View file

@ -0,0 +1,405 @@
# Web API Benchmark Results
**Date:** 2026-02-08
**Base URL:** http://localhost:51100
**Iterations:** 3 (+ 1 warmup)
## Summary
| Endpoint | Scenarios | Avg Latency | Status |
|----------|-----------|-------------|--------|
| Health | 1 | ~10ms | OK |
| Search (`/v1/search`) | 8 | 1181ms | OK |
| Fetch (`/v1/fetch`) | 7 | 939ms | OK |
| Image Search (`/v1/image-search`) | 2 | 1469ms | 3 errors |
| Gather (`/v1/gather`) | 9 | 6165ms | OK |
---
## Health
### `health`
| Metric | Min | Mean | Median | P95 | Max |
|--------|-----|------|--------|-----|-----|
| curl_ttfb_s | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 |
| curl_total_s | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 |
---
## Search (`/v1/search`)
### `search-single`
**Config:** 1 query, max_results=10
| Metric | Min | Mean | Median | P95 | Max |
|--------|-----|------|--------|-----|-----|
| total_ms | 574 | 661 | 650 | 749 | 760 |
| curl_ttfb_s | 0.58 | 0.67 | 0.66 | 0.75 | 0.76 |
| curl_total_s | 0.58 | 0.67 | 0.66 | 0.75 | 0.76 |
**Results:** 10 items returned
### `search-multi-query`
**Config:** 5 queries, max_results=10
| Metric | Min | Mean | Median | P95 | Max |
|--------|-----|------|--------|-----|-----|
| total_ms | 1775 | 1936 | 1972 | 2053 | 2062 |
| curl_ttfb_s | 1.78 | 1.94 | 1.98 | 2.06 | 2.07 |
| curl_total_s | 1.78 | 1.94 | 1.98 | 2.06 | 2.07 |
**Results:** 50 items returned
### `search-max-queries`
**Config:** 10 queries, max_results=5
| Metric | Min | Mean | Median | P95 | Max |
|--------|-----|------|--------|-----|-----|
| total_ms | 2776 | 2945 | 3012 | 3045 | 3049 |
| curl_ttfb_s | 2.78 | 2.95 | 3.02 | 3.05 | 3.05 |
| curl_total_s | 2.78 | 2.95 | 3.02 | 3.05 | 3.05 |
**Results:** 50 items returned
### `search-large-results`
**Config:** 2 queries, max_results=50
| Metric | Min | Mean | Median | P95 | Max |
|--------|-----|------|--------|-----|-----|
| total_ms | 930 | 1087 | 1105 | 1213 | 1225 |
| curl_ttfb_s | 0.94 | 1.09 | 1.11 | 1.22 | 1.23 |
| curl_total_s | 0.94 | 1.09 | 1.11 | 1.22 | 1.23 |
**Results:** 37 items returned
### `search-freshness-day`
**Config:** 1 query, max_results=10, freshness=day
| Metric | Min | Mean | Median | P95 | Max |
|--------|-----|------|--------|-----|-----|
| total_ms | 662 | 690 | 704 | 705 | 706 |
| curl_ttfb_s | 0.67 | 0.70 | 0.71 | 0.71 | 0.71 |
| curl_total_s | 0.67 | 0.70 | 0.71 | 0.71 | 0.71 |
**Results:** 10 items returned
### `search-freshness-week`
**Config:** 1 query, max_results=10, freshness=week
| Metric | Min | Mean | Median | P95 | Max |
|--------|-----|------|--------|-----|-----|
| total_ms | 573 | 623 | 614 | 675 | 681 |
| curl_ttfb_s | 0.58 | 0.63 | 0.62 | 0.68 | 0.69 |
| curl_total_s | 0.58 | 0.63 | 0.62 | 0.68 | 0.69 |
**Results:** 10 items returned
### `search-site-filter`
**Config:** 1 query, max_results=20, site_allowlist=[wikipedia.org, python.org, realpython.com]
| Metric | Min | Mean | Median | P95 | Max |
|--------|-----|------|--------|-----|-----|
| total_ms | 586 | 701 | 612 | 876 | 905 |
| curl_ttfb_s | 0.59 | 0.71 | 0.62 | 0.88 | 0.91 |
| curl_total_s | 0.59 | 0.71 | 0.62 | 0.88 | 0.91 |
**Results:** 9 items returned
### `search-non-english`
**Config:** 1 query (French), max_results=10, language=fr, country=FR
| Metric | Min | Mean | Median | P95 | Max |
|--------|-----|------|--------|-----|-----|
| total_ms | 570 | 805 | 779 | 1036 | 1065 |
| curl_ttfb_s | 0.58 | 0.81 | 0.78 | 1.04 | 1.07 |
| curl_total_s | 0.58 | 0.81 | 0.79 | 1.04 | 1.07 |
**Results:** 10 items returned
---
## Fetch (`/v1/fetch`)
### `fetch-single`
**Config:** 1 URL
| Metric | Min | Mean | Median | P95 | Max |
|--------|-----|------|--------|-----|-----|
| total_ms | 138 | 142 | 144 | 145 | 146 |
| curl_ttfb_s | 0.14 | 0.15 | 0.15 | 0.15 | 0.15 |
| curl_total_s | 0.14 | 0.15 | 0.15 | 0.15 | 0.15 |
**Results:** 1 item returned
### `fetch-multi`
**Config:** 5 URLs, parallel_fetches=5
| Metric | Min | Mean | Median | P95 | Max |
|--------|-----|------|--------|-----|-----|
| total_ms | 481 | 512 | 496 | 552 | 558 |
| curl_ttfb_s | 0.49 | 0.52 | 0.50 | 0.56 | 0.57 |
| curl_total_s | 0.49 | 0.52 | 0.51 | 0.56 | 0.57 |
**Results:** 5 items returned
### `fetch-large`
**Config:** 10 URLs, parallel_fetches=10
| Metric | Min | Mean | Median | P95 | Max |
|--------|-----|------|--------|-----|-----|
| total_ms | 3342 | 3521 | 3552 | 3658 | 3670 |
| curl_ttfb_s | 3.36 | 3.54 | 3.57 | 3.68 | 3.69 |
| curl_total_s | 3.36 | 3.54 | 3.57 | 3.68 | 3.69 |
**Results:** 9 items (1 failed — reuters.com timeout)
### `fetch-serial`
**Config:** 5 URLs, parallel_fetches=1
| Metric | Min | Mean | Median | P95 | Max |
|--------|-----|------|--------|-----|-----|
| total_ms | 791 | 895 | 894 | 990 | 1001 |
| curl_ttfb_s | 0.80 | 0.90 | 0.90 | 1.00 | 1.01 |
| curl_total_s | 0.80 | 0.90 | 0.90 | 1.00 | 1.01 |
**Results:** 5 items returned
### `fetch-no-text`
**Config:** 3 URLs, extract_text=false, include_html=true
| Metric | Min | Mean | Median | P95 | Max |
|--------|-----|------|--------|-----|-----|
| total_ms | 492 | 501 | 502 | 508 | 509 |
| curl_ttfb_s | 0.52 | 0.53 | 0.53 | 0.53 | 0.53 |
| curl_total_s | 0.52 | 0.53 | 0.53 | 0.54 | 0.54 |
**Results:** 3 items returned
### `fetch-no-fallback`
**Config:** 3 URLs, auto_fallback=false, method=http
| Metric | Min | Mean | Median | P95 | Max |
|--------|-----|------|--------|-----|-----|
| total_ms | 497 | 501 | 501 | 505 | 505 |
| curl_ttfb_s | 0.51 | 0.51 | 0.51 | 0.51 | 0.51 |
| curl_total_s | 0.51 | 0.51 | 0.51 | 0.51 | 0.52 |
**Results:** 3 items returned
### `fetch-short-timeout`
**Config:** 3 URLs, timeout_seconds=5
| Metric | Min | Mean | Median | P95 | Max |
|--------|-----|------|--------|-----|-----|
| total_ms | 485 | 498 | 496 | 511 | 512 |
| curl_ttfb_s | 0.49 | 0.51 | 0.50 | 0.52 | 0.52 |
| curl_total_s | 0.50 | 0.51 | 0.51 | 0.52 | 0.52 |
**Results:** 3 items returned
---
## Image Search (`/v1/image-search`)
### `image-search-small`
**Config:** 1 query, max_results=5
| Metric | Min | Mean | Median | P95 | Max |
|--------|-----|------|--------|-----|-----|
| total_ms | 1365 | 1469 | 1487 | 1549 | 1555 |
| curl_ttfb_s | 1.37 | 1.47 | 1.49 | 1.55 | 1.56 |
| curl_total_s | 1.37 | 1.47 | 1.49 | 1.55 | 1.56 |
**Results:** 5 items returned
### `image-search-large`
**Config:** 2 queries, max_results=100
**Status:** 3/3 FAILED
**Error:** `internal_error: An unexpected error occurred`
> **Note:** Large image search result counts (100+) trigger internal errors. Needs investigation.
---
## Gather (`/v1/gather`)
### `gather-minimal`
**Config:** max_search_results=5, max_evidence_items=3
| Metric | Min | Mean | Median | P95 | Max |
|--------|-----|------|--------|-----|-----|
| total_ms | 2499 | 2843 | 2641 | 3315 | 3390 |
| search_ms | 855 | 1172 | 1008 | 1588 | 1653 |
| fetch_ms | 1537 | 1575 | 1553 | 1626 | 1635 |
| evidence_ms | 84 | 84 | 84 | 85 | 85 |
**Results:** 3 evidence items
### `gather-default`
**Config:** max_search_results=10, max_evidence_items=8
| Metric | Min | Mean | Median | P95 | Max |
|--------|-----|------|--------|-----|-----|
| total_ms | 4604 | 6402 | 5055 | 9099 | 9548 |
| search_ms | 901 | 1493 | 1120 | 2324 | 2458 |
| fetch_ms | 3183 | 4516 | 3762 | 6319 | 6603 |
| evidence_ms | 206 | 208 | 208 | 211 | 211 |
**Results:** 8 evidence items
### `gather-snippets`
**Config:** max_search_results=5, max_evidence_items=5, extract_snippets=true
| Metric | Min | Mean | Median | P95 | Max |
|--------|-----|------|--------|-----|-----|
| total_ms | 8000 | 8074 | 8087 | 8130 | 8134 |
| search_ms | 595 | 908 | 1024 | 1098 | 1106 |
| fetch_ms | 1742 | 1764 | 1773 | 1777 | 1777 |
| evidence_ms | 5107 | 5370 | 5355 | 5619 | 5648 |
**Results:** 5 evidence items
> **Note:** LLM snippet extraction adds ~5s to evidence processing.
### `gather-high-parallel`
**Config:** max_search_results=20, max_evidence_items=10, parallel_fetches=10
| Metric | Min | Mean | Median | P95 | Max |
|--------|-----|------|--------|-----|-----|
| total_ms | 3043 | 3704 | 3663 | 4333 | 4407 |
| search_ms | 797 | 1227 | 1389 | 1485 | 1496 |
| fetch_ms | 2223 | 2357 | 2260 | 2555 | 2588 |
**Results:** 0 evidence items (no matching content)
### `gather-serial-fetch`
**Config:** max_search_results=10, max_evidence_items=5, parallel_fetches=1
| Metric | Min | Mean | Median | P95 | Max |
|--------|-----|------|--------|-----|-----|
| total_ms | 6100 | 9994 | 10926 | 12754 | 12957 |
| search_ms | 775 | 1478 | 982 | 2509 | 2678 |
| fetch_ms | 4209 | 5771 | 5171 | 7656 | 7932 |
**Results:** 0 evidence items
> **Note:** Serial fetching is 2-3x slower than parallel.
### `gather-no-fallback`
**Config:** max_search_results=10, max_evidence_items=5, fetch_method=http, auto_fallback=false
| Metric | Min | Mean | Median | P95 | Max |
|--------|-----|------|--------|-----|-----|
| total_ms | 2475 | 3091 | 3339 | 3446 | 3458 |
| search_ms | 460 | 1055 | 895 | 1718 | 1809 |
| fetch_ms | 1061 | 1467 | 1593 | 1732 | 1747 |
| evidence_ms | 30 | 79 | 37 | 158 | 171 |
**Results:** 5 evidence items
### `gather-site-restricted`
**Config:** max_search_results=15, max_evidence_items=8, site_allowlist=[wikipedia.org, bbc.com, reuters.com]
| Metric | Min | Mean | Median | P95 | Max |
|--------|-----|------|--------|-----|-----|
| total_ms | 5941 | 6925 | 6357 | 8266 | 8478 |
| search_ms | 3411 | 4141 | 3767 | 5098 | 5246 |
| fetch_ms | 2338 | 2530 | 2372 | 2828 | 2879 |
| evidence_ms | 184 | 206 | 209 | 223 | 224 |
**Results:** 8 evidence items
> **Note:** Site filtering increases search latency (SearXNG site: prefix).
### `gather-max-evidence`
**Config:** max_search_results=30, max_evidence_items=25
| Metric | Min | Mean | Median | P95 | Max |
|--------|-----|------|--------|-----|-----|
| total_ms | 5780 | 6238 | 6123 | 6743 | 6812 |
| search_ms | 875 | 1005 | 921 | 1191 | 1221 |
| fetch_ms | 4796 | 5199 | 4885 | 5812 | 5916 |
**Results:** 0 evidence items
### `gather-large`
**Config:** max_search_results=20, max_evidence_items=15
| Metric | Min | Mean | Median | P95 | Max |
|--------|-----|------|--------|-----|-----|
| total_ms | 6053 | 8215 | 8421 | 9996 | 10171 |
| search_ms | 792 | 987 | 1047 | 1115 | 1122 |
| fetch_ms | 4603 | 6648 | 6789 | 8376 | 8552 |
| evidence_ms | 134 | 167 | 148 | 213 | 220 |
**Results:** 15 evidence items
---
## Key Findings
### Performance Rankings
**Fastest scenarios:**
1. `fetch-single` — 142ms
2. `fetch-short-timeout` — 498ms
3. `fetch-no-text` — 501ms
4. `fetch-no-fallback` — 501ms
5. `fetch-multi` — 512ms
**Slowest scenarios:**
1. `gather-serial-fetch` — 9994ms
2. `gather-large` — 8215ms
3. `gather-snippets` — 8074ms
4. `gather-site-restricted` — 6925ms
5. `gather-default` — 6402ms
### Issues Detected
- `image-search-large`: Internal error when max_results=100 — needs investigation
### Observations
| Finding | Impact |
|---------|--------|
| **Search scaling** | ~300ms per additional query (linear) |
| **Fetch parallelism** | parallel_fetches=5-10 is 2x faster than serial |
| **LLM snippets** | extract_snippets=true adds ~5s to evidence stage |
| **Site filtering** | site_allowlist adds 2-3s to search (SearXNG overhead) |
| **Image search limits** | max_results > ~50 causes internal errors |
### Recommendations
1. **Use parallel fetches** — Default of 5 is good, 10 for large workloads
2. **Avoid snippets for speed** — Only enable when LLM extraction is needed
3. **Site filters are expensive** — Use sparingly, prefer post-fetch filtering
4. **Image search cap** — Keep max_results ≤ 50 until bug is fixed

63
artefacte_lot1/output_teste/web/gather_results.md Normal file
View file

@ -0,0 +1,63 @@
# Web API Gather Results - NER & Fact-Checking Research
Generated: 2026-01-30
---
## Query 1: MultiNERD Entity Types
**Claim:** `MultiNERD entity types list NER categories person organization location date time money percentage fact-checking`
**Execution Time:** 5203ms
### URLs Accessed
- [Named-entity recognition - Wikipedia](https://en.wikipedia.org/wiki/Named-entity_recognition)
- [Gold standard, multi-genre dataset for named entity recognition and linking | Scientific Data](https://www.nature.com/articles/s41597-025-05274-4)
- [Named Entity Recognition (NER): Ultimate Guide | Encord](https://encord.com/blog/named-entity-recognition/)
- [A Comprehensive Guide to Named Entity Recognition](https://www.turing.com/kb/a-comprehensive-guide-to-named-entity-recognition)
- [Entity categories recognized by Named Entity Recognition in Azure](https://learn.microsoft.com/en-us/azure/ai-services/language-service/named-entity-recognition/concepts/named-entity-categories)
---
## Query 2: Fact-Checking Methodology
**Claim:** `fact-checking entity extraction named entity recognition claim verification ClaimBuster Full Fact methodology 2024`
**Execution Time:** 9698ms
### URLs Accessed
- [arxiv.org PDF 1809.08193](https://arxiv.org/pdf/1809.08193)
- [Claim Extraction for Fact-Checking: Data, Models, and Automated Metrics](https://arxiv.org/html/2502.04955v1)
- [Facilitating automated fact-checking: a machine learning based weighted ensemble technique for claim detection](https://link.springer.com/article/10.1007/s42452-024-06444-6)
- [Document-level Claim Extraction and Decontextualisation for Fact-Checking](https://arxiv.org/html/2406.03239v2)
- [An Entity-based Claim Extraction Pipeline for Real-world Biomedical Fact-checking](https://arxiv.org/abs/2304.05268)
---
## Query 3: NER Query Generation for Fact-Checking
**Claim:** `named entity recognition query generation fact-checking entity types person organization location date event claim verification search query templates`
**Execution Time:** 3485ms
### URLs Accessed
- [Tools for Named Entity Recognition | CLARIN ERIC](https://www.clarin.eu/resource-families/tools-named-entity-recognition)
- [Named-entity recognition - Wikipedia](https://en.wikipedia.org/wiki/Named-entity_recognition)
- [Named Entity Recognition - GeeksforGeeks](https://www.geeksforgeeks.org/nlp/named-entity-recognition/)
- [Named Entity Recognition | Yext](https://www.yext.com/platform/features/named-entity-recognition)
- [Named Entity Recognition on Search Engine Queries with Python - Stack Overflow](https://stackoverflow.com/questions/78151241/named-entity-recognition-on-search-engine-queries-with-python)
---
## Summary
| Query | Evidence URLs | Execution Time |
|-------|---------------|----------------|
| 1 - MultiNERD Entity Types | 5 | 5.2s |
| 2 - Fact-Checking Methodology | 5 | 9.7s |
| 3 - NER Query Generation | 5 | 3.5s |
**Total URLs:** 15

63
artefacte_lot1/output_teste/web/pytest_output.txt Normal file
View file

@ -0,0 +1,63 @@
============================= test session starts ==============================
platform linux -- Python 3.11.15, pytest-9.1.1, pluggy-1.6.0
rootdir: /app
configfile: pyproject.toml
testpaths: tests
plugins: anyio-4.14.1, respx-0.23.1, asyncio-1.4.0, cov-7.1.0
asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=function, asyncio_default_test_loop_scope=function
collected 152 items
tests/test_api.py ..................... [ 13%]
tests/test_browse_client.py .F.... [ 17%]
tests/test_config.py ........ [ 23%]
tests/test_evidence_packer.py ...................... [ 37%]
tests/test_fetch_client.py ............. [ 46%]
tests/test_image_search_client.py ................ [ 56%]
tests/test_llm_provider.py ....... [ 61%]
tests/test_middleware.py ... [ 63%]
tests/test_orchestrator.py ......... [ 69%]
tests/test_schemas_common.py .......... [ 75%]
tests/test_search_client.py ............ [ 83%]
tests/test_validation.py .................FF [ 96%]
tests/test_vision_client.py ...... [100%]
=================================== FAILURES ===================================
________________ TestBrowseClientBrowse.test_browse_single_page ________________
tests/test_browse_client.py:100: in test_browse_single_page
assert "Main content" in response.pages[0].text
E AssertionError: assert 'Main content' in 'Content'
E + where 'Content' = BrowsePageResult(url='https://example.com/page', final_url='https://example.com/page', canonical_url=None, title='Test....040318+00:00', extraction_time_ms=5.37, warnings=[], screenshot_base64=None, viewport_width=1280, viewport_height=720).text
______________ TestValidateUrlsAsync.test_validates_multiple_urls ______________
tests/test_validation.py:137: in test_validates_multiple_urls
assert result == urls
E AssertionError: assert (['https://ex...e.com/c'], []) == ['https://exa...xample.com/c']
E
E At index 0 diff: ['https://example.com/a', 'https://example.com/b', 'https://example.com/c'] != 'https://example.com/a'
E Right contains one more item: 'https://example.com/c'
E Use -v to get more diff
___________ TestValidateUrlsAsync.test_raises_on_private_ip_in_batch ___________
tests/test_validation.py:145: in test_raises_on_private_ip_in_batch
with pytest.raises(ValueError, match="private/reserved"):
E Failed: DID NOT RAISE ValueError
------------------------------ Captured log call -------------------------------
WARNING web.validation:validation.py:150 Dropping URL https://evil.example.com/steal: URL hostname 'evil.example.com' resolves to private/reserved address 10.0.0.1
=============================== warnings summary ===============================
../usr/local/lib/python3.11/site-packages/fastapi/testclient.py:1
/usr/local/lib/python3.11/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
tests/test_browse_client.py::TestBrowseClientBrowse::test_browse_single_page
tests/test_browse_client.py::TestBrowseClientBrowse::test_browse_handles_timeout
tests/test_browse_client.py::TestBrowseClientBrowse::test_browse_multiple_urls
/app/src/web/browse/client.py:445: RuntimeWarning: coroutine 'AsyncMockMixin._execute_mock_call' was never awaited
page.on("response", _on_response)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
------------------ generated xml file: /app/test-results.xml -------------------
=========================== short test summary info ============================
FAILED tests/test_browse_client.py::TestBrowseClientBrowse::test_browse_single_page
FAILED tests/test_validation.py::TestValidateUrlsAsync::test_validates_multiple_urls
FAILED tests/test_validation.py::TestValidateUrlsAsync::test_raises_on_private_ip_in_batch
================== 3 failed, 149 passed, 4 warnings in 16.74s ==================

12
artefacte_lot1/output_teste/web/test-results.xml Normal file
View file

File diff suppressed because one or more lines are too long