62 lines
6.7 KiB
Markdown
62 lines
6.7 KiB
Markdown
# CLAUDE.md
|
||
|
||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||
|
||
## What This Is
|
||
|
||
Domain Check API — a domain verification and risk-scoring service for detecting disinformation sources (newly registered / suspicious domains). Flask REST API + PostgreSQL + Redis + Celery worker, all run via Docker Compose. Live on this server at `http://domain-check-api:11000` (production ports: API 11000, PostgreSQL 12000, Redis 12300 — set in `.env`, the README sometimes still shows dev ports 5xxxx).
|
||
|
||
## Commands
|
||
|
||
```bash
|
||
# Run the full stack (the only supported way to run it — the API expects Postgres/Redis containers)
|
||
docker compose up -d --build
|
||
docker compose ps
|
||
docker compose logs -f domain_check_api
|
||
|
||
# Smoke test
|
||
curl http://localhost:11000/health
|
||
./test-api-complete.sh # exercises the check endpoint
|
||
./test-complete-lan.sh # LAN integration test
|
||
|
||
# Manual check call
|
||
curl -X POST http://localhost:11000/api/v1/check/check \
|
||
-H "Content-Type: application/json" \
|
||
-d '{"domain": "example.com", "check_options": {"whois": true, "dns": true, "ssl": true}}'
|
||
|
||
# Lint (matches .gitlab-ci.yml)
|
||
flake8 api/app --max-line-length=120 --ignore=E501,W503
|
||
black --check api/app
|
||
isort --check-only api/app
|
||
|
||
# Database shell
|
||
docker exec -it didiAI-domain-check-db psql -U dns_admin -d domain_check
|
||
|
||
# Deploy helper (wraps docker compose)
|
||
./deploy.sh status|logs|start|stop|restart|update
|
||
```
|
||
|
||
There is no test suite — CI's "test" stage only verifies `create_app()` succeeds. CI (GitLab) runs lint → SAST (bandit/safety/trivy) → build → push to `didiai-domain-check` → SSH deploy; all lint/security jobs are `allow_failure: true`.
|
||
|
||
## Architecture
|
||
|
||
Everything lives in `api/app/` (the `dashboard/` directory is vestigial — only a Dockerfile/requirements, not in docker-compose; the actual dashboard is `api/app/static/index.html` served at `/`).
|
||
|
||
**Request flow:** `run.py` → `create_app()` app factory ([api/app/__init__.py](api/app/__init__.py)) → flask-restx namespaces under `/api/v1` → route handlers in `app/routes/` call service classes in `app/services/` → results persisted via SQLAlchemy models in `app/models/` and cached in Redis. Served by **gunicorn** (gthread workers) in the container — `python run.py` is dev-only.
|
||
|
||
- **`app/routes/check.py`** is the only fully implemented route (POST `/api/v1/check/check`). It runs the independent lookups **concurrently** in two phases via `ThreadPoolExecutor` (phase 1 needs only the domain: whois/dns/ssl/http/subdomains; phase 2 needs DNS results: ip/blacklist/port/mail). Each task runs inside `app.app_context()` because services read `current_app`. **All DB writes happen in the main thread afterwards** — the SQLAlchemy session is not thread-safe, so never add DB writes inside the threaded lookups. `domain.py`, `search.py`, `stats.py`, `batch.py` are stubs registered inside try/except ImportError so missing implementations fail silently.
|
||
- **`app/services/`** — one class per check type: `whois_service` (RDAP first, Whoxy API fallback; `normalize_whois_date()` coerces all dates to `datetime|None` — raw strings here were the root cause of the `.ro` 500), `dns_service`, `ssl_service`, `ip_intelligence_service`, `http_analysis_service`, `blacklist_service` (DNSBL), `port_scan_service`, `subdomain_service` (Certificate Transparency), and `mail_intelligence_service` (SPF/DKIM/DMARC parsing, MTA-STS/TLS-RPT/DANE, MX provider fingerprint + STARTTLS, optional SMTP RCPT probe — degrades gracefully when outbound :25 is blocked). `risk_scorer.py` combines outputs into a weighted 0–100 score (weights/thresholds in `config.py`); `_coerce_datetime()` there is the defense-in-depth guard against stray string dates.
|
||
- **`check_options`** flags (all default true except `port_scan`/`subdomains`/`smtp_probe` which default false): `whois, dns, ssl, ip_intelligence, http_analysis, blacklist, mail, port_scan, subdomains, smtp_probe, force_refresh`. The response includes a top-level `availability` block (`is_registered`/`is_available`/`confidence`) derived from combined WHOIS+DNS signals — WHOIS alone is unreliable for sparse registries like ROTLD.
|
||
- **Rate limiting** (`Flask-Limiter`, Redis-backed) is wired in `__init__.py` with internal LAN/loopback and `/health` exempt (`_is_internal_request`), errors swallowed. Tune via `RATELIMIT_DEFAULT` env. It exists to protect the paid Whoxy/VirusTotal quotas from external abuse, not internal callers.
|
||
- **`app/api_models.py`** defines all flask-restx request/response models. Note the pattern in `check.py`: models are created on a throwaway namespace at import time for decorators, then `__init__.py` re-attaches them to the real namespaces — keep model definitions in `api_models.py`, not inline.
|
||
- **Celery** is wired up (`celery_app.py`, separate `domain_check_worker` container) but `app/tasks/__init__.py` is empty — no tasks exist yet; batch processing is the intended use.
|
||
|
||
**Configuration:** `app/config.py` selects Development/Production/Testing via `FLASK_ENV`. Production flips important defaults: Swagger/ReDoc disabled (`DISABLE_SWAGGER`), minimal `/health` response (`SIMPLE_HEALTH`), CORS empty unless `CORS_ORIGINS` is set. All runtime config comes from `.env` (see `.env.example`); ports are only ever changed there, never in compose/code.
|
||
|
||
**Database schema** is created by `init-scripts/01-init-db.sql`, which runs only on first Postgres volume creation. There is no Flask-Migrate `migrations/` directory despite the extension being initialized — schema changes must be made in both the SQL init script and the SQLAlchemy models, and applied manually to existing databases. All child tables (`whois_records`, `dns_records`, `ssl_certificates`, etc.) FK to `domains` with `ON DELETE CASCADE`.
|
||
|
||
**Caching:** Redis with three TTL tiers (`REDIS_TTL_HOT/WARM/COLD`); a check request with `force_refresh: false` returns cached/DB data when fresh. Redis failure is non-fatal — the app logs a warning and runs with caching disabled.
|
||
|
||
## Network Context
|
||
|
||
This is the **T4 module (evaluare credibilitate sursă)** of the DIDI platform, living at `ai_platform/modules/domain_check`. It runs as an isolated stack (own Postgres + Redis on the internal `domain_check_network`); the API container also joins the shared external `didi-network` under the alias **`domain-check-api`**, so `agent-v3` (Lot 2) calls it at `http://domain-check-api:11000/api/v1/check/check` with no extra config. The API port is **not** published to the host (the platform gateway owns host:11000) — reach it via the didi-network alias, or `docker exec` into a container on that network. Docs: `API_ARCHITECTURE.md` (detailed design), `use_api.md` (consumer-facing API docs), `ACCESS_INFO.md`, `CURRENT_STATUS.md`, `NEXT_STEPS.md` (roadmap).
|