didi-lot1-ai/ai_platform/modules/domain_check

Domain Check API - Anti-Fake News Platform

Sistema de verificare domenii pentru detectarea dezinformarii prin analiza vechimii domeniilor si riscurilor asociate.

🚀 Quick Start - API Usage

Endpoints Disponibile

Base URL: http://domain-check-api:11000

Endpoint principal: POST /api/v1/check/check

Apel Complet - Toate Datele Posibile

Pentru a obține toate datele disponibile despre un domeniu:

curl -X POST "http://domain-check-api:11000/api/v1/check/check" \
  -H "Content-Type: application/json" \
  -d '{
    "domain": "example.com",
    "check_options": {
      "whois": true,
      "dns": true,
      "ssl": true,
      "ip_intelligence": true,
      "http_analysis": true,
      "blacklist": true,
      "port_scan": true,
      "subdomains": true,
      "force_refresh": false
    }
  }'

Răspuns: JSON cu:

• whois - Date WHOIS/RDAP complete (vârstă domeniu, registrar, name servers, DNSSEC)
• dns - Toate recordurile DNS (A, AAAA, MX, TXT, NS, CNAME, SOA)
• ssl - Certificate SSL/TLS details (issuer, validity, chain, self-signed status)
• ip_intelligence - Geolocation, ASN, ISP, reverse DNS pentru toate IP-urile
• http_analysis - HTTP headers, security headers, redirects, technologies detected
• blacklist - Verificare în DNSBL (spam, malware, phishing blacklists)
• port_scan - Port scanning TCP common (22, 80, 443, 8080, etc.)
• subdomains - Subdomain enumeration (www, mail, ftp, api, etc.)
• risk_score - Scor de risc calculat (0-100) cu breakdown pe categorii

Apel Rapid - Date Esențiale

Pentru verificări rapide (doar WHOIS + DNS + SSL):

curl -X POST "http://domain-check-api:11000/api/v1/check/check" \
  -H "Content-Type: application/json" \
  -d '{
    "domain": "example.com",
    "check_options": {
      "whois": true,
      "dns": true,
      "ssl": true
    }
  }'

Health Check

curl http://domain-check-api:11000/health

Răspuns:

{
  "status": "healthy",
  "database": "connected",
  "redis": "connected",
  "environment": "production",
  "version": "v1"
}

---

🌐 Integrare ca Serviciu (Kong API Gateway / Microservices)

Configurare ca Backend Service

API-ul poate fi folosit ca backend service pentru Kong, NGINX, Traefik sau alte API gateways.

1. Configurare Kong Gateway

# Adaugă service
curl -X POST http://kong-admin:8001/services \
  --data name=domain-check-api \
  --data url=http://domain-check-api:11000

# Adaugă route
curl -X POST http://kong-admin:8001/services/domain-check-api/routes \
  --data paths[]=/domain-check \
  --data strip_path=false

# Activează rate limiting
curl -X POST http://kong-admin:8001/services/domain-check-api/plugins \
  --data name=rate-limiting \
  --data config.minute=100 \
  --data config.hour=1000

Acum API-ul e disponibil la: http://your-kong-gateway/domain-check/api/v1/check/check

2. Configurare NGINX Reverse Proxy

upstream domain_check_backend {
    server domain-check-api:11000;
}

server {
    listen 80;
    server_name api.example.com;

    location /domain-check/ {
        proxy_pass http://domain_check_backend/;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        
        # Timeouts pentru operații lungi (port scan, subdomain enum)
        proxy_read_timeout 300s;
        proxy_connect_timeout 10s;
    }
}

3. Docker Swarm / Kubernetes Service

Docker Swarm:

version: '3.8'
services:
  domain-check:
    image: didiai-domain-check:latest
    deploy:
      replicas: 3
      update_config:
        parallelism: 1
        delay: 10s
    environment:
      DATABASE_URL: postgresql://user:pass@db-cluster:5432/domain_check
      REDIS_URL: redis://redis-cluster:6379/0
    networks:
      - backend

Kubernetes Service:

apiVersion: v1
kind: Service
metadata:
  name: domain-check-api
spec:
  selector:
    app: domain-check
  ports:
    - protocol: TCP
      port: 80
      targetPort: 51000
  type: ClusterIP

4. Environment Variables pentru External DB

Toate parametrii DB sunt în .env și pot fi schimbați pentru a folosi un cluster PostgreSQL extern:

# .env - Configurare pentru DB Cluster extern
DB_HOST=postgres-cluster.example.com
DB_PORT=5432
DB_USER=domain_check_user
DB_PASSWORD=secure_password_here
DB_NAME=domain_check

# Redis Cluster extern
REDIS_HOST=redis-cluster.example.com
REDIS_PORT=6379
REDIS_PASSWORD=redis_password_here

API-ul se va conecta automat la DB-ul extern specificat în variabilele de mediu.

---

📊 Database Schema - Structura Tabelelor

API-ul folosește PostgreSQL cu următoarele tabele:

1. domains - Domenii Verificate

Tabela principală cu toate domeniile procesate.

Coloană	Tip	Descriere
id	UUID	Primary key
domain	VARCHAR(255)	Domeniul principal (ex: example din example.com)
subdomain	VARCHAR(255)	Subdomenul (ex: www din www.example.com)
tld	VARCHAR(50)	Top Level Domain (ex: com, org, ro)
full_domain	VARCHAR(255)	Domeniul complet (UNIQUE)
first_seen_at	TIMESTAMP	Prima verificare
last_checked_at	TIMESTAMP	Ultima verificare
check_count	INTEGER	Număr total verificări
is_active	BOOLEAN	Status activ/inactiv
created_at	TIMESTAMP	Data creării înregistrării
updated_at	TIMESTAMP	Data ultimei actualizări

Indexes: full_domain (UNIQUE), domain, tld, last_checked_at, is_active

2. whois_records - Date WHOIS/RDAP

Stochează informații WHOIS pentru fiecare domeniu.

Coloană	Tip	Descriere
id	UUID	Primary key
domain_id	UUID	Foreign key → domains(id)
creation_date	TIMESTAMP	Data creării domeniului
expiration_date	TIMESTAMP	Data expirării
updated_date	TIMESTAMP	Ultima actualizare WHOIS
registrar	VARCHAR(255)	Registrar (ex: GoDaddy, Namecheap)
registrar_url	VARCHAR(500)	URL registrar
registrant_org	VARCHAR(255)	Organizația proprietarului
registrant_country	VARCHAR(2)	Țara (cod ISO)
admin_email	VARCHAR(255)	Email administrator
name_servers	TEXT[]	Array cu name servers
status	TEXT[]	Array cu statusuri domeniu
dnssec	BOOLEAN	DNSSEC activat/nu
raw_whois_data	JSONB	Date WHOIS raw (JSON)
data_source	VARCHAR(50)	Sursa datelor (rdap, whoxy, manual)
fetched_at	TIMESTAMP	Când au fost preluate datele

Indexes: domain_id, creation_date, registrar, data_source, fetched_at

3. dns_records - Înregistrări DNS

Toate recordurile DNS pentru fiecare domeniu.

Coloană	Tip	Descriere
id	UUID	Primary key
domain_id	UUID	Foreign key → domains(id)
record_type	VARCHAR(10)	Tip record (A, AAAA, MX, TXT, NS, CNAME, SOA)
record_value	TEXT	Valoarea recordului
ttl	INTEGER	Time To Live (secunde)
priority	INTEGER	Prioritate (pentru MX records)
fetched_at	TIMESTAMP	Când a fost preluat recordul

Indexes: domain_id, (domain_id, record_type), record_type, fetched_at

4. ssl_certificates - Certificate SSL/TLS

Informații despre certificatele SSL ale domeniilor.

Coloană	Tip	Descriere
id	UUID	Primary key
domain_id	UUID	Foreign key → domains(id)
issuer	VARCHAR(255)	Emitent certificat (ex: Let's Encrypt)
subject	VARCHAR(255)	Subiect certificat
valid_from	TIMESTAMP	Valabil de la
valid_until	TIMESTAMP	Valabil până la
serial_number	VARCHAR(255)	Număr serial certificat
signature_algorithm	VARCHAR(100)	Algoritm semnătură (ex: SHA256)
key_size	INTEGER	Mărime cheie (2048, 4096 bits)
is_wildcard	BOOLEAN	Certificat wildcard (*.example.com)
is_self_signed	BOOLEAN	Self-signed certificate
is_valid	BOOLEAN	Certificat valid
certificate_chain	JSONB	Lanțul complet de certificate (JSON)
fetched_at	TIMESTAMP	Când a fost verificat certificatul

Indexes: domain_id, valid_until, is_valid, fetched_at

Relații între Tabele

domains (1) ─── (N) whois_records
              ├─ (N) dns_records
              └─ (N) ssl_certificates

Toate relațiile au ON DELETE CASCADE - ștergerea unui domeniu șterge automat toate datele asociate.

Migrare la DB Cluster Extern

Pentru a crea schema în cluster PostgreSQL nou:

# 1. Export schema
docker exec domain_check_postgres pg_dump -U dns_admin -d domain_check --schema-only > schema.sql

# 2. Import în cluster nou
psql -h postgres-cluster.example.com -U admin -d domain_check < schema.sql

# 3. Actualizează .env cu noile credentials
DB_HOST=postgres-cluster.example.com
DB_PORT=5432
DB_USER=domain_check_user
DB_PASSWORD=new_secure_password
DB_NAME=domain_check

# 4. Restart API
docker compose restart domain_check_api

Sau folosește direct script-ul de inițializare:

psql -h postgres-cluster.example.com -U admin -d domain_check < init-scripts/01-init-db.sql

---

Caracteristici

• Verificare WHOIS/RDAP complet
• Integrare Whoxy API
• DNS records checking (A, AAAA, MX, TXT, NS, CNAME, SOA)
• SSL certificate validation
• IP Intelligence (geolocation, ASN, reverse DNS)
• HTTP Security Analysis
• DNSBL Blacklist checking
• Port scanning
• Subdomain enumeration
• Risk scoring engine avansat
• PostgreSQL + Redis caching
• Swagger UI + ReDoc documentation
• Docker Compose setup complet
• GitLab CI/CD pipeline cu SAST

---

Deployment pe Orice Server

Metoda 1: Script Automat (Recomandat)

# Descarca si ruleaza scriptul de deployment
curl -sSL (modul domain_check al platformei DIDI) -o deploy.sh
chmod +x deploy.sh
./deploy.sh install

Scriptul va:

• Verifica prerequisitele (Docker, Docker Compose, Git)
• Clona repository-ul
• Genera fisierul .env cu parole securizate
• Construi si porni containerele

Metoda 2: Manual

# 1. Cloneaza repository-ul
git clone <didi-lot1-ai>/ai_platform/modules/domain_check
cd domain-check

# 2. Copiaza si editeaza configuratia
cp .env.example .env
nano .env  # sau vim, sau orice editor

# 3. Porneste serviciile
docker compose up -d --build

# 4. Verifica statusul
docker compose ps
curl http://domain-check-api:11000/health

Metoda 3: Din GitLab Container Registry

# Login la registry (daca e necesar)
docker login <registry-didi>

# Pull imaginea
docker pull didiai-domain-check:latest

# Cloneaza doar fisierele de configurare
git clone <didi-lot1-ai>/ai_platform/modules/domain_check
cd domain-check

# Editeaza .env
nano .env

# Porneste cu imaginea din registry
docker compose up -d

---

Configurare pentru Serverul Tau

Fisierul .env

Toate setarile sunt in .env. Editeaza pentru mediul tau:

# ===========================================
# PORTURI - Modifica dupa nevoie
# ===========================================
# Schema: Dev=5xxxx, Prod=1xxxx
# x1xxx = API/Gateway
# x20xx = PostgreSQL
# x23xx = Redis

API_PORT=51000              # Portul API-ului (schimba la 11000 pentru prod)
DB_EXTERNAL_PORT=52000      # Port extern PostgreSQL (12000 pentru prod)
REDIS_EXTERNAL_PORT=52300   # Port extern Redis (12300 pentru prod)

# ===========================================
# DATABASE - Genereaza parole noi!
# ===========================================
DB_PASSWORD=SCHIMBA_ACEASTA_PAROLA
DB_USER=dns_admin
DB_NAME=domain_check

# ===========================================
# API KEYS - Adauga cheile tale
# ===========================================
WHOXY_API_KEY=cheia_ta_whoxy
VIRUSTOTAL_API_KEY=cheia_ta_virustotal

# ===========================================
# SECURITATE - OBLIGATORIU pentru productie!
# ===========================================
SECRET_KEY=genereaza_un_string_random_de_32_caractere
FLASK_ENV=production        # development sau production
DEBUG=False                 # True doar pentru development

Setare Hostname/DNS

API-ul va fi accesibil prin:

• IP direct: http://<IP_SERVER>:<API_PORT>/
• Hostname: http://<hostname>:<API_PORT>/

Pentru a seta un hostname custom:

# 1. Pe DNS server (daca ai control)
check-dns.example.com  A  <IP_SERVER>

# 2. Sau in /etc/hosts pe clienti
echo "<IP_SERVER>  check-dns.example.com" >> /etc/hosts

# 3. Sau cu reverse proxy (nginx/traefik)
# - Configureaza proxy pass catre domain-check-api:<API_PORT>

Porturi Utilizate

Serviciu	Port Default (Dev)	Port Productie	Variabila
API Flask	51000	11000	API_PORT
PostgreSQL	52000	12000	DB_EXTERNAL_PORT
Redis	52300	12300	REDIS_EXTERNAL_PORT

Porturile pot fi schimbate in .env - nu este nevoie de modificari in alte fisiere.

Firewall

Deschide doar portul API pentru acces extern:

# UFW (Ubuntu/Debian)
sudo ufw allow 51000/tcp

# firewalld (CentOS/RHEL)
sudo firewall-cmd --add-port=51000/tcp --permanent
sudo firewall-cmd --reload

# iptables
sudo iptables -A INPUT -p tcp --dport 51000 -j ACCEPT

PostgreSQL si Redis nu trebuie expuse public (sunt doar pentru comunicatie interna).

---

Structura Proiectului

domain-check/
├── api/                        # Flask API
│   ├── app/
│   │   ├── models/            # SQLAlchemy models
│   │   ├── routes/            # API endpoints
│   │   ├── services/          # Business logic (WHOIS, DNS, SSL, etc.)
│   │   ├── utils/             # Helper functions
│   │   ├── tasks/             # Celery tasks
│   │   ├── static/            # Dashboard HTML/CSS/JS
│   │   ├── config.py          # Configuration
│   │   └── __init__.py        # App factory
│   ├── Dockerfile
│   ├── requirements.txt
│   └── run.py                 # Entry point
├── init-scripts/              # PostgreSQL init scripts
│   └── 01-init-db.sql
├── docker-compose.yml         # Docker orchestration
├── .gitlab-ci.yml             # CI/CD pipeline
├── deploy.sh                  # Deployment script
├── .env                       # Environment variables
├── use_api.md                 # API documentation
└── README.md                  # This file

---

Servicii Docker

Container	Descriere	Health Check
domain_check_api	Flask REST API + Dashboard	/health
domain_check_postgres	PostgreSQL 15 database	pg_isready
domain_check_redis	Redis cache	redis-cli ping
domain_check_worker	Celery background worker	celery inspect ping

---

Utilizare API

Interfata Web (Dashboard)

Deschide in browser: http://<server>:<API_PORT>/

Verificare Domain (curl)

# Check complet
curl -X POST "http://domain-check-api:11000/api/v1/check/check" \
  -H "Content-Type: application/json" \
  -d '{
    "domain": "example.com",
    "check_options": {
      "whois": true,
      "dns": true,
      "ssl": true,
      "ip_intelligence": true,
      "http_analysis": true,
      "blacklist": true,
      "port_scan": true,
      "subdomains": true
    }
  }'

Health Check

curl http://domain-check-api:11000/health

Documentatie Swagger

• Swagger UI: http://<server>:<API_PORT>/docs
• ReDoc: http://<server>:<API_PORT>/redoc

Pentru documentatie API completa, vezi use_api.md.

---

CI/CD Pipeline

Pipeline-ul GitLab include:

Stages

1. lint - Code quality (flake8, black, isort, hadolint)
2. security - SAST scanning (bandit, safety, trivy)
3. test - Unit tests
4. build - Docker build & push to registry
5. release - Create deployment bundle
6. deploy - Deploy to server via SSH

Container Registry

Imaginile sunt publicate la:

didiai-domain-check:latest
didiai-domain-check:<branch>

Configurare Deploy Automat

Adauga in GitLab CI/CD Variables:

Variable	Descriere
DEPLOY_HOST	IP sau hostname server
DEPLOY_USER	User SSH (ex: admin365)
SSH_PRIVATE_KEY	Cheia privata SSH
SSH_KNOWN_HOSTS	Output din ssh-keyscan <server>

---

Comenzi Utile

Deploy Script

./deploy.sh install   # Instalare fresh
./deploy.sh update    # Update la ultima versiune
./deploy.sh status    # Verifica statusul
./deploy.sh logs      # Vezi toate logs
./deploy.sh logs domain_check_api  # Logs doar pentru API
./deploy.sh start     # Porneste serviciile
./deploy.sh stop      # Opreste serviciile
./deploy.sh restart   # Restart servicii

Docker Direct

# Start/Stop
docker compose up -d
docker compose down

# Rebuild
docker compose up -d --build

# Logs
docker compose logs -f domain_check_api

# Shell in container
docker exec -it domain_check_api bash

# Database access
docker exec -it domain_check_postgres psql -U dns_admin -d domain_check

---

Troubleshooting

Container nu porneste

# Verifica logs
docker compose logs domain_check_api

# Verifica network
docker network ls | grep dns-network

# Recreate containers
docker compose down && docker compose up -d --build

Database connection error

# Verifica ca PostgreSQL e healthy
docker compose ps domain_check_postgres

# Verifica health
docker exec -it domain_check_postgres pg_isready -U dns_admin

# Restart database
docker compose restart domain_check_postgres

API returns 500 error

# Check logs
docker compose logs -f domain_check_api

# Verify database schema
docker exec -it domain_check_postgres psql -U dns_admin -d domain_check -c "\dt"

Port deja in uz

# Gaseste ce foloseste portul
sudo lsof -i :51000

# Schimba portul in .env
API_PORT=51001  # sau alt port liber

---

Securitate - Checklist Productie

• Schimba toate parolele default in .env
• Seteaza FLASK_ENV=production
• Seteaza DEBUG=False
• Genereaza SECRET_KEY random (32+ caractere)
• Configureaza firewall (doar port API public)
• Setup HTTPS cu reverse proxy (nginx/traefik)
• Backup automat PostgreSQL
• Monitorizare logs

---

Variabile de Mediu Complete

Variabila	Default	Descriere
API_PORT	51000	Port pentru Flask API
API_HOST	0.0.0.0	Bind address
API_VERSION	v1	API version prefix
DB_HOST	domain_check_postgres	PostgreSQL hostname (container)
DB_PORT	5432	PostgreSQL port intern
DB_EXTERNAL_PORT	52000	PostgreSQL port extern
DB_USER	dns_admin	Database user
DB_PASSWORD	-	Database password
DB_NAME	domain_check	Database name
REDIS_HOST	domain_check_redis	Redis hostname (container)
REDIS_PORT	6379	Redis port intern
REDIS_EXTERNAL_PORT	52300	Redis port extern
REDIS_DB	0	Redis database number
WHOXY_API_KEY	-	Whoxy API key
VIRUSTOTAL_API_KEY	-	VirusTotal API key
FLASK_ENV	development	Flask environment
DEBUG	True	Debug mode
SECRET_KEY	-	Flask secret key
LOG_LEVEL	INFO	Logging level
REDIS_TTL_HOT	21600	Cache TTL hot (6h)
REDIS_TTL_WARM	86400	Cache TTL warm (24h)
REDIS_TTL_COLD	604800	Cache TTL cold (7d)

---

License

MIT License - Free for use in anti-fake news projects

---

Built for fighting disinformation

Last Updated: 2026-02-04