# Domain Check API - Anti-Fake News Platform

Sistema de verificare domenii pentru detectarea dezinformarii prin analiza vechimii domeniilor si riscurilor asociate.

## 🚀 Quick Start - API Usage

### Endpoints Disponibile

**Base URL:** `http://domain-check-api:11000`

**Endpoint principal:** `POST /api/v1/check/check`

### Apel Complet - Toate Datele Posibile

Pentru a obține **toate datele disponibile** despre un domeniu:

```bash
curl -X POST "http://domain-check-api:11000/api/v1/check/check" \
  -H "Content-Type: application/json" \
  -d '{
    "domain": "example.com",
    "check_options": {
      "whois": true,
      "dns": true,
      "ssl": true,
      "ip_intelligence": true,
      "http_analysis": true,
      "blacklist": true,
      "port_scan": true,
      "subdomains": true,
      "force_refresh": false
    }
  }'
```

**Răspuns:** JSON cu:
- `whois` - Date WHOIS/RDAP complete (vârstă domeniu, registrar, name servers, DNSSEC)
- `dns` - Toate recordurile DNS (A, AAAA, MX, TXT, NS, CNAME, SOA)
- `ssl` - Certificate SSL/TLS details (issuer, validity, chain, self-signed status)
- `ip_intelligence` - Geolocation, ASN, ISP, reverse DNS pentru toate IP-urile
- `http_analysis` - HTTP headers, security headers, redirects, technologies detected
- `blacklist` - Verificare în DNSBL (spam, malware, phishing blacklists)
- `port_scan` - Port scanning TCP common (22, 80, 443, 8080, etc.)
- `subdomains` - Subdomain enumeration (www, mail, ftp, api, etc.)
- `risk_score` - Scor de risc calculat (0-100) cu breakdown pe categorii

### Apel Rapid - Date Esențiale

Pentru verificări rapide (doar WHOIS + DNS + SSL):

```bash
curl -X POST "http://domain-check-api:11000/api/v1/check/check" \
  -H "Content-Type: application/json" \
  -d '{
    "domain": "example.com",
    "check_options": {
      "whois": true,
      "dns": true,
      "ssl": true
    }
  }'
```

### Health Check

```bash
curl http://domain-check-api:11000/health
```

Răspuns:
```json
{
  "status": "healthy",
  "database": "connected",
  "redis": "connected",
  "environment": "production",
  "version": "v1"
}
```

---

## 🌐 Integrare ca Serviciu (Kong API Gateway / Microservices)

### Configurare ca Backend Service

API-ul poate fi folosit ca **backend service** pentru Kong, NGINX, Traefik sau alte API gateways.

#### 1. Configurare Kong Gateway

```bash
# Adaugă service
curl -X POST http://kong-admin:8001/services \
  --data name=domain-check-api \
  --data url=http://domain-check-api:11000

# Adaugă route
curl -X POST http://kong-admin:8001/services/domain-check-api/routes \
  --data paths[]=/domain-check \
  --data strip_path=false

# Activează rate limiting
curl -X POST http://kong-admin:8001/services/domain-check-api/plugins \
  --data name=rate-limiting \
  --data config.minute=100 \
  --data config.hour=1000
```

Acum API-ul e disponibil la: `http://your-kong-gateway/domain-check/api/v1/check/check`

#### 2. Configurare NGINX Reverse Proxy

```nginx
upstream domain_check_backend {
    server domain-check-api:11000;
}

server {
    listen 80;
    server_name api.example.com;

    location /domain-check/ {
        proxy_pass http://domain_check_backend/;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        
        # Timeouts pentru operații lungi (port scan, subdomain enum)
        proxy_read_timeout 300s;
        proxy_connect_timeout 10s;
    }
}
```

#### 3. Docker Swarm / Kubernetes Service

**Docker Swarm:**
```yaml
version: '3.8'
services:
  domain-check:
    image: didiai-domain-check:latest
    deploy:
      replicas: 3
      update_config:
        parallelism: 1
        delay: 10s
    environment:
      DATABASE_URL: postgresql://user:pass@db-cluster:5432/domain_check
      REDIS_URL: redis://redis-cluster:6379/0
    networks:
      - backend
```

**Kubernetes Service:**
```yaml
apiVersion: v1
kind: Service
metadata:
  name: domain-check-api
spec:
  selector:
    app: domain-check
  ports:
    - protocol: TCP
      port: 80
      targetPort: 51000
  type: ClusterIP
```

#### 4. Environment Variables pentru External DB

Toate parametrii DB sunt în `.env` și pot fi schimbați pentru a folosi un **cluster PostgreSQL extern**:

```bash
# .env - Configurare pentru DB Cluster extern
DB_HOST=postgres-cluster.example.com
DB_PORT=5432
DB_USER=domain_check_user
DB_PASSWORD=secure_password_here
DB_NAME=domain_check

# Redis Cluster extern
REDIS_HOST=redis-cluster.example.com
REDIS_PORT=6379
REDIS_PASSWORD=redis_password_here
```

API-ul se va conecta automat la DB-ul extern specificat în variabilele de mediu.

---

## 📊 Database Schema - Structura Tabelelor

API-ul folosește PostgreSQL cu următoarele tabele:

### 1. `domains` - Domenii Verificate

Tabela principală cu toate domeniile procesate.

| Coloană | Tip | Descriere |
|---------|-----|-----------|
| `id` | UUID | Primary key |
| `domain` | VARCHAR(255) | Domeniul principal (ex: `example` din `example.com`) |
| `subdomain` | VARCHAR(255) | Subdomenul (ex: `www` din `www.example.com`) |
| `tld` | VARCHAR(50) | Top Level Domain (ex: `com`, `org`, `ro`) |
| `full_domain` | VARCHAR(255) | Domeniul complet (UNIQUE) |
| `first_seen_at` | TIMESTAMP | Prima verificare |
| `last_checked_at` | TIMESTAMP | Ultima verificare |
| `check_count` | INTEGER | Număr total verificări |
| `is_active` | BOOLEAN | Status activ/inactiv |
| `created_at` | TIMESTAMP | Data creării înregistrării |
| `updated_at` | TIMESTAMP | Data ultimei actualizări |

**Indexes:** `full_domain` (UNIQUE), `domain`, `tld`, `last_checked_at`, `is_active`

### 2. `whois_records` - Date WHOIS/RDAP

Stochează informații WHOIS pentru fiecare domeniu.

| Coloană | Tip | Descriere |
|---------|-----|-----------|
| `id` | UUID | Primary key |
| `domain_id` | UUID | Foreign key → `domains(id)` |
| `creation_date` | TIMESTAMP | Data creării domeniului |
| `expiration_date` | TIMESTAMP | Data expirării |
| `updated_date` | TIMESTAMP | Ultima actualizare WHOIS |
| `registrar` | VARCHAR(255) | Registrar (ex: GoDaddy, Namecheap) |
| `registrar_url` | VARCHAR(500) | URL registrar |
| `registrant_org` | VARCHAR(255) | Organizația proprietarului |
| `registrant_country` | VARCHAR(2) | Țara (cod ISO) |
| `admin_email` | VARCHAR(255) | Email administrator |
| `name_servers` | TEXT[] | Array cu name servers |
| `status` | TEXT[] | Array cu statusuri domeniu |
| `dnssec` | BOOLEAN | DNSSEC activat/nu |
| `raw_whois_data` | JSONB | Date WHOIS raw (JSON) |
| `data_source` | VARCHAR(50) | Sursa datelor (`rdap`, `whoxy`, `manual`) |
| `fetched_at` | TIMESTAMP | Când au fost preluate datele |

**Indexes:** `domain_id`, `creation_date`, `registrar`, `data_source`, `fetched_at`

### 3. `dns_records` - Înregistrări DNS

Toate recordurile DNS pentru fiecare domeniu.

| Coloană | Tip | Descriere |
|---------|-----|-----------|
| `id` | UUID | Primary key |
| `domain_id` | UUID | Foreign key → `domains(id)` |
| `record_type` | VARCHAR(10) | Tip record (`A`, `AAAA`, `MX`, `TXT`, `NS`, `CNAME`, `SOA`) |
| `record_value` | TEXT | Valoarea recordului |
| `ttl` | INTEGER | Time To Live (secunde) |
| `priority` | INTEGER | Prioritate (pentru MX records) |
| `fetched_at` | TIMESTAMP | Când a fost preluat recordul |

**Indexes:** `domain_id`, `(domain_id, record_type)`, `record_type`, `fetched_at`

### 4. `ssl_certificates` - Certificate SSL/TLS

Informații despre certificatele SSL ale domeniilor.

| Coloană | Tip | Descriere |
|---------|-----|-----------|
| `id` | UUID | Primary key |
| `domain_id` | UUID | Foreign key → `domains(id)` |
| `issuer` | VARCHAR(255) | Emitent certificat (ex: Let's Encrypt) |
| `subject` | VARCHAR(255) | Subiect certificat |
| `valid_from` | TIMESTAMP | Valabil de la |
| `valid_until` | TIMESTAMP | Valabil până la |
| `serial_number` | VARCHAR(255) | Număr serial certificat |
| `signature_algorithm` | VARCHAR(100) | Algoritm semnătură (ex: SHA256) |
| `key_size` | INTEGER | Mărime cheie (2048, 4096 bits) |
| `is_wildcard` | BOOLEAN | Certificat wildcard (`*.example.com`) |
| `is_self_signed` | BOOLEAN | Self-signed certificate |
| `is_valid` | BOOLEAN | Certificat valid |
| `certificate_chain` | JSONB | Lanțul complet de certificate (JSON) |
| `fetched_at` | TIMESTAMP | Când a fost verificat certificatul |

**Indexes:** `domain_id`, `valid_until`, `is_valid`, `fetched_at`

### Relații între Tabele

```
domains (1) ─── (N) whois_records
              ├─ (N) dns_records
              └─ (N) ssl_certificates
```

Toate relațiile au `ON DELETE CASCADE` - ștergerea unui domeniu șterge automat toate datele asociate.

### Migrare la DB Cluster Extern

Pentru a crea schema în **cluster PostgreSQL nou**:

```bash
# 1. Export schema
docker exec domain_check_postgres pg_dump -U dns_admin -d domain_check --schema-only > schema.sql

# 2. Import în cluster nou
psql -h postgres-cluster.example.com -U admin -d domain_check < schema.sql

# 3. Actualizează .env cu noile credentials
DB_HOST=postgres-cluster.example.com
DB_PORT=5432
DB_USER=domain_check_user
DB_PASSWORD=new_secure_password
DB_NAME=domain_check

# 4. Restart API
docker compose restart domain_check_api
```

Sau folosește direct script-ul de inițializare:
```bash
psql -h postgres-cluster.example.com -U admin -d domain_check < init-scripts/01-init-db.sql
```

---

## Caracteristici

- Verificare WHOIS/RDAP complet
- Integrare Whoxy API
- DNS records checking (A, AAAA, MX, TXT, NS, CNAME, SOA)
- SSL certificate validation
- IP Intelligence (geolocation, ASN, reverse DNS)
- HTTP Security Analysis
- DNSBL Blacklist checking
- Port scanning
- Subdomain enumeration
- Risk scoring engine avansat
- PostgreSQL + Redis caching
- Swagger UI + ReDoc documentation
- Docker Compose setup complet
- GitLab CI/CD pipeline cu SAST

---

## Deployment pe Orice Server

### Metoda 1: Script Automat (Recomandat)

```bash
# Descarca si ruleaza scriptul de deployment
curl -sSL (modul domain_check al platformei DIDI) -o deploy.sh
chmod +x deploy.sh
./deploy.sh install
```

Scriptul va:
- Verifica prerequisitele (Docker, Docker Compose, Git)
- Clona repository-ul
- Genera fisierul `.env` cu parole securizate
- Construi si porni containerele

### Metoda 2: Manual

```bash
# 1. Cloneaza repository-ul
git clone <didi-lot1-ai>/ai_platform/modules/domain_check
cd domain-check

# 2. Copiaza si editeaza configuratia
cp .env.example .env
nano .env  # sau vim, sau orice editor

# 3. Porneste serviciile
docker compose up -d --build

# 4. Verifica statusul
docker compose ps
curl http://domain-check-api:11000/health
```

### Metoda 3: Din GitLab Container Registry

```bash
# Login la registry (daca e necesar)
docker login <registry-didi>

# Pull imaginea
docker pull didiai-domain-check:latest

# Cloneaza doar fisierele de configurare
git clone <didi-lot1-ai>/ai_platform/modules/domain_check
cd domain-check

# Editeaza .env
nano .env

# Porneste cu imaginea din registry
docker compose up -d
```

---

## Configurare pentru Serverul Tau

### Fisierul .env

Toate setarile sunt in `.env`. Editeaza pentru mediul tau:

```bash
# ===========================================
# PORTURI - Modifica dupa nevoie
# ===========================================
# Schema: Dev=5xxxx, Prod=1xxxx
# x1xxx = API/Gateway
# x20xx = PostgreSQL
# x23xx = Redis

API_PORT=51000              # Portul API-ului (schimba la 11000 pentru prod)
DB_EXTERNAL_PORT=52000      # Port extern PostgreSQL (12000 pentru prod)
REDIS_EXTERNAL_PORT=52300   # Port extern Redis (12300 pentru prod)

# ===========================================
# DATABASE - Genereaza parole noi!
# ===========================================
DB_PASSWORD=SCHIMBA_ACEASTA_PAROLA
DB_USER=dns_admin
DB_NAME=domain_check

# ===========================================
# API KEYS - Adauga cheile tale
# ===========================================
WHOXY_API_KEY=cheia_ta_whoxy
VIRUSTOTAL_API_KEY=cheia_ta_virustotal

# ===========================================
# SECURITATE - OBLIGATORIU pentru productie!
# ===========================================
SECRET_KEY=genereaza_un_string_random_de_32_caractere
FLASK_ENV=production        # development sau production
DEBUG=False                 # True doar pentru development
```

### Setare Hostname/DNS

API-ul va fi accesibil prin:
- **IP direct**: `http://<IP_SERVER>:<API_PORT>/`
- **Hostname**: `http://<hostname>:<API_PORT>/`

Pentru a seta un hostname custom:

```bash
# 1. Pe DNS server (daca ai control)
check-dns.example.com  A  <IP_SERVER>

# 2. Sau in /etc/hosts pe clienti
echo "<IP_SERVER>  check-dns.example.com" >> /etc/hosts

# 3. Sau cu reverse proxy (nginx/traefik)
# - Configureaza proxy pass catre domain-check-api:<API_PORT>
```

### Porturi Utilizate

| Serviciu | Port Default (Dev) | Port Productie | Variabila |
|----------|-------------------|----------------|-----------|
| API Flask | 51000 | 11000 | `API_PORT` |
| PostgreSQL | 52000 | 12000 | `DB_EXTERNAL_PORT` |
| Redis | 52300 | 12300 | `REDIS_EXTERNAL_PORT` |

Porturile pot fi schimbate in `.env` - nu este nevoie de modificari in alte fisiere.

### Firewall

Deschide doar portul API pentru acces extern:

```bash
# UFW (Ubuntu/Debian)
sudo ufw allow 51000/tcp

# firewalld (CentOS/RHEL)
sudo firewall-cmd --add-port=51000/tcp --permanent
sudo firewall-cmd --reload

# iptables
sudo iptables -A INPUT -p tcp --dport 51000 -j ACCEPT
```

PostgreSQL si Redis nu trebuie expuse public (sunt doar pentru comunicatie interna).

---

## Structura Proiectului

```
domain-check/
├── api/                        # Flask API
│   ├── app/
│   │   ├── models/            # SQLAlchemy models
│   │   ├── routes/            # API endpoints
│   │   ├── services/          # Business logic (WHOIS, DNS, SSL, etc.)
│   │   ├── utils/             # Helper functions
│   │   ├── tasks/             # Celery tasks
│   │   ├── static/            # Dashboard HTML/CSS/JS
│   │   ├── config.py          # Configuration
│   │   └── __init__.py        # App factory
│   ├── Dockerfile
│   ├── requirements.txt
│   └── run.py                 # Entry point
├── init-scripts/              # PostgreSQL init scripts
│   └── 01-init-db.sql
├── docker-compose.yml         # Docker orchestration
├── .gitlab-ci.yml             # CI/CD pipeline
├── deploy.sh                  # Deployment script
├── .env                       # Environment variables
├── use_api.md                 # API documentation
└── README.md                  # This file
```

---

## Servicii Docker

| Container | Descriere | Health Check |
|-----------|-----------|--------------|
| `domain_check_api` | Flask REST API + Dashboard | `/health` |
| `domain_check_postgres` | PostgreSQL 15 database | `pg_isready` |
| `domain_check_redis` | Redis cache | `redis-cli ping` |
| `domain_check_worker` | Celery background worker | `celery inspect ping` |

---

## Utilizare API

### Interfata Web (Dashboard)

Deschide in browser: `http://<server>:<API_PORT>/`

### Verificare Domain (curl)

```bash
# Check complet
curl -X POST "http://domain-check-api:11000/api/v1/check/check" \
  -H "Content-Type: application/json" \
  -d '{
    "domain": "example.com",
    "check_options": {
      "whois": true,
      "dns": true,
      "ssl": true,
      "ip_intelligence": true,
      "http_analysis": true,
      "blacklist": true,
      "port_scan": true,
      "subdomains": true
    }
  }'
```

### Health Check

```bash
curl http://domain-check-api:11000/health
```

### Documentatie Swagger

- Swagger UI: `http://<server>:<API_PORT>/docs`
- ReDoc: `http://<server>:<API_PORT>/redoc`

Pentru documentatie API completa, vezi [use_api.md](use_api.md).

---

## CI/CD Pipeline

Pipeline-ul GitLab include:

### Stages

1. **lint** - Code quality (flake8, black, isort, hadolint)
2. **security** - SAST scanning (bandit, safety, trivy)
3. **test** - Unit tests
4. **build** - Docker build & push to registry
5. **release** - Create deployment bundle
6. **deploy** - Deploy to server via SSH

### Container Registry

Imaginile sunt publicate la:
```
didiai-domain-check:latest
didiai-domain-check:<branch>
```

### Configurare Deploy Automat

Adauga in GitLab CI/CD Variables:

| Variable | Descriere |
|----------|-----------|
| `DEPLOY_HOST` | IP sau hostname server |
| `DEPLOY_USER` | User SSH (ex: admin365) |
| `SSH_PRIVATE_KEY` | Cheia privata SSH |
| `SSH_KNOWN_HOSTS` | Output din `ssh-keyscan <server>` |

---

## Comenzi Utile

### Deploy Script

```bash
./deploy.sh install   # Instalare fresh
./deploy.sh update    # Update la ultima versiune
./deploy.sh status    # Verifica statusul
./deploy.sh logs      # Vezi toate logs
./deploy.sh logs domain_check_api  # Logs doar pentru API
./deploy.sh start     # Porneste serviciile
./deploy.sh stop      # Opreste serviciile
./deploy.sh restart   # Restart servicii
```

### Docker Direct

```bash
# Start/Stop
docker compose up -d
docker compose down

# Rebuild
docker compose up -d --build

# Logs
docker compose logs -f domain_check_api

# Shell in container
docker exec -it domain_check_api bash

# Database access
docker exec -it domain_check_postgres psql -U dns_admin -d domain_check
```

---

## Troubleshooting

### Container nu porneste

```bash
# Verifica logs
docker compose logs domain_check_api

# Verifica network
docker network ls | grep dns-network

# Recreate containers
docker compose down && docker compose up -d --build
```

### Database connection error

```bash
# Verifica ca PostgreSQL e healthy
docker compose ps domain_check_postgres

# Verifica health
docker exec -it domain_check_postgres pg_isready -U dns_admin

# Restart database
docker compose restart domain_check_postgres
```

### API returns 500 error

```bash
# Check logs
docker compose logs -f domain_check_api

# Verify database schema
docker exec -it domain_check_postgres psql -U dns_admin -d domain_check -c "\dt"
```

### Port deja in uz

```bash
# Gaseste ce foloseste portul
sudo lsof -i :51000

# Schimba portul in .env
API_PORT=51001  # sau alt port liber
```

---

## Securitate - Checklist Productie

- [ ] Schimba toate parolele default in `.env`
- [ ] Seteaza `FLASK_ENV=production`
- [ ] Seteaza `DEBUG=False`
- [ ] Genereaza `SECRET_KEY` random (32+ caractere)
- [ ] Configureaza firewall (doar port API public)
- [ ] Setup HTTPS cu reverse proxy (nginx/traefik)
- [ ] Backup automat PostgreSQL
- [ ] Monitorizare logs

---

## Variabile de Mediu Complete

| Variabila | Default | Descriere |
|-----------|---------|-----------|
| `API_PORT` | 51000 | Port pentru Flask API |
| `API_HOST` | 0.0.0.0 | Bind address |
| `API_VERSION` | v1 | API version prefix |
| `DB_HOST` | domain_check_postgres | PostgreSQL hostname (container) |
| `DB_PORT` | 5432 | PostgreSQL port intern |
| `DB_EXTERNAL_PORT` | 52000 | PostgreSQL port extern |
| `DB_USER` | dns_admin | Database user |
| `DB_PASSWORD` | - | Database password |
| `DB_NAME` | domain_check | Database name |
| `REDIS_HOST` | domain_check_redis | Redis hostname (container) |
| `REDIS_PORT` | 6379 | Redis port intern |
| `REDIS_EXTERNAL_PORT` | 52300 | Redis port extern |
| `REDIS_DB` | 0 | Redis database number |
| `WHOXY_API_KEY` | - | Whoxy API key |
| `VIRUSTOTAL_API_KEY` | - | VirusTotal API key |
| `FLASK_ENV` | development | Flask environment |
| `DEBUG` | True | Debug mode |
| `SECRET_KEY` | - | Flask secret key |
| `LOG_LEVEL` | INFO | Logging level |
| `REDIS_TTL_HOT` | 21600 | Cache TTL hot (6h) |
| `REDIS_TTL_WARM` | 86400 | Cache TTL warm (24h) |
| `REDIS_TTL_COLD` | 604800 | Cache TTL cold (7d) |

---

## License

MIT License - Free for use in anti-fake news projects

---

**Built for fighting disinformation**

**Last Updated:** 2026-02-04