# DOMAIN-CHECK API - Arhitectură Completă

**Versiune:** 1.0.0
**Data:** 2026-01-29
**Scop:** Anti-Fake News - Domain Verification & Risk Scoring

---

## 📋 CUPRINS

1. [Overview](#overview)
2. [Stack Tehnologic](#stack-tehnologic)
3. [Arhitectura Sistemului](#arhitectura-sistemului)
4. [API Endpoints](#api-endpoints)
5. [Database Schema](#database-schema)
6. [Risk Scoring Engine](#risk-scoring-engine)
7. [Caching Strategy](#caching-strategy)
8. [External Services](#external-services)
9. [Docker Setup](#docker-setup)
10. [Deployment](#deployment)

---

## 🎯 OVERVIEW

### Obiectiv Principal
Verificare automată a domeniilor/subdomeniilor pentru identificarea surselor potențiale de dezinformare bazată pe:
- **Vechimea domeniului** (risc ridicat pentru domenii < 6 luni)
- **Reputația domeniului** (blacklists, typosquatting)
- **Date WHOIS/RDAP** (registrar, istoric modificări)
- **Verificări DNS și SSL**

### Flow Principal
```
Input (Text/Link)
    ↓
Domain Extraction
    ↓
Domain-Check API
    ↓
Risk Scoring
    ↓
Output (JSON + Dashboard)
```

---

## 🛠 STACK TEHNOLOGIC

### Backend
- **Framework:** Flask 3.0+ (Python 3.10+)
- **API Documentation:** Flask-RESTX (Swagger UI + ReDoc)
- **ORM:** SQLAlchemy 2.0+
- **Migration:** Alembic
- **Validation:** Marshmallow
- **Task Queue:** Celery (optional pentru batch processing)

### Database & Cache
- **Primary DB:** PostgreSQL 15+
- **Cache:** Redis 7+ (caching agresiv)
- **Connection Pooling:** pgBouncer (optional)

### External Services
- **WHOIS/RDAP:**
  - Primary: `whoisit` library (RDAP protocol) - FREE
  - Fallback: Whoxy API (250k requests/lună FREE) - API Key: `876528325417e0bgs418d2cc7d8f193a6`
- **Reputation Check:**
  - VirusTotal API (500 requests/day FREE)
  - openSquat (self-hosted)
  - Custom blacklist checking
- **DNS:** `dnspython` library
- **SSL:** Python `ssl` + `socket` modules

### Dashboard & CLI
- **Dashboard:** Streamlit (MVP) → Flask Admin (production)
- **CLI Tools:** Click framework pentru scripturi dedicate

### DevOps
- **Containerization:** Docker + Docker Compose
- **Network:** `dns-network` (custom bridge)
- **Container Prefix:** `dns_*`
- **Reverse Proxy:** Traefik (optional pentru Kong integration)

---

## 🏗 ARHITECTURA SISTEMULUI

```
┌─────────────────────────────────────────────────────────────┐
│                    EXTERNAL INPUT                            │
│  • REST API Calls                                            │
│  • CLI Scripts                                               │
│  • Dashboard Interface                                       │
└────────────────────────┬────────────────────────────────────┘
                         │
                         ▼
┌─────────────────────────────────────────────────────────────┐
│                  FLASK API GATEWAY                           │
│                  (Port: 5000)                                │
│  ┌───────────────────────────────────────────────────────┐  │
│  │  Routes:                                              │  │
│  │  • /api/v1/check          [POST]   Single domain     │  │
│  │  • /api/v1/check/batch    [POST]   Multiple domains  │  │
│  │  • /api/v1/domain/{id}    [GET]    Domain details    │  │
│  │  • /api/v1/history/{id}   [GET]    Check history     │  │
│  │  • /api/v1/stats          [GET]    Statistics        │  │
│  │  • /docs                  [GET]    Swagger UI        │  │
│  │  • /redoc                 [GET]    ReDoc UI          │  │
│  └───────────────────────────────────────────────────────┘  │
└────────────────────────┬────────────────────────────────────┘
                         │
         ┌───────────────┼───────────────┐
         │               │               │
         ▼               ▼               ▼
┌─────────────┐  ┌──────────────┐  ┌──────────────┐
│   CACHE     │  │   BUSINESS   │  │   EXTERNAL   │
│   LAYER     │  │    LOGIC     │  │   SERVICES   │
│             │  │              │  │              │
│  Redis      │  │  • Domain    │  │  • RDAP      │
│  (Port:     │  │    Extractor │  │  • Whoxy     │
│   6379)     │  │  • WHOIS     │  │  • VT API    │
│             │  │    Fetcher   │  │  • DNS       │
│  TTL:       │  │  • Risk      │  │  • SSL       │
│  • 24h hot  │  │    Scorer    │  │              │
│  • 7d warm  │  │  • Validator │  │              │
└──────┬──────┘  └──────┬───────┘  └──────┬───────┘
       │                │                  │
       └────────────────┼──────────────────┘
                        ▼
           ┌─────────────────────────┐
           │   POSTGRESQL DATABASE   │
           │   (Port: 5432)          │
           │                         │
           │   Tables:               │
           │   • domains             │
           │   • whois_records       │
           │   • dns_records         │
           │   • ssl_certificates    │
           │   • reputation_scores   │
           │   • check_history       │
           │   • blacklists          │
           └─────────────────────────┘
```

---

## 🔌 API ENDPOINTS

### **Base URL:** `http://localhost:5000/api/v1`

### 1. **POST /api/v1/check**
Verifică un singur domeniu.

**Request Body:**
```json
{
  "domain": "example.com",
  "check_options": {
    "whois": true,
    "dns": true,
    "ssl": true,
    "reputation": true,
    "force_refresh": false
  }
}
```

**Response (200 OK):**
```json
{
  "success": true,
  "data": {
    "domain": "example.com",
    "check_id": "uuid-here",
    "timestamp": "2026-01-29T12:00:00Z",
    "whois": {
      "creation_date": "1995-08-14T04:00:00Z",
      "expiration_date": "2027-08-13T04:00:00Z",
      "registrar": "IANA",
      "age_days": 11125,
      "status": ["clientDeleteProhibited", "clientTransferProhibited"]
    },
    "dns": {
      "a_records": ["93.184.216.34"],
      "mx_records": ["0 ."],
      "ns_records": ["a.iana-servers.net", "b.iana-servers.net"],
      "txt_records": ["v=spf1 -all"]
    },
    "ssl": {
      "valid": true,
      "issuer": "DigiCert Inc",
      "expires": "2026-12-25T23:59:59Z",
      "days_until_expiry": 330
    },
    "reputation": {
      "virustotal": {
        "malicious": 0,
        "suspicious": 0,
        "harmless": 85,
        "undetected": 5,
        "last_analysis": "2026-01-28T10:00:00Z"
      },
      "blacklists": [],
      "typosquatting_match": false
    },
    "risk_score": {
      "total": 15,
      "level": "LOW",
      "factors": [
        {"factor": "domain_age", "score": 0, "weight": 0.3, "reason": "Domain is 30+ years old"},
        {"factor": "ssl_valid", "score": 0, "weight": 0.2, "reason": "Valid SSL certificate"},
        {"factor": "reputation", "score": 0, "weight": 0.3, "reason": "No malicious reports"},
        {"factor": "dns_records", "score": 15, "weight": 0.2, "reason": "Complete DNS configuration"}
      ],
      "thresholds": {
        "low": "0-30",
        "medium": "31-60",
        "high": "61-85",
        "critical": "86-100"
      }
    }
  },
  "metadata": {
    "cached": false,
    "processing_time_ms": 1243,
    "api_version": "1.0.0"
  }
}
```

**Response (400 Bad Request):**
```json
{
  "success": false,
  "error": {
    "code": "INVALID_DOMAIN",
    "message": "Domain format is invalid",
    "details": "Domain must match pattern: ^[a-zA-Z0-9][a-zA-Z0-9-]{0,61}[a-zA-Z0-9]?\\.[a-zA-Z]{2,}$"
  }
}
```

---

### 2. **POST /api/v1/check/batch**
Verifică multiple domenii în paralel.

**Request Body:**
```json
{
  "domains": [
    "example.com",
    "google.com",
    "suspicious-news-2026.com"
  ],
  "check_options": {
    "whois": true,
    "dns": true,
    "ssl": false,
    "reputation": true
  },
  "priority": "normal"
}
```

**Response (202 Accepted):**
```json
{
  "success": true,
  "data": {
    "batch_id": "batch-uuid-here",
    "total_domains": 3,
    "status": "processing",
    "estimated_completion": "2026-01-29T12:05:00Z"
  },
  "links": {
    "status": "/api/v1/batch/batch-uuid-here/status",
    "results": "/api/v1/batch/batch-uuid-here/results"
  }
}
```

---

### 3. **GET /api/v1/domain/{domain}**
Recuperează detalii complete pentru un domeniu verificat anterior.

**Path Parameters:**
- `domain` (string): Domain name (ex: `example.com`)

**Query Parameters:**
- `include_history` (boolean, default: false): Include all previous checks

**Response (200 OK):**
```json
{
  "success": true,
  "data": {
    "domain": "example.com",
    "first_checked": "2026-01-15T08:00:00Z",
    "last_checked": "2026-01-29T12:00:00Z",
    "total_checks": 12,
    "current_risk_score": 15,
    "risk_trend": "stable",
    "latest_data": {
      "whois": {...},
      "dns": {...},
      "ssl": {...},
      "reputation": {...}
    },
    "history": [
      {
        "check_id": "uuid-1",
        "timestamp": "2026-01-29T12:00:00Z",
        "risk_score": 15,
        "changes": []
      }
    ]
  }
}
```

---

### 4. **GET /api/v1/history/{domain}**
Istoricul complet de verificări pentru un domeniu.

**Response (200 OK):**
```json
{
  "success": true,
  "data": {
    "domain": "example.com",
    "total_checks": 12,
    "date_range": {
      "from": "2026-01-15T08:00:00Z",
      "to": "2026-01-29T12:00:00Z"
    },
    "checks": [
      {
        "check_id": "uuid-here",
        "timestamp": "2026-01-29T12:00:00Z",
        "risk_score": 15,
        "risk_level": "LOW",
        "changes_detected": false,
        "whois_snapshot": {...},
        "dns_snapshot": {...}
      }
    ]
  },
  "pagination": {
    "page": 1,
    "per_page": 50,
    "total": 12
  }
}
```

---

### 5. **GET /api/v1/stats**
Statistici generale ale sistemului.

**Response (200 OK):**
```json
{
  "success": true,
  "data": {
    "overview": {
      "total_domains_checked": 8456,
      "unique_domains": 3421,
      "checks_today": 234,
      "checks_this_month": 9876
    },
    "risk_distribution": {
      "low": 2845,
      "medium": 412,
      "high": 134,
      "critical": 30
    },
    "top_risky_domains": [
      {
        "domain": "fake-news-urgent.com",
        "risk_score": 92,
        "creation_date": "2026-01-20T00:00:00Z",
        "age_days": 9
      }
    ],
    "cache_performance": {
      "hit_rate": 0.78,
      "total_requests": 12340,
      "cache_hits": 9625,
      "cache_misses": 2715
    },
    "api_usage": {
      "whoxy_calls_today": 145,
      "whoxy_quota_remaining": 249855,
      "virustotal_calls_today": 67,
      "virustotal_quota_remaining": 433
    }
  }
}
```

---

### 6. **GET /api/v1/batch/{batch_id}/status**
Verifică statusul unei operații batch.

**Response (200 OK):**
```json
{
  "success": true,
  "data": {
    "batch_id": "batch-uuid",
    "status": "completed",
    "progress": {
      "total": 100,
      "completed": 100,
      "failed": 2,
      "percentage": 100
    },
    "started_at": "2026-01-29T12:00:00Z",
    "completed_at": "2026-01-29T12:08:32Z",
    "duration_seconds": 512
  }
}
```

---

### 7. **GET /api/v1/search**
Caută domenii în baza de date.

**Query Parameters:**
- `q` (string): Search query
- `risk_level` (enum): LOW, MEDIUM, HIGH, CRITICAL
- `min_age_days` (int): Minimum domain age
- `max_age_days` (int): Maximum domain age
- `registrar` (string): Filter by registrar
- `has_ssl` (boolean): Filter by SSL presence
- `page` (int, default: 1)
- `per_page` (int, default: 50, max: 100)

**Response (200 OK):**
```json
{
  "success": true,
  "data": {
    "results": [
      {
        "domain": "example.com",
        "risk_score": 15,
        "risk_level": "LOW",
        "age_days": 11125,
        "last_checked": "2026-01-29T12:00:00Z"
      }
    ],
    "pagination": {
      "page": 1,
      "per_page": 50,
      "total_results": 234,
      "total_pages": 5
    }
  }
}
```

---

### 8. **POST /api/v1/webhook**
Configurare webhook pentru notificări.

**Request Body:**
```json
{
  "url": "https://your-app.com/webhook/domain-alerts",
  "events": ["high_risk_detected", "domain_age_threshold"],
  "filters": {
    "min_risk_score": 70
  }
}
```

---

## 🗄 DATABASE SCHEMA

### **PostgreSQL Tables**

#### 1. **domains**
```sql
CREATE TABLE domains (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    domain VARCHAR(255) UNIQUE NOT NULL,
    subdomain VARCHAR(255),
    tld VARCHAR(50) NOT NULL,
    first_seen_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
    last_checked_at TIMESTAMP WITH TIME ZONE,
    check_count INTEGER DEFAULT 0,
    is_active BOOLEAN DEFAULT TRUE,
    created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
    updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),

    INDEX idx_domain (domain),
    INDEX idx_last_checked (last_checked_at),
    INDEX idx_tld (tld)
);
```

#### 2. **whois_records**
```sql
CREATE TABLE whois_records (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    domain_id UUID REFERENCES domains(id) ON DELETE CASCADE,
    creation_date TIMESTAMP WITH TIME ZONE,
    expiration_date TIMESTAMP WITH TIME ZONE,
    updated_date TIMESTAMP WITH TIME ZONE,
    registrar VARCHAR(255),
    registrar_url VARCHAR(500),
    registrant_org VARCHAR(255),
    registrant_country VARCHAR(2),
    admin_email VARCHAR(255),
    name_servers TEXT[], -- Array of name servers
    status TEXT[], -- Array of domain statuses
    dnssec BOOLEAN,
    raw_whois_data JSONB,
    data_source VARCHAR(50), -- 'rdap', 'whoxy', 'manual'
    fetched_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),

    INDEX idx_creation_date (creation_date),
    INDEX idx_registrar (registrar),
    INDEX idx_data_source (data_source)
);
```

#### 3. **dns_records**
```sql
CREATE TABLE dns_records (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    domain_id UUID REFERENCES domains(id) ON DELETE CASCADE,
    record_type VARCHAR(10) NOT NULL, -- 'A', 'AAAA', 'MX', 'TXT', 'NS', 'CNAME'
    record_value TEXT NOT NULL,
    ttl INTEGER,
    priority INTEGER, -- For MX records
    fetched_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),

    INDEX idx_domain_type (domain_id, record_type),
    INDEX idx_fetched_at (fetched_at)
);
```

#### 4. **ssl_certificates**
```sql
CREATE TABLE ssl_certificates (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    domain_id UUID REFERENCES domains(id) ON DELETE CASCADE,
    issuer VARCHAR(255),
    subject VARCHAR(255),
    valid_from TIMESTAMP WITH TIME ZONE,
    valid_until TIMESTAMP WITH TIME ZONE,
    serial_number VARCHAR(255),
    signature_algorithm VARCHAR(100),
    key_size INTEGER,
    is_wildcard BOOLEAN DEFAULT FALSE,
    is_self_signed BOOLEAN DEFAULT FALSE,
    certificate_chain JSONB,
    fetched_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),

    INDEX idx_valid_until (valid_until),
    INDEX idx_issuer (issuer)
);
```

#### 5. **reputation_scores**
```sql
CREATE TABLE reputation_scores (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    domain_id UUID REFERENCES domains(id) ON DELETE CASCADE,
    source VARCHAR(50) NOT NULL, -- 'virustotal', 'custom', 'opensquat'
    score INTEGER, -- 0-100
    malicious_count INTEGER DEFAULT 0,
    suspicious_count INTEGER DEFAULT 0,
    harmless_count INTEGER DEFAULT 0,
    undetected_count INTEGER DEFAULT 0,
    is_blacklisted BOOLEAN DEFAULT FALSE,
    blacklist_names TEXT[],
    is_typosquatting BOOLEAN DEFAULT FALSE,
    typosquatting_target VARCHAR(255),
    raw_response JSONB,
    checked_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),

    INDEX idx_source (source),
    INDEX idx_is_blacklisted (is_blacklisted),
    INDEX idx_checked_at (checked_at),
    UNIQUE(domain_id, source, checked_at)
);
```

#### 6. **risk_assessments**
```sql
CREATE TABLE risk_assessments (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    domain_id UUID REFERENCES domains(id) ON DELETE CASCADE,
    check_id UUID UNIQUE NOT NULL,
    total_score INTEGER NOT NULL, -- 0-100
    risk_level VARCHAR(20) NOT NULL, -- 'LOW', 'MEDIUM', 'HIGH', 'CRITICAL'

    -- Individual factor scores
    domain_age_score INTEGER,
    domain_age_days INTEGER,
    ssl_score INTEGER,
    dns_score INTEGER,
    reputation_score INTEGER,
    whois_score INTEGER,

    -- Risk factors breakdown
    factors JSONB, -- Array of {factor, score, weight, reason}

    -- Flags
    is_new_domain BOOLEAN DEFAULT FALSE, -- < 6 months
    is_suspicious BOOLEAN DEFAULT FALSE,
    requires_manual_review BOOLEAN DEFAULT FALSE,

    assessed_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),

    INDEX idx_risk_level (risk_level),
    INDEX idx_total_score (total_score),
    INDEX idx_is_new_domain (is_new_domain),
    INDEX idx_assessed_at (assessed_at)
);
```

#### 7. **check_history**
```sql
CREATE TABLE check_history (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    check_id UUID UNIQUE NOT NULL,
    domain_id UUID REFERENCES domains(id) ON DELETE CASCADE,
    risk_assessment_id UUID REFERENCES risk_assessments(id),

    -- Check metadata
    requested_by VARCHAR(100), -- 'api', 'cli', 'dashboard', 'batch'
    request_ip VARCHAR(45),
    check_options JSONB,

    -- Performance metrics
    processing_time_ms INTEGER,
    cache_hit BOOLEAN DEFAULT FALSE,

    -- Changes detection
    changes_detected BOOLEAN DEFAULT FALSE,
    change_summary JSONB,

    created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),

    INDEX idx_check_id (check_id),
    INDEX idx_domain_created (domain_id, created_at),
    INDEX idx_created_at (created_at)
);
```

#### 8. **batch_operations**
```sql
CREATE TABLE batch_operations (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    batch_id UUID UNIQUE NOT NULL,
    total_domains INTEGER NOT NULL,
    completed_count INTEGER DEFAULT 0,
    failed_count INTEGER DEFAULT 0,
    status VARCHAR(20) DEFAULT 'pending', -- 'pending', 'processing', 'completed', 'failed'
    priority VARCHAR(20) DEFAULT 'normal',
    started_at TIMESTAMP WITH TIME ZONE,
    completed_at TIMESTAMP WITH TIME ZONE,
    error_log JSONB,
    created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),

    INDEX idx_batch_id (batch_id),
    INDEX idx_status (status)
);
```

#### 9. **blacklists**
```sql
CREATE TABLE blacklists (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    domain VARCHAR(255) UNIQUE NOT NULL,
    reason TEXT,
    category VARCHAR(50), -- 'phishing', 'malware', 'spam', 'fake_news'
    source VARCHAR(100),
    added_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
    is_active BOOLEAN DEFAULT TRUE,

    INDEX idx_domain (domain),
    INDEX idx_category (category)
);
```

#### 10. **api_usage_logs**
```sql
CREATE TABLE api_usage_logs (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    service_name VARCHAR(50) NOT NULL, -- 'whoxy', 'virustotal', 'rdap'
    endpoint VARCHAR(255),
    request_count INTEGER DEFAULT 1,
    response_time_ms INTEGER,
    status_code INTEGER,
    quota_used INTEGER,
    quota_remaining INTEGER,
    error_message TEXT,
    logged_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),

    INDEX idx_service_logged (service_name, logged_at),
    INDEX idx_logged_at (logged_at)
);
```

---

## 🎯 RISK SCORING ENGINE

### **Scoring Algorithm**

```python
RISK_SCORE = (
    DOMAIN_AGE_SCORE * 0.30 +
    REPUTATION_SCORE * 0.30 +
    SSL_SCORE * 0.20 +
    DNS_SCORE * 0.10 +
    WHOIS_SCORE * 0.10
)
```

### **1. Domain Age Score (Weight: 30%)**

| Age | Score | Risk Level | Reason |
|-----|-------|------------|--------|
| 0-3 months | 100 | CRITICAL | Very new domain, high phishing risk |
| 3-6 months | 80 | HIGH | New domain, elevated risk |
| 6-12 months | 50 | MEDIUM | Moderately new domain |
| 1-2 years | 30 | LOW | Established domain |
| 2+ years | 0 | LOW | Trusted age |

### **2. Reputation Score (Weight: 30%)**

| Condition | Score | Weight |
|-----------|-------|--------|
| Blacklisted | +100 | Critical |
| VirusTotal malicious > 5 | +80 | High |
| VirusTotal suspicious > 10 | +50 | Medium |
| Typosquatting detected | +70 | High |
| No reputation data | +20 | Low |
| Clean reputation | 0 | None |

### **3. SSL Score (Weight: 20%)**

| Condition | Score |
|-----------|-------|
| No SSL certificate | +100 |
| Self-signed certificate | +80 |
| Expired certificate | +100 |
| Certificate < 30 days old | +40 |
| Certificate expires < 30 days | +30 |
| Valid certificate | 0 |

### **4. DNS Score (Weight: 10%)**

| Condition | Score |
|-----------|-------|
| No MX records | +30 |
| No TXT records (SPF/DKIM) | +20 |
| Suspicious NS records | +40 |
| Recently changed NS | +50 |
| Complete DNS config | 0 |

### **5. WHOIS Score (Weight: 10%)**

| Condition | Score |
|-----------|-------|
| WHOIS privacy protection | +30 |
| Registrant country mismatch | +20 |
| Registrar known for abuse | +50 |
| Recent WHOIS changes | +30 |
| Complete transparent WHOIS | 0 |

### **Final Risk Level Classification**

| Total Score | Risk Level | Action |
|-------------|------------|--------|
| 0-30 | LOW | No action needed |
| 31-60 | MEDIUM | Monitor |
| 61-85 | HIGH | Flag for review |
| 86-100 | CRITICAL | Block/Alert immediately |

### **Special Rules for Fake News Detection**

```python
# CRITICAL FLAG: New domain + breaking news
if domain_age < 180 days AND content_type == "breaking_news":
    risk_level = "CRITICAL"
    score += 50

# HIGH FLAG: Typosquatting + news content
if typosquatting_detected AND content_type == "news":
    risk_level = "HIGH"
    score += 40

# MEDIUM FLAG: No SSL + sensitive topics
if not has_ssl AND topic in ["politics", "health", "finance"]:
    risk_level = "MEDIUM"
    score += 30
```

---

## 💾 CACHING STRATEGY

### **Redis Cache Layers**

#### **Layer 1: Hot Cache (TTL: 6 hours)**
- Recent domain checks (< 6 hours old)
- High-traffic domains
- Pattern: `domain:hot:{domain}:check`

#### **Layer 2: Warm Cache (TTL: 24 hours)**
- Domain WHOIS data
- DNS records
- SSL certificate info
- Pattern: `domain:warm:{domain}:{data_type}`

#### **Layer 3: Cold Cache (TTL: 7 days)**
- Reputation scores (VirusTotal)
- Historical risk scores
- Pattern: `domain:cold:{domain}:reputation`

### **Cache Key Patterns**

```python
# Domain check result
f"dns:check:{domain}:{timestamp}"

# WHOIS data
f"dns:whois:{domain}"

# DNS records
f"dns:records:{domain}:{record_type}"

# SSL certificate
f"dns:ssl:{domain}"

# Reputation score
f"dns:reputation:{domain}:{source}"

# Risk assessment
f"dns:risk:{domain}"

# API quota tracking
f"dns:quota:{service_name}:{date}"
```

### **Cache Invalidation Rules**

1. **Force Refresh:** `check_options.force_refresh = true` bypasses cache
2. **Auto-Invalidate:**
   - WHOIS data: if domain age changes significantly
   - DNS records: if TTL expires
   - SSL: if certificate expiration < 30 days
   - Reputation: if blacklist status changes
3. **Manual Invalidate:** API endpoint for cache clearing

---

## 🌐 EXTERNAL SERVICES

### **1. WHOXY API**

**API Key:** `876528325417e0bgs418d2cc7d8f193a6`
**Quota:** 250,000 requests/month FREE
**Endpoint:** `https://api.whoxy.com/`

**Usage:**
```python
import requests

response = requests.get(
    "https://api.whoxy.com/",
    params={
        "key": "876528325417e0bgs418d2cc7d8f193a6",
        "whois": "example.com"
    }
)
```

**Rate Limiting:**
- Implement exponential backoff
- Track daily usage in `api_usage_logs`
- Alert at 80% quota usage

---

### **2. VirusTotal API**

**Quota:** 500 requests/day FREE
**Endpoint:** `https://www.virustotal.com/api/v3/`

**Priority System:**
```python
# Only use VirusTotal for:
1. New domains (< 6 months)
2. Domains with suspicious patterns
3. High-priority batch requests
4. Manual override requests
```

**Fallback Strategy:**
```python
if virustotal_quota_exceeded:
    # Fallback to local blacklist check
    # Use cached reputation data
    # Assign conservative risk score
```

---

### **3. RDAP (via whoisit library)**

**Free, unlimited, no API key needed**

```python
from whoisit import Domain

domain = Domain("example.com")
print(domain.creation_date)
print(domain.registrar)
```

---

## 🐳 DOCKER SETUP

### **docker-compose.yml**

```yaml
version: '3.9'

networks:
  dns-network:
    driver: bridge
    name: dns-network

services:
  dns_postgres:
    image: postgres:15-alpine
    container_name: dns_postgres
    networks:
      - dns-network
    environment:
      POSTGRES_DB: domain_check
      POSTGRES_USER: dns_admin
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      PGDATA: /var/lib/postgresql/data/pgdata
    volumes:
      - postgres_data:/var/lib/postgresql/data
      - ./init-scripts:/docker-entrypoint-initdb.d
    ports:
      - "5432:5432"
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U dns_admin -d domain_check"]
      interval: 10s
      timeout: 5s
      retries: 5
    restart: unless-stopped

  dns_redis:
    image: redis:7-alpine
    container_name: dns_redis
    networks:
      - dns-network
    command: redis-server --appendonly yes --maxmemory 512mb --maxmemory-policy allkeys-lru
    volumes:
      - redis_data:/data
    ports:
      - "6379:6379"
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 3s
      retries: 5
    restart: unless-stopped

  dns_api:
    build:
      context: ./api
      dockerfile: Dockerfile
    container_name: dns_api
    networks:
      - dns-network
    environment:
      - DATABASE_URL=postgresql://dns_admin:${DB_PASSWORD}@dns_postgres:5432/domain_check
      - REDIS_URL=redis://dns_redis:6379/0
      - WHOXY_API_KEY=${WHOXY_API_KEY}
      - VIRUSTOTAL_API_KEY=${VIRUSTOTAL_API_KEY}
      - FLASK_ENV=development
      - LOG_LEVEL=INFO
    volumes:
      - ./api:/app
      - api_logs:/app/logs
    ports:
      - "5000:5000"
    depends_on:
      dns_postgres:
        condition: service_healthy
      dns_redis:
        condition: service_healthy
    restart: unless-stopped

  dns_dashboard:
    build:
      context: ./dashboard
      dockerfile: Dockerfile
    container_name: dns_dashboard
    networks:
      - dns-network
    environment:
      - API_URL=http://dns_api:5000
    volumes:
      - ./dashboard:/app
    ports:
      - "8501:8501"
    depends_on:
      - dns_api
    restart: unless-stopped

  dns_worker:
    build:
      context: ./api
      dockerfile: Dockerfile
    container_name: dns_worker
    networks:
      - dns-network
    command: celery -A app.celery worker --loglevel=info
    environment:
      - DATABASE_URL=postgresql://dns_admin:${DB_PASSWORD}@dns_postgres:5432/domain_check
      - REDIS_URL=redis://dns_redis:6379/0
      - WHOXY_API_KEY=${WHOXY_API_KEY}
      - VIRUSTOTAL_API_KEY=${VIRUSTOTAL_API_KEY}
    volumes:
      - ./api:/app
    depends_on:
      - dns_api
      - dns_redis
    restart: unless-stopped

volumes:
  postgres_data:
    name: dns_postgres_data
  redis_data:
    name: dns_redis_data
  api_logs:
    name: dns_api_logs
```

### **.env File**

```bash
# Database
DB_PASSWORD=your_secure_password_here

# API Keys
WHOXY_API_KEY=876528325417e0bgs418d2cc7d8f193a6
VIRUSTOTAL_API_KEY=your_vt_api_key_here

# Application
FLASK_ENV=development
LOG_LEVEL=INFO

# Cache
REDIS_TTL_HOT=21600
REDIS_TTL_WARM=86400
REDIS_TTL_COLD=604800
```

---

## 📁 PROJECT STRUCTURE

```
domain-check/
├── api/
│   ├── app/
│   │   ├── __init__.py
│   │   ├── config.py
│   │   ├── models/
│   │   │   ├── __init__.py
│   │   │   ├── domain.py
│   │   │   ├── whois.py
│   │   │   ├── dns.py
│   │   │   ├── ssl.py
│   │   │   ├── reputation.py
│   │   │   └── risk.py
│   │   ├── routes/
│   │   │   ├── __init__.py
│   │   │   ├── check.py
│   │   │   ├── batch.py
│   │   │   ├── domain.py
│   │   │   ├── stats.py
│   │   │   └── search.py
│   │   ├── services/
│   │   │   ├── __init__.py
│   │   │   ├── whois_service.py
│   │   │   ├── dns_service.py
│   │   │   ├── ssl_service.py
│   │   │   ├── reputation_service.py
│   │   │   ├── risk_scorer.py
│   │   │   └── cache_service.py
│   │   ├── utils/
│   │   │   ├── __init__.py
│   │   │   ├── validators.py
│   │   │   ├── extractors.py
│   │   │   └── logger.py
│   │   └── tasks/
│   │       ├── __init__.py
│   │       └── batch_tasks.py
│   ├── migrations/
│   ├── tests/
│   ├── Dockerfile
│   ├── requirements.txt
│   └── run.py
├── dashboard/
│   ├── app.py
│   ├── pages/
│   │   ├── 1_🔍_Domain_Check.py
│   │   ├── 2_📊_Statistics.py
│   │   ├── 3_📜_History.py
│   │   └── 4_⚙️_Settings.py
│   ├── components/
│   ├── Dockerfile
│   └── requirements.txt
├── cli/
│   ├── check_domain.py
│   ├── batch_check.py
│   ├── export_data.py
│   └── requirements.txt
├── init-scripts/
│   └── 01-init-db.sql
├── docker-compose.yml
├── .env
├── .env.example
├── .gitignore
├── README.md
└── API_ARCHITECTURE.md (this file)
```

---

## 🚀 DEPLOYMENT

### **Initial Setup**

```bash
# 1. Clone & setup
git clone <repo> domain-check
cd domain-check

# 2. Configure environment
cp .env.example .env
# Edit .env with your credentials

# 3. Start services
docker-compose up -d

# 4. Run migrations
docker exec -it dns_api flask db upgrade

# 5. Seed initial data
docker exec -it dns_api python scripts/seed_blacklists.py

# 6. Verify health
curl http://localhost:5000/health
curl http://localhost:5000/docs
```

### **Testing**

```bash
# Test single domain
curl -X POST http://localhost:5000/api/v1/check \
  -H "Content-Type: application/json" \
  -d '{"domain": "example.com"}'

# Access dashboard
open http://localhost:8501
```

### **Production Checklist**

- [ ] Change default passwords in `.env`
- [ ] Configure proper logging (Sentry, Datadog)
- [ ] Setup backup strategy for PostgreSQL
- [ ] Configure Kong API Gateway integration
- [ ] Enable HTTPS (Let's Encrypt)
- [ ] Setup monitoring (Prometheus + Grafana)
- [ ] Configure alerting (PagerDuty, Slack)
- [ ] Implement rate limiting
- [ ] Add authentication middleware
- [ ] Security audit (OWASP Top 10)

---

## 📝 API DOCUMENTATION

### **Swagger UI**
Access at: `http://localhost:5000/docs`

### **ReDoc**
Access at: `http://localhost:5000/redoc`

### **Postman Collection**
Available at: `/docs/postman_collection.json`

---

## 🔄 UPDATE HISTORY

| Version | Date | Changes |
|---------|------|---------|
| 1.0.0 | 2026-01-29 | Initial architecture design |

---

**Last Updated:** 2026-01-29
**Maintainer:** Domain-Check Team
**Contact:** [Add contact info]