Dezvoltari Evotech 42ff22bf85 LOT 1 - Optimizare script build -Instalare mono comanda

2026-06-27 06:42:02 -07:00

16 KiB

Raw Permalink Blame History

DOMAIN-CHECK - Status Actual și Documentație Completă

🗄️ DATABASE - Ce se Salvează Exact

Fluxul Complet de Date

INPUT: {"domain": "example.com"}
    ↓
1. DOMAIN EXTRACTION & PARSING
    ↓
2. WHOIS LOOKUP (python-whois + Whoxy API fallback)
    ↓
3. RISK SCORING (5 factori)
    ↓
4. SAVE TO DATABASE (3 tabele principale)
    ↓
OUTPUT: JSON cu risk score + toate datele

📋 TABELE DATABASE - Date Salvate

1. domains - Informații Domain Principal

Câmpuri salvate:
- id (UUID)
- domain (ex: "example")
- subdomain (ex: "www" sau NULL)
- tld (ex: "com")
- full_domain (ex: "www.example.com")
- first_seen_at (când a fost văzut prima dată)
- last_checked_at (ultima verificare)
- check_count (de câte ori a fost verificat)
- is_active (boolean)
- created_at, updated_at

Exemplu real:

{
  "id": "587b61bf-e6bf-436b-9867-14df7d5a9cfb",
  "domain": "google",
  "subdomain": null,
  "tld": "com",
  "full_domain": "google.com",
  "first_seen_at": "2026-01-29T13:38:27Z",
  "last_checked_at": "2026-01-29T13:38:33Z",
  "check_count": 1,
  "is_active": true
}

2. whois_records - Date WHOIS Complete

Câmpuri salvate:
- id (UUID)
- domain_id (referință la domains)
- creation_date (când a fost creat domeniul)
- expiration_date (când expiră)
- updated_date (ultima actualizare WHOIS)
- registrar (ex: "GoDaddy", "Namecheap")
- registrar_url
- registrant_org (organizația proprietarului)
- registrant_country (țara - cod 2 litere)
- admin_email
- name_servers (array de DNS servers)
- status (array de statusuri domeniu)
- dnssec (boolean)
- raw_whois_data (JSONB - toate datele raw)
- data_source ("whois" sau "whoxy")
- fetched_at (când au fost preluate datele)

Exemplu real pentru un domeniu funcțional:

{
  "domain_id": "587b61bf-e6bf-436b-9867-14df7d5a9cfb",
  "creation_date": "1997-09-15T04:00:00Z",
  "expiration_date": "2028-09-14T04:00:00Z",
  "registrar": "MarkMonitor Inc.",
  "registrant_org": "Google LLC",
  "registrant_country": "US",
  "admin_email": "dns-admin@google.com",
  "name_servers": ["ns1.google.com", "ns2.google.com"],
  "status": ["clientDeleteProhibited", "clientTransferProhibited"],
  "data_source": "whoxy",
  "age_days": 10359
}

3. risk_assessments - Risk Scoring Rezultate

Câmpuri salvate:
- id (UUID)
- domain_id (referință)
- check_id (UUID unic pentru fiecare check)
- total_score (0-100)
- risk_level ("LOW", "MEDIUM", "HIGH", "CRITICAL")
- domain_age_score (scor 0-100)
- domain_age_days (vârsta în zile)
- ssl_score (0-100)
- dns_score (0-100)
- reputation_score (0-100)
- whois_score (0-100)
- factors (JSONB - detalii fiecare factor)
- is_new_domain (boolean - < 6 luni)
- is_suspicious (boolean)
- requires_manual_review (boolean)
- assessed_at (timestamp)

Exemplu real:

{
  "check_id": "76d6eb1a-d74b-4430-8b09-f2474a088699",
  "total_score": 8,
  "risk_level": "LOW",
  "domain_age_score": 20,
  "domain_age_days": null,
  "ssl_score": 0,
  "dns_score": 0,
  "reputation_score": 0,
  "whois_score": 20,
  "is_new_domain": false,
  "is_suspicious": false,
  "factors": [
    {
      "factor": "domain_age",
      "score": 20,
      "weight": 0.3,
      "reason": "No creation date available"
    },
    {
      "factor": "whois",
      "score": 20,
      "weight": 0.1,
      "reason": "No registrant organization"
    }
  ]
}

4. check_history - Istoric Verificări

Câmpuri salvate:
- id (UUID)
- check_id (UUID unic)
- domain_id (referință)
- risk_assessment_id (referință)
- requested_by ("api", "cli", "dashboard")
- request_ip (IP-ul clientului)
- user_agent (browser/client info)
- check_options (JSONB - ce a fost verificat)
- processing_time_ms (cât a durat)
- cache_hit (boolean - a fost în cache?)
- changes_detected (boolean)
- change_summary (JSONB)
- status ("completed", "failed")
- error_message (dacă a fost eroare)
- created_at

Exemplu real:

{
  "check_id": "76d6eb1a-d74b-4430-8b09-f2474a088699",
  "domain_id": "587b61bf-e6bf-436b-9867-14df7d5a9cfb",
  "requested_by": "api",
  "request_ip": "172.26.0.1",
  "user_agent": "curl/7.81.0",
  "check_options": {
    "whois": true,
    "dns": false,
    "ssl": false,
    "reputation": false,
    "force_refresh": false
  },
  "processing_time_ms": 2009,
  "cache_hit": false,
  "status": "completed"
}

🔌 API ENDPOINTS - Ce Funcționează ACUM

✅ 1. Health Check - Verificare Status Sistem

Endpoint: GET /health Funcțional: DA Parametri: Niciunul

Test:

curl http://localhost:5000/health

Răspuns:

{
  "status": "healthy",
  "version": "v1",
  "environment": "development",
  "database": "connected",
  "redis": "disabled"
}

✅ 2. API Root - Informații Generale

Endpoint: GET / Funcțional: DA Parametri: Niciunul

Test:

curl http://localhost:5000/

Răspuns:

{
  "name": "Domain Check API",
  "version": "v1",
  "description": "Anti-Fake News Domain Verification & Risk Scoring API",
  "documentation": {
    "swagger": "/docs",
    "redoc": "/redoc"
  },
  "endpoints": {
    "health": "/health",
    "api": "/api/v1"
  }
}

✅ 3. Domain Check - Verificare Domeniu (PRINCIPAL)

Endpoint: POST /api/v1/check Funcțional: DA (WHOIS + Risk Scoring) Content-Type: application/json

Parametri Request:

{
  "domain": "string (required)",
  "check_options": {
    "whois": boolean (default: true),
    "dns": boolean (default: false) [NOT IMPLEMENTED YET],
    "ssl": boolean (default: false) [NOT IMPLEMENTED YET],
    "reputation": boolean (default: false) [NOT IMPLEMENTED YET],
    "force_refresh": boolean (default: false)
  }
}

Teste Complete:

Test 1 - Domeniu Normal

curl -X POST http://localhost:5000/api/v1/check \
  -H "Content-Type: application/json" \
  -d '{
    "domain": "google.com"
  }'

Test 2 - Cu Toate Opțiunile

curl -X POST http://localhost:5000/api/v1/check \
  -H "Content-Type: application/json" \
  -d '{
    "domain": "example.com",
    "check_options": {
      "whois": true,
      "dns": false,
      "ssl": false,
      "reputation": false,
      "force_refresh": true
    }
  }'

Test 3 - Domeniu Suspect (Nou)

curl -X POST http://localhost:5000/api/v1/check \
  -H "Content-Type: application/json" \
  -d '{
    "domain": "suspicious-news-2026.com"
  }'

Test 4 - Subdomeniu

curl -X POST http://localhost:5000/api/v1/check \
  -H "Content-Type: application/json" \
  -d '{
    "domain": "www.facebook.com"
  }'

Test 5 - Pretty Print (JSON formatat)

curl -X POST http://localhost:5000/api/v1/check \
  -H "Content-Type: application/json" \
  -d '{"domain": "github.com"}' \
  | python3 -m json.tool

Răspuns Complet (Exemplu):

{
  "success": true,
  "data": {
    "domain": "example.com",
    "check_id": "uuid-here",
    "timestamp": "2026-01-29T15:30:00Z",
    "whois": {
      "creation_date": "1995-08-14T04:00:00Z",
      "expiration_date": "2027-08-13T04:00:00Z",
      "registrar": "IANA",
      "age_days": 11125,
      "status": ["clientDeleteProhibited"],
      "name_servers": ["a.iana-servers.net", "b.iana-servers.net"]
    },
    "dns": null,
    "ssl": null,
    "reputation": null,
    "risk_score": {
      "total": 15,
      "level": "LOW",
      "factors": [
        {
          "factor": "domain_age",
          "score": 0,
          "weight": 0.3,
          "reason": "Trusted age (11125 days old, 2+ years)",
          "age_days": 11125
        },
        {
          "factor": "reputation",
          "score": 0,
          "weight": 0.3,
          "reason": "No reputation data"
        },
        {
          "factor": "ssl",
          "score": 0,
          "weight": 0.2,
          "reason": "Valid SSL certificate"
        },
        {
          "factor": "dns",
          "score": 0,
          "weight": 0.1,
          "reason": "Good DNS setup"
        },
        {
          "factor": "whois",
          "score": 0,
          "weight": 0.1,
          "reason": "Transparent WHOIS"
        }
      ],
      "thresholds": {
        "low": "0-30",
        "medium": "31-60",
        "high": "61-85",
        "critical": "86-100"
      }
    }
  },
  "metadata": {
    "cached": false,
    "processing_time_ms": 2009,
    "api_version": "v1"
  }
}

Erori Posibile:

// 400 - Domain invalid
{
  "success": false,
  "error": {
    "code": "INVALID_REQUEST",
    "message": "Domain parameter is required"
  }
}

// 500 - Eroare internă
{
  "success": false,
  "error": {
    "code": "INTERNAL_ERROR",
    "message": "An error occurred while checking the domain: ..."
  }
}

⏸️ 4. Get Domain Details - Detalii Domeniu

Endpoint: GET /api/v1/domain/{domain} Funcțional: NU (stub - returnează 501) Parametri: domain (string în URL)

Test:

curl http://localhost:5000/api/v1/domain/google.com

Răspuns actual:

{
  "message": "Not yet implemented",
  "domain": "google.com"
}

⏸️ 5. Batch Check - Verificare Multiplă

Endpoint: POST /api/v1/check/batch Funcțional: NU (stub - returnează 501)

Test:

curl -X POST http://localhost:5000/api/v1/check/batch \
  -H "Content-Type: application/json" \
  -d '{
    "domains": ["google.com", "facebook.com", "twitter.com"]
  }'

⏸️ 6. Batch Status - Status Verificare Multiplă

Endpoint: GET /api/v1/batch/{batch_id}/status Funcțional: NU (stub)

⏸️ 7. Statistics - Statistici Sistem

Endpoint: GET /api/v1/stats Funcțional: NU (stub)

Test:

curl http://localhost:5000/api/v1/stats

⏸️ 8. Search Domains - Căutare Domenii

Endpoint: GET /api/v1/search Funcțional: NU (stub)

Test:

curl "http://localhost:5000/api/v1/search?q=google&risk_level=LOW"

📚 SWAGGER UI - Documentație Interactivă

URL: http://localhost:5000/docs

Aici poți:

Vedea toate endpoint-urile
Testa direct din browser
Vedea request/response examples
Download OpenAPI spec

🔍 VERIFICARE DATE ÎN DATABASE

Verifică toate domeniile

docker exec dns_postgres psql -U dns_admin -d domain_check \
  -c "SELECT full_domain, check_count, last_checked_at FROM domains ORDER BY last_checked_at DESC LIMIT 10;"

Verifică ultimele verificări

docker exec dns_postgres psql -U dns_admin -d domain_check \
  -c "SELECT d.full_domain, ra.total_score, ra.risk_level, ra.assessed_at 
      FROM risk_assessments ra 
      JOIN domains d ON ra.domain_id = d.id 
      ORDER BY ra.assessed_at DESC LIMIT 10;"

Verifică date WHOIS

docker exec dns_postgres psql -U dns_admin -d domain_check \
  -c "SELECT d.full_domain, w.creation_date, w.registrar, w.data_source 
      FROM whois_records w 
      JOIN domains d ON w.domain_id = d.id 
      ORDER BY w.fetched_at DESC LIMIT 10;"

Export JSON complet

docker exec dns_postgres psql -U dns_admin -d domain_check \
  -c "SELECT row_to_json(d) FROM domains d LIMIT 5;" \
  | python3 -m json.tool

🔧 SERVICII EXTERNE FOLOSITE

1. python-whois Library

Cost: FREE, unlimited
Folosit pentru: WHOIS lookups primary
Limitări: Rate limiting de la registrars individuali
Status: ✅ ACTIV

2. Whoxy API (Fallback)

API Key: 876528325417e0bgs418d2cc7d8f193a6
Cost: FREE - 250,000 requests/lună
Endpoint: https://api.whoxy.com/
Folosit pentru: WHOIS când python-whois eșuează
Status: ✅ CONFIGURAT (nu e folosit încă ca fallback automat)

Test Direct Whoxy:

curl "https://api.whoxy.com/?key=876528325417e0bgs418d2cc7d8f193a6&whois=example.com"

3. VirusTotal API (Pregătit)

API Key: NU este configurat
Cost: 500 requests/day FREE
Status: ⏸️ Pregătit în cod, nefuncțional

📊 RISK SCORING - Algoritmul Exact

Formula Finală:

TOTAL_SCORE = (
    domain_age_score × 0.30 +
    reputation_score × 0.30 +
    ssl_score × 0.20 +
    dns_score × 0.10 +
    whois_score × 0.10
)

1. Domain Age Score (30% weight)

Vârsta Domini	Score	Risk Level	Reason
0-90 zile	100	CRITICAL	Very new domain (<3 months)
91-180 zile	80	HIGH	New domain (<6 months)
181-365 zile	50	MEDIUM	Moderately new
366-730 zile	30	LOW	Established (1-2 years)
731+ zile	0	LOW	Trusted age (2+ years)
No data	20	LOW	No creation date

2. Reputation Score (30% weight)

Condiție	Score	Weight
Blacklisted	+100	Critical
VirusTotal malicious > 5	+80	High
VirusTotal suspicious > 10	+50	Medium
Typosquatting detected	+70	High
No reputation data	+20	Low
Clean reputation	0	None

3. SSL Score (20% weight)

Condiție	Score
No SSL certificate	+100
Self-signed certificate	+80
Expired certificate	+100
Certificate < 30 days old	+40
Expires < 30 days	+30
Valid certificate	0

4. DNS Score (10% weight)

Condiție	Score
No MX records	+30
No TXT records (SPF/DKIM)	+20
Suspicious NS records	+40
Recently changed NS	+50
Complete DNS config	0

5. WHOIS Score (10% weight)

Condiție	Score
WHOIS privacy protection	+30
No registrant org	+20
Country mismatch	+20
Known abusive registrar	+50
Transparent WHOIS	0

Risk Level Classification:

if total_score <= 30: return "LOW"
elif total_score <= 60: return "MEDIUM"
elif total_score <= 85: return "HIGH"
else: return "CRITICAL"

🎯 CE FUNCȚIONEAZĂ vs CE NU

✅ FUNCȚIONAL ACUM:

✅ Health check endpoint
✅ Domain check cu WHOIS
✅ Risk scoring (domain age + whois factors)
✅ Database storage (domains, whois_records, risk_assessments, check_history)
✅ Swagger UI documentation
✅ Docker Compose infrastructure
✅ PostgreSQL cu toate tabelele
✅ Error handling și logging

⏸️ PREGĂTIT DAR NEFUNCȚIONAL:

⏸️ DNS checking
⏸️ SSL certificate validation
⏸️ VirusTotal reputation
⏸️ Redis caching
⏸️ Batch processing
⏸️ Domain history endpoint
⏸️ Search endpoint
⏸️ Statistics endpoint

❌ NEPREGĂTIT:

❌ Frontend dashboard
❌ Email validation
❌ Subdomain enumeration
❌ IP geolocation
❌ Traffic estimation
❌ Webhook notifications

🧪 SCRIPT DE TESTARE COMPLETĂ

Salvează ca test-all.sh:

#!/bin/bash

echo "=== Testing Domain Check API ==="
echo ""

# Test 1: Health
echo "1. Health Check:"
curl -s http://localhost:5000/health | python3 -m json.tool
echo ""

# Test 2: Root
echo "2. API Root:"
curl -s http://localhost:5000/ | python3 -m json.tool
echo ""

# Test 3: Check google.com
echo "3. Check google.com:"
curl -s -X POST http://localhost:5000/api/v1/check \
  -H "Content-Type: application/json" \
  -d '{"domain": "google.com"}' \
  | python3 -m json.tool
echo ""

# Test 4: Check facebook.com
echo "4. Check facebook.com:"
curl -s -X POST http://localhost:5000/api/v1/check \
  -H "Content-Type: application/json" \
  -d '{"domain": "facebook.com"}' \
  | python3 -m json.tool
echo ""

echo "=== Tests Complete ==="

Rulează:

chmod +x test-all.sh
./test-all.sh

Ultima actualizare: 2026-01-29 15:50:00

16 KiB Raw Permalink Blame History Unescape Escape

DOMAIN-CHECK - Status Actual și Documentație Completă

🗄️ DATABASE - Ce se Salvează Exact

Fluxul Complet de Date

📋 TABELE DATABASE - Date Salvate

1. domains - Informații Domain Principal

2. whois_records - Date WHOIS Complete

3. risk_assessments - Risk Scoring Rezultate

4. check_history - Istoric Verificări

🔌 API ENDPOINTS - Ce Funcționează ACUM

✅ 1. Health Check - Verificare Status Sistem

✅ 2. API Root - Informații Generale

✅ 3. Domain Check - Verificare Domeniu (PRINCIPAL)

Test 1 - Domeniu Normal

Test 2 - Cu Toate Opțiunile

Test 3 - Domeniu Suspect (Nou)

Test 4 - Subdomeniu

Test 5 - Pretty Print (JSON formatat)

⏸️ 4. Get Domain Details - Detalii Domeniu

⏸️ 5. Batch Check - Verificare Multiplă

⏸️ 6. Batch Status - Status Verificare Multiplă

⏸️ 7. Statistics - Statistici Sistem

⏸️ 8. Search Domains - Căutare Domenii

📚 SWAGGER UI - Documentație Interactivă

🔍 VERIFICARE DATE ÎN DATABASE

Verifică toate domeniile

Verifică ultimele verificări

Verifică date WHOIS

Export JSON complet

🔧 SERVICII EXTERNE FOLOSITE

1. python-whois Library

2. Whoxy API (Fallback)

3. VirusTotal API (Pregătit)

📊 RISK SCORING - Algoritmul Exact

Formula Finală:

1. Domain Age Score (30% weight)

2. Reputation Score (30% weight)

3. SSL Score (20% weight)

4. DNS Score (10% weight)

5. WHOIS Score (10% weight)

Risk Level Classification:

🎯 CE FUNCȚIONEAZĂ vs CE NU

✅ FUNCȚIONAL ACUM:

⏸️ PREGĂTIT DAR NEFUNCȚIONAL:

❌ NEPREGĂTIT:

🧪 SCRIPT DE TESTARE COMPLETĂ

16 KiB

Raw Permalink Blame History