Livrare LOT 1 - Didi

This commit is contained in:
Dezvoltari Evotech 2026-06-25 14:13:25 -07:00
commit 5380c3fc63
990 changed files with 133308 additions and 0 deletions

View file

@ -0,0 +1,611 @@
# AUDIT COMPLET — Atomic Knowledge Base
> Audit generat prin analiză paralelă cu 6 agenți specializați.
> Versiune cod auditată: `1.19.2` (clone GitHub `kenforthewin/atomic`)
> Data: 2026-04-10
---
## 0. CE ESTE ATOMIC (TL;DR)
**Atomic** este un **personal knowledge base** care transformă note Markdown ("atoms") într-un **graf semantic AI-augmented**. Fiecare notă este automat:
1. **Chunked** (bucățită markdown-aware)
2. **Embedded** (vectorizată prin LLM)
3. **Tagged** (taguri ierarhice extrase de LLM)
4. **Linked** (edges semantice către alte atoms similare)
Pe baza acestui graf oferă: **căutare semantică**, **wiki articles auto-sintetizate cu citații**, **canvas vizual interactiv**, **chat agentic RAG**, **sincronizare offline**.
Rulează ca:
- **App desktop Tauri** (macOS/Linux/Windows) — sidecar care pornește atomic-server local
- **Server headless Docker/Fly.io** — REST + WebSocket + MCP
- **iOS native SwiftUI** — client thin HTTP
- **Browser extension** (Web Clipper) — capturi web
- **Plugin Obsidian** — sincronizare vault
- **Bot Discord** — capturi mesaje/threads
- **MCP server** — expune knowledge base la Claude Desktop & alte AI tools
---
## 1. ARHITECTURA GENERALĂ — "Core + Thin Wrappers"
```
┌────────────────────┐
│ atomic-core │ ← TOATĂ logica de business
│ (Rust crate, no │ (no actix/no tauri deps)
│ framework deps) │
└─────────┬──────────┘
┌───────────────┼─────────────────┐
▼ ▼ ▼
┌────────────┐ ┌──────────────┐ ┌───────────────┐
│ src-tauri │ │atomic-server │ │ mcp-bridge │
│ (sidecar │ │(REST+WS+MCP) │ │ (stdio→HTTP │
│ launcher) │ │ actix-web │ │ pentru Claude│
└─────┬──────┘ └──────┬───────┘ │ Desktop) │
│ │ └───────┬───────┘
▼ ▼ ▼
┌──────────────────────────────┐ ┌──────────────┐
│ React UI (TypeScript+Vite) │ │ MCP clients │
│ (folosește același │ │ (Claude etc) │
│ HttpTransport în desktop │ └──────────────┘
│ și browser) │
└──────────────────────────────┘
+
┌──────────────┐ ┌─────────────┐ ┌─────────────┐
│ iOS app │ │ Browser ext │ │ Discord bot │
│ SwiftUI │ │ (MV3) │ │ Obsidian │
└──────────────┘ └─────────────┘ └─────────────┘
```
**Principiu cheie:** `atomic-core` este transport-agnostic. Toate evenimentele async sunt livrate prin **callback-uri** (`Fn(EmbeddingEvent)`, `Fn(ChatEvent)`). Fiecare wrapper traduce callback-uri în mecanismul lui (Tauri `app_handle.emit`, sau actix `broadcast::Sender → WebSocket`).
---
## 2. WORKSPACE — STRUCTURA REPO
```
atomic/
├── Cargo.toml # Workspace Rust (5 crate-uri)
├── package.json # Frontend npm (versiune 1.19.2)
├── crates/
│ ├── atomic-core/ # ~16.4k linii Rust — toată logica
│ ├── atomic-server/ # ~7.5k linii — actix-web wrapper
│ ├── mcp-bridge/ # ~285 linii — bridge stdio↔HTTP MCP
│ └── atomic-cloud/ # Control plane SaaS (Stripe + Fly.io)
├── src-tauri/ # Tauri v2 desktop launcher
├── src/ # React 18 + TS + Tailwind v4 + Zustand
├── ios/ # SwiftUI + XcodeGen (Swift 6, iOS 17+)
├── extension/ # Manifest V3 Web Clipper
├── plugins/
│ ├── discord/ # discord.js 14 bot
│ └── obsidian-plugin/ # Obsidian plugin (TS)
├── scripts/ # 15 scripturi Node (build, import, reset)
├── docker/ # nginx, supervisord, litestream
├── docs/ # 5 spec markdown (planuri arhitecturale)
├── docker-compose.{yml,dev,test,build}.yml
├── Dockerfile # Multi-stage: server, web, all-in-one
├── server.dockerfile / web.dockerfile
└── fly.toml.example
```
---
## 3. `atomic-core` — INIMA SISTEMULUI
### 3.1 Module principale
| Modul | Linii | Rol |
|---|---:|---|
| `lib.rs` | 3,399 | Facade `AtomicCore`, orchestrare toate operațiile |
| `db.rs` | 916 | Init SQLite, migrații v0→v10, PRAGMA tuning, schema |
| `manager.rs` | 511 | Multi-database manager (lazy-load instance per DB) |
| `models.rs` | 696 | `Atom`, `Tag`, `WikiArticle`, `ChatMessage`, `SemanticEdge` etc. |
| `registry.rs` | 863 | Multi-database registry (cross-DB settings/tokens) |
| `tokens.rs` | 430 | API tokens (SHA-256 hash + revocable) |
| `settings.rs` | 191 | Key-value settings (provider, modele, thresholds) |
| `executor.rs` | 39 | Background runtime (4 worker threads) + semafoare concurrență |
| `projection.rs` | 287 | PCA/t-SNE pentru canvas (cache poziții) |
| `error.rs` | 70 | `AtomicCoreError` enum |
### 3.2 Pipeline de procesare atom (fire-and-forget)
```
POST /api/atoms (sau core.create_atom)
▼ caller primește atom-ul instant (saved)
│ background, prin executor + callback Fn(EmbeddingEvent):
├─[1]─► CHUNKING (markdown-aware)
│ respectă code blocks, headers, paragraphs
│ token-aware via tiktoken-rs
├─[2]─► EMBEDDING (provider configurat)
│ OpenRouter / Ollama / OpenAI-compat
│ inserează în vec_chunks (sqlite-vec virtual table)
│ → emit EmbeddingEvent::Started/Complete/Failed
├─[3]─► AUTO-TAGGING (LLM structured outputs)
│ categorii: Topics, People, Locations,
│ Organizations, Events
│ → emit TaggingComplete/Failed/Skipped
├─[4]─► SEMANTIC EDGES (cosine similarity)
│ threshold: 0.5 default → tabela semantic_edges
└─[5]─► WIKI INCREMENTAL UPDATE (dacă tagged)
LLM integrează atom-ul nou în wiki article-ul existent
```
**Praguri (thresholds):**
- Similaritate edges/related atoms: **0.5**
- Semantic search & wiki chunk selection: **0.3**
- Formula: `similarity = 1.0 - (distance² / 2.0)` (din Euclidean al sqlite-vec pe vectori normalizați)
### 3.3 Storage abstraction (SQLite + Postgres)
`atomic-core` are un trait `StorageBackend` cu **două implementări**:
- **SQLite** (`storage/sqlite/`) — default, prin `rusqlite 0.32` (bundled) + `sqlite-vec 0.1.6` (vector search)
- **Postgres** (`storage/postgres/`) — feature-gated, prin `sqlx 0.8` + `pgvector 0.4`
Dispatch macro `dispatch!` rulează 111 metode peste backend-uri. Există un plan documentat (`docs/plan-async-migration.md`) pentru a face întregul `AtomicCore` async-native (acum Postgres face sync→async bridge prin `PG_RUNTIME.block_on`).
### 3.4 Schema bază date (data DB-uri)
Tabele cheie:
- `atoms` — content, source_url, embedding_status, tagging_status, created/updated_at
- `atom_chunks` — chunks per atom (cu offset-uri în text)
- `vec_chunks`**virtual table sqlite-vec** (vector index)
- `tags` — ierarhie (parent_id), category
- `atom_tags` — many-to-many
- `semantic_edges` — pereche atom_a, atom_b, similarity
- `atom_clusters` — rezultate clustering
- `atom_positions` — poziții persistate canvas
- `wiki_articles` — content + metadata per tag
- `wiki_proposals` — (M1+) human-in-the-loop updates
- `conversations`, `chat_messages` — chat agentic
- FTS virtual table pentru keyword search
`registry.db` (separat): `settings`, `api_tokens`, `databases` (UUIDs + nume).
### 3.5 AI Provider abstraction
Trait-uri în `providers/traits.rs`:
- `EmbeddingProvider` — generare batch embeddings
- `LlmProvider` — chat completions
- `StreamingLlmProvider` — streaming + tool calling
Implementări:
- `providers/openrouter/` — cloud, OAuth flow, modele separate per capability (embedding/tagging/wiki/chat)
- `providers/ollama/` — local, **auto-discovery modele**
- `providers/openai_compat/` — generic (Azure OpenAI, Groq, Together, etc.)
Selecția runtime: factory întoarce `Arc<dyn Trait>` în funcție de setting `provider_type`.
### 3.6 Wiki synthesis
- **Generation**: LLM produce articol din atomii unui tag, cu citații inline (`[atom_id]`)
- **Centroid mode** (`wiki/centroid.rs`): selecție chunks după centroid embedding al tagului
- **Agentic mode** (`wiki/agentic.rs`): agent care pune query-uri pe baza de cunoștințe
- **Section operations** (`wiki/section_ops.rs`): `WikiSectionOp::{NoChange, AppendToSection, ReplaceSection, InsertSection}` — LLM emite *operații* peste secțiuni, **nu rewrite total** → diff-uri review-abile (M1 wiki proposals)
- **Versioning**: tabela `wiki_versions` păstrează istoric
### 3.7 Chat agentic / RAG
- Conversații pot fi **scoped la tag-uri**
- Agent are tool-uri: search semantic, read atom, citation
- Streaming via `ChatEvent::{Delta, ToolStart, ToolComplete, Complete, CanvasAction, Error}`
- `ChatCanvasAction` permite agentului să declanșeze acțiuni vizuale în UI
### 3.8 Ingestion URL & RSS & Obsidian
- `ingest/fetch.rs` — HTTP GET cu reqwest
- `ingest/extract.rs` — HTML → Markdown via `dom_smoothie 0.15`
- `ingest/obsidian.rs` — vault Obsidian, păstrează `[[wikilinks]]`, extrage tags din foldere + YAML frontmatter
- RSS — `feed-rs 2.3`, polling configurabil per feed (default 60s)
---
## 4. `atomic-server` — REST + WebSocket + MCP
### 4.1 Stack
- **actix-web 4.9** + actix-cors + actix-ws
- **rmcp 0.15** + rmcp-actix-web 0.11 (MCP Streamable HTTP)
- **utoipa 5** + utoipa-scalar (OpenAPI auto-generat la `/api/docs`)
- **clap 4** (CLI cu subcomenzi)
- **tokio broadcast channel** (256 buffer) pentru events
### 4.2 ~78 Endpoints REST (grupate)
| Domeniu | # | Exemple |
|---|---:|---|
| **Atoms** | 10 | `GET/POST/PUT/DELETE /api/atoms`, `/api/atoms/bulk`, `/api/atoms/{id}/embedding-status`, `/api/atoms/by-source-url`, `/api/atoms/sources` |
| **Tags** | 5 | `GET/POST/PUT/DELETE /api/tags`, `/api/tags/{id}/children` |
| **Search** | 2 | `POST /api/search` (modes: keyword/semantic/hybrid), `GET /api/atoms/{id}/similar` |
| **Wiki** | 13 | `/api/wiki/{tag_id}` cu generate, update, versions, suggestions, proposal/{accept,dismiss}, related, links |
| **Embeddings pipeline** | 8 | process-pending, process-tagging, retry/{atom_id}, reembed-all, reset-stuck, status |
| **Canvas** | 5 | positions GET/PUT, atoms-with-embeddings, level, global (PCA proj) |
| **Graph** | 3 | edges, neighborhood/{atom_id}, rebuild-edges |
| **Clustering** | 3 | compute, get clusters, connection-counts |
| **Chat** | 8 | conversations CRUD, scope add/remove tags, send-message |
| **Settings** | 5 | get/set, test-openrouter, test-openai-compat, models, embedding-models |
| **Databases** | 7 | list, create, rename, delete, activate, set-default, stats |
| **Feeds (RSS)** | 6 | list/create/get/update/delete, poll |
| **Ingest/Import** | 3 | ingest URL (single/batch), import Obsidian vault |
| **Ollama** | 5 | test, models (all/embedding/llm), provider verify |
| **Auth tokens** | 3 | create, list, revoke |
| **Setup** | 2 | status, claim instance |
| **OAuth 2.0 (MCP)** | 7 | `.well-known/*`, `/oauth/{register,authorize,token}` cu PKCE + DCR |
| **Utils & logs** | 3 | sqlite-vec check, compact-tags, logs export |
| **Public** | 2 | `/health`, `/api/docs/openapi.json` |
### 4.3 WebSocket
- Endpoint: `GET /ws?token=<api_token>`
- Subscribe la `tokio::sync::broadcast::Sender<ServerEvent>` (buffer 256)
- `ServerEvent` enum (~25 variante) include:
- **Embedding pipeline**: `EmbeddingStarted/Complete/Failed`, `TaggingComplete/Failed/Skipped`, `BatchProgress`
- **Atom lifecycle**: `AtomCreated`
- **Ingestion**: `IngestionFetchStarted/Complete/Failed/Skipped`, `IngestionComplete/Failed`
- **Feeds**: `FeedPollComplete/Failed`
- **Chat streaming**: `ChatStreamDelta`, `ChatToolStart/Complete`, `ChatComplete`, `ChatCanvasAction`, `ChatError`
- **Import**: `ImportProgress`
### 4.4 MCP endpoint `/mcp`
- **Transport**: Streamable HTTP (stateful, SSE keep-alive 30s)
- **Auth**: `McpAuth` middleware → 401 cu `WWW-Authenticate` care indică spre `/.well-known/oauth-protected-resource` (Claude.ai compatible)
- **Multi-DB**: `?db=<uuid>` în query
- **Tools expuse:**
1. `semantic_search(query, limit)` — hybrid search
2. `read_atom(atom_id, limit, offset)` — paginated 500 lines
3. `create_atom(content, source_url)` — broadcastează `AtomCreated`
4. `update_atom(atom_id, content, source_url)`
### 4.5 Auth
- **API tokens**: SHA-256 hash în DB, prefix de 8 chars pentru lookup rapid
- `BearerAuth` middleware pe `/api/*`, `McpAuth` pe `/mcp`
- `last_used_at` updated fire-and-forget
- **OAuth 2.0 + PKCE + DCR** (1139 linii în `routes/oauth.rs`):
- Dynamic Client Registration → genere client_secret (32 bytes random, base64url)
- Authorization endpoint cu consent HTML
- Code → 5 min expiration, hash storage
- Token exchange cu PKCE S256 verification
- Folosit pentru Claude.ai remote MCP
### 4.6 CLI
```bash
atomic-server [--data-dir PATH] serve --port 8080 --bind 127.0.0.1 \
--public-url https://... --storage sqlite|postgres
atomic-server token create --name "..."
atomic-server token list
atomic-server token revoke <id>
```
### 4.7 Startup behavior
1. Init logging (tracing + ring buffer 1000 entries pentru `/api/logs`)
2. `DatabaseManager::new()` (SQLite sau Postgres)
3. Migrate legacy tokens
4. Create broadcast channel(256)
5. **Recovery**: reset atoms blocate în `processing`, process pending embeddings/tagging
6. **Spawn RSS poll loop** (60s tick, all DBs)
7. Bind HTTP server (4 workers), CORS permissive
8. Graceful shutdown → `PRAGMA optimize`
---
## 5. `mcp-bridge` — stdio↔HTTP
Binary mic (~285 linii) care se compilează cross-platform și e embedded ca **sidecar Tauri** + distribuit pentru Claude Desktop.
**Flow:**
1. Citește JSON-RPC din **stdin**
2. POST la `http://127.0.0.1:44380/mcp` cu header `Mcp-Protocol-Version: 2025-03-26`
3. Captează `mcp-session-id` din răspunsul `initialize`, îl folosește pe requesturile următoare
4. Parsează SSE (`text/event-stream`) și emite linii data: ca JSON-RPC pe **stdout**
5. Timeout HTTP: **300s** (operații AI lungi)
Env vars: `ATOMIC_HOST` (127.0.0.1), `ATOMIC_PORT` (44380).
Config Claude Desktop:
```json
{
"mcpServers": {
"atomic": { "url": "http://localhost:44380/mcp" }
}
}
```
---
## 6. `atomic-cloud` — Control Plane SaaS
Aplicație **Actix separată** care gestionează hosting managed pe Fly.io.
**Module:**
- `clients/fly.rs` — Fly Machines API (create_app, allocate_ips, machines, volumes, delete_app)
- `clients/stripe.rs` — Checkout sessions + webhook HMAC verify
- `clients/mailgun.rs` — magic link delivery
- `routes/checkout.rs` — POST /api/checkout creează Stripe session + Fly app
- `routes/webhooks.rs``customer.subscription.{created,updated,deleted}`
- `routes/instances.rs` — start/stop/restart/billing_portal (auth via management_token)
- `routes/admin.rs` — list instances, stats (MRR estimate), rollout image
- `routes/auth.rs` — magic link send/verify
- `jobs.rs`**cleanup background**: după 30 zile cancel → `fly.delete_app()` (machines + volumes + IPs)
**Modele Postgres:** `Customer`, `Subscription`, `Instance`, `Event`.
**Env vars necesare:** `DATABASE_URL`, `STRIPE_SECRET_KEY`, `STRIPE_WEBHOOK_SECRET`, `STRIPE_PRICE_ID`, `FLY_API_TOKEN`, `FLY_ORG`, `FLY_REGION`, `BASE_DOMAIN`, `MAILGUN_API_KEY/DOMAIN/FROM`, `ADMIN_API_KEY`, `PUBLIC_URL`.
---
## 7. `src-tauri` — Desktop wrapper
- **Tauri v2** + plugins: `opener`, `dialog`, `shell`, `fs`
- Pornește `atomic-server` ca **sidecar process** (binar pre-built în `src-tauri/binaries/{target_triple}/`)
- Expune o singură comandă IPC: `get_local_server_config()``{ url, token }`
- Frontend folosește apoi același `HttpTransport` ca în browser → conectează la sidecar
- Pe exit, Tauri kill-uiește sidecar-ul
**Database location** auto:
- macOS: `~/Library/Application Support/com.atomic.app/`
- Linux: `~/.local/share/com.atomic.app/`
- Windows: `%APPDATA%/com.atomic.app/`
---
## 8. Frontend React (`src/`)
### 8.1 Stack
- **React 18** + TypeScript strict
- **Vite 6** + Tailwind CSS v4 (`@tailwindcss/vite` + `@tailwindcss/typography`)
- **Zustand 5** (state management modular)
- **CodeMirror 6** (`@uiw/react-codemirror`, lang-markdown, theme-one-dark)
- **react-markdown** + remark-gfm
- **Sigma.js v3** + `@sigma/edge-curve` + **Graphology** (canvas graph)
- **d3-force v3** (physics simulation)
- **react-zoom-pan-pinch**, `@tanstack/react-virtual`
- **sonner** (toasts), **diff** (diff view), **qrcode**
### 8.2 Stores Zustand (`src/stores/`)
`atoms`, `tags`, `ui`, `settings`, `wiki`, `chat`, `databases`, `canvas`, `embedding-progress`.
`ui` store: tag filter selectat, drawer state, view mode (canvas/grid/list — persistat în localStorage), search query.
### 8.3 Componente principale (`src/components/`)
- `atoms/``AtomCard`, `AtomGrid`, `AtomList`, `AtomViewer` (markdown render + search), `AtomReader`, `RelatedAtoms`, `FilterBar`
- `canvas/``CanvasView`, `SigmaCanvas`, `HierarchicalCanvas`, `LocalGraphView`, `MiniGraphPreview`, `ClusterBubble`, `ClusterVisualization`, `ConnectionLines`, `CanvasControls`, `CanvasBreadcrumb`, `useForceSimulation` hooks
- `wiki/``WikiArticleContent`, `CitationPopover`, proposal diff view
- `chat/``ChatMessage`, conversations list
- `onboarding/` — wizard cu `AIProviderStep`, OpenRouter OAuth callback
- `settings/`, `tags/`, `search/`
### 8.4 Transport abstraction
Interfața `Transport { invoke(name, args), subscribe(event, handler) }`.
Singura implementare: `HttpTransport` (folosită și în desktop, și în browser):
- Mapează nume comandă → HTTP spec (method, path, body/query transform) printr-un **command map**
- WebSocket pentru events
- În Tauri: înainte cheamă `get_local_server_config` via Tauri IPC → primește URL+token sidecar, apoi totul e HTTP
Codul React e **transport-unaware**.
### 8.5 Vite config (`vite.config.ts`)
- Desktop: fără proxy, stub-uri Tauri în `src/lib/stubs/tauri-*.ts`
- Web (`VITE_BUILD_TARGET=web`): proxy `/api`, `/health`, `/ws``http://127.0.0.1:8080`
### 8.6 Design system
Dark theme inspirat Obsidian:
- Backgrounds: `#1e1e1e` / `#252525` / `#2d2d2d`
- Accent purple: `#7c3aed`
- Text: white / `#8c8c8c`
- 3-panel layout: tag tree stânga (fix) | main view (canvas/grid/list) | right drawer (editor/viewer/wiki/chat)
---
## 9. iOS App (`ios/`)
- **SwiftUI** + Swift 6 strict concurrency, iOS 17.0+
- **XcodeGen** (`project.yml``.xcodeproj` regenerat)
- **Bundle**: `com.atomic.mobile` + extension `com.atomic.mobile.share` (Share Extension)
- **App Group**: `group.com.atomic.mobile` (shared credentials)
- **Dependencies** (SPM): `MarkdownUI 2.4`, `Runestone 0.5`, `TreeSitterLanguages`
**Fișiere cheie:**
- `AtomicApp.swift` — entry point, QR scanner setup
- `APIClient.swift``@Observable` HTTP client (Bearer + opțional `X-Atomic-Database`)
- `AtomStore.swift` — state cu DiskCache (atoms, tags) + OfflineQueue
- `Models.swift` — Codable: `Atom`, `AtomSummary`, `SearchResult`, `TagWithCount`, `DatabaseInfo`
- `OfflineQueue.swift``Documents/pending_atoms.json`
- `SharedConfig.swift` — UserDefaults suite pentru extensie
- `Theme.swift` — culori match cu desktop
**Endpoint-uri folosite:** `/api/atoms` (CRUD + sources), `/api/tags`, `/api/search`, `/api/databases/{activate}`.
---
## 10. Browser Extension (`extension/`)
- **Manifest V3** (Chrome/Edge/Brave)
- **Permisiuni**: activeTab, scripting, contextMenus, storage, notifications, alarms
**Structură:**
- `background/service-worker.js` — capture flow, queue offline, sync alarm 30s, badge status
- `content/content-script.js` — extragere via **Readability** (Mozilla) + **Turndown** (HTML→Markdown), suportă full page și selection
- `popup/` — toolbar UI cu Capture Page / Selection / Sync Now
- `options/` — server URL + API token + Test Connection
- `lib/config.js``chrome.storage.local` cu cheia `serverConfig`
**Endpoint-uri:** `POST /api/atoms`, `GET /health`, `GET /api/atoms?limit=1` (test).
**Offline queue** în `chrome.storage.local`, drain la fiecare alarm tick (30s).
---
## 11. Plugins externe
### 11.1 Discord bot (`plugins/discord/`)
- **discord.js 14.16** + **better-sqlite3** + **yaml**
- Slash commands: `/atomic-subscribe`, `/atomic-unsubscribe`, `/atomic-config`, `/atomic-save`, `/atomic-search`, `/atomic-status`
- **Reaction-based capture**: emoji custom (sau fallback Unicode) pe orice mesaj → ingestie
- **Settle window** (default 300s): debounce per channel pentru thread-uri active (resetat la fiecare nou mesaj)
- Suportă **Forum channels** + **Voice/Stage** + **Threads** (cu fetch full history)
- **NormalizedMessage** structure unifică text/forum/voice/dm
- Local SQLite tables: `dedup_index` (guild:channel:message → atom_id), `channel_configs`
- Templates de formatting per tip (single message / thread / forum post)
- Tag resolution cu cache + auto-create pentru taguri ierarhice (`discord/<channel>`)
### 11.2 Obsidian plugin (`plugins/obsidian-plugin/`)
- **TypeScript** + Obsidian API (`requestUrl`)
- Comenzi: `semantic-search`, `sync-current-note`, `sync-vault`, `toggle-auto-sync`, `open-similar-notes`, `open-wiki`
- 2 sidebar views: **SimilarView** (related notes), **WikiView** (browse wiki articles)
- **Sync engine**:
- File watcher cu debounce configurabil (default 2s)
- Hash SHA-256 pe content → skip dacă neschimbat
- Source URL: `obsidian://VaultName/path/to/note.md`
- Handle: create/modify/delete/rename
- State în `plugin-data.json`: `Map<path, {atomId, contentHash, lastSynced}>`
- Settings: server URL, token, vault name, auto-sync, debounce, folder→tags, delete-on-remove, exclude patterns
---
## 12. Scripturi Node (`scripts/`)
| Script | Rol |
|---|---|
| `dev-server.js` | Pornește atomic-server + Vite simultan; opțional `--postgres` (docker pgvector) |
| `build-server.js` | Compile `atomic-server``src-tauri/binaries/{target}/` |
| `build-mcp-bridge.js` | Compile `mcp-bridge` → același folder pentru sidecar Tauri |
| `build-release.js` | Bump versiune + git tag + tauri build cross-platform + GitHub upload |
| `import/obsidian.js` | Import vault Obsidian (folders→tags, YAML frontmatter, dedup, dry-run) |
| `import-rss.js` | Import feed RSS (turndown HTML→MD) |
| `import-wikipedia.js` | Crawl Wikipedia (BFS din 3 domenii: Computing/Philosophy/History), 100ms rate limit |
| `stress-test-wikipedia.js` | Volume test 1000+ articole |
| `stress-test-summaries.js` | Lightweight: Wikipedia summary endpoint, concurrency 10 |
| `reset-database.js` | Drop completă a DB |
| `reset-tags.js` | Reset tags + remark atoms pentru re-tagging |
| `reset-chunks.js` | Șterge chunks/embeddings/edges/positions, păstrează atoms |
| `drop-database.js` | Delete persistent (registry + databases) |
| `open-in-db-browser.sh` | Quick sqlite3 inspection |
---
## 13. Docker & Deployment
### 13.1 Dockerfile multi-stage
1. **planner**`cargo-chef prepare` (cache deps)
2. **rust-builder** — mold linker, `cargo chef cook`, build `atomic-server` cu profile `server`
3. **frontend-builder**`npm ci` + `vite build` web
4. **server** target — `debian:bookworm-slim`, EXPOSE 8080, ENTRYPOINT atomic-server
5. **web** target — `nginx:1.28`, copy dist-web
6. **all-in-one** target — supervisord rulează atomic-server (127.0.0.1:8080) + nginx (8081), VOLUME /data — folosit pentru Fly.io single-machine
### 13.2 docker-compose.yml (production)
Servicii:
- `server` (ghcr.io/kenforthewin/atomic-server)
- `web` (ghcr.io/kenforthewin/atomic-web)
- `proxy` (nginx :8080→80) cu config în `docker/nginx.conf`
- `litestream` (profile backup) — replicare DB → S3 (sync 10s, snapshot 1h)
- volume `atomic-data:/data`
### 13.3 nginx config
- `/api/` → server (proxy_buffering off pentru SSE, read_timeout 300s)
- `/ws` → server (Upgrade headers, 86400s timeout)
- `/.well-known/`, `/oauth/`, `/mcp` → server
- `/` → web frontend
- `/assets/` → cache 1y immutable
- SPA fallback `try_files $uri /index.html`
### 13.4 docker-compose.dev.yml
- `pgvector/pgvector:pg16` pe portul 5434, healthcheck `pg_isready`
- Folosit de `npm run dev:server:pg` și `npm run db:reset:pg`
### 13.5 Fly.io (`fly.toml.example`)
- Target build: `all-in-one`
- Mount volume `atomic_data → /data`
- Internal port 8081 (nginx)
- Auto-stop suspend, min 0 machines
- Health check `/health` 30s
- VM `shared-cpu-1x` 512MB
---
## 14. Documentația din `docs/`
| Fișier | Subiect |
|---|---|
| `foreign-keys.md` | Plan de a activa `PRAGMA foreign_keys` (acum off). 6 probleme + 6 pași: virtual table cleanup, tranzacții, validare tag IDs, stale positions, wiki migration on tag merge. |
| `llm-wiki-gist-analysis.md` | Comparație cu gist-ul Karpathy "LLM Wiki". Top idee nouă: **Lint pass** (flag contradictions, orphan pages) — distinctive feature. |
| `plan-async-migration.md` | Plan în 6 pași pentru a face `AtomicCore` async-native (eliminarea sync→async bridge la Postgres prin `PG_RUNTIME.block_on`). |
| `url-ingestion-improvements.md` | Roadmap inspirat din Obsidian Web Clipper / Defuddle: wire metadata în columns, auto published_at, site-specific extractors (AI chats prioritate), MathML/footnotes/callouts. |
| `wiki-proposal-loop-plan.md` | Spec arhitectural M1+M2+M3 pentru **wiki updates ca propuneri review-abile** în loc de mutații directe. Tabela `wiki_proposals`, `WikiSectionOp`, dirty set, quiet window, supersede budget, daily caps. |
---
## 15. Tech stack — TABEL FINAL
| Layer | Tehnologii |
|---|---|
| **Core (Rust)** | rusqlite 0.32 (bundled) + sqlite-vec 0.1.6, sqlx 0.8 + pgvector 0.4, tokio 1, reqwest 0.12, tiktoken-rs 0.6, pulldown-cmark 0.12, dom_smoothie 0.15, feed-rs 2.3, sha2, uuid, chrono, tracing |
| **Server** | actix-web 4.9, actix-cors, actix-ws, rmcp 0.15 (MCP Streamable HTTP), utoipa 5 + scalar, clap 4 |
| **Desktop** | Tauri v2 + plugins (opener, dialog, shell, fs) |
| **Frontend** | React 18, TypeScript strict, Vite 6, Tailwind v4, Zustand 5, CodeMirror 6, react-markdown + remark-gfm, Sigma.js 3 + Graphology, d3-force 3, react-zoom-pan-pinch, @tanstack/react-virtual, sonner, diff, qrcode |
| **iOS** | SwiftUI, Swift 6, iOS 17+, MarkdownUI, Runestone, TreeSitterLanguages, XcodeGen |
| **Extension** | Manifest V3, Mozilla Readability, Turndown |
| **Discord** | discord.js 14.16, better-sqlite3, yaml |
| **Obsidian** | Obsidian API, Web Crypto SHA-256 |
| **Cloud (SaaS)** | actix-web, sqlx postgres, Stripe API, Fly Machines API, Mailgun, hmac+sha2 (webhook verify) |
| **Containere** | Docker multi-stage (cargo-chef + mold), nginx, supervisord, litestream backup → S3, pgvector/pgvector:pg16 dev |
| **Deploy** | Fly.io single-machine (volume), Docker Compose self-host, GHCR images |
| **AI providers** | OpenRouter (cloud, OAuth), Ollama (local, auto-discover), OpenAI-compatible (Azure/Groq/Together) |
| **MCP** | Streamable HTTP server + stdio bridge pentru Claude Desktop, OAuth 2.0 + PKCE + DCR pentru Claude.ai remote |
---
## 16. Integrări externe (efective)
| Integrare | Cum |
|---|---|
| **OpenRouter** | OAuth callback `public/oauth/openrouter-callback.html`, modele separate per capability, settings persistate |
| **Ollama** | Auto-discovery via `/api/ollama/models` |
| **OpenAI-compatible** | Base URL + API key generic |
| **Stripe** | Cloud only — checkout sessions + webhook signed (HMAC-SHA256) |
| **Fly.io Machines API** | Cloud only — provision/start/stop/destroy machines + volumes + IPs |
| **Mailgun** | Cloud only — magic link auth |
| **Claude Desktop / Claude.ai** | MCP `/mcp` (Streamable HTTP) + `mcp-bridge` (stdio) + OAuth 2.0 PKCE pentru remote |
| **GitHub Releases** | Auto-upload via `build-release.js` |
| **Wikipedia REST API** | Doar import scripts |
| **Obsidian** | Plugin oficial + import script CLI |
| **Discord API** | discord.js gateway + REST |
| **S3 / R2** | Litestream backup opțional |
---
## 17. Tehnical debt & roadmap (din docs)
1. **FK constraints OFF** → plan în `foreign-keys.md` (cleanup virtual tables, tranzacții, validare)
2. **AtomicCore sync→async bridge la Postgres** → plan în `plan-async-migration.md` (6 pași)
3. **Wiki proposals M1/M2/M3** → spec în `wiki-proposal-loop-plan.md` (M1 manual ready, M2 background scheduler planificat)
4. **URL ingestion** → metadata wiring, published_at extraction, site-specific extractors (AI chat transcripts prioritate)
5. **Lint pass** (contradicții/orfani) → idee nouă din analiza gist Karpathy
---
## 18. CONCLUZIE
**Atomic** este o arhitectură excepțional curat designed:
- **Single source of truth** în `atomic-core` (16k+ linii Rust, zero framework deps)
- **Storage abstraction completă** SQLite ↔ Postgres
- **AI provider abstraction** plug-and-play (3 implementări trait-based)
- **Wrappers thin** pentru fiecare transport: Tauri (sidecar IPC), actix-web (REST/WS/MCP), mcp-bridge (stdio)
- **Frontend transport-unaware** (același cod în desktop și browser)
- **Pipeline async fire-and-forget** cu callback eventing → broadcast → WebSocket
- **Multi-database** cu registry separat
- **API token + OAuth 2.0 PKCE + DCR** pentru securitate enterprise
- **6 platforme client**: desktop, web, iOS, Chrome ext, Discord, Obsidian
- **Deployment flexibil**: standalone binary, Docker Compose, Fly.io single-machine, SaaS managed (atomic-cloud)
- **Documentație tehnică matură** (5 spec docs cu planuri executabile)
Este un proiect **production-ready** cu o roadmap clară de tehnical debt și features noi.