livrare lot 2

2026-07-10 03:39:53 -07:00 · 2026-07-10 03:39:53 -07:00 · 8ecc78e729
commit 8ecc78e729
763 changed files with 164593 additions and 0 deletions
--- a/backend/services/data-layer/didiStorage/README.md
+++ b/backend/services/data-layer/didiStorage/README.md
@ -0,0 +1,253 @@
+# DIDI Storage Service 📦
+
+## Super Simple Start Guide 🚀
+
+### One Command - That's It!
+```bash
+docker compose up -d
+```
+
+**DONE!** Everything is automatically configured! 🎉
+
+## What Just Happened? 🤔
+
+When you ran that one command:
+1. MinIO storage server started
+2. A helper container automatically:
+   - Created 7 buckets for different file types
+   - Set up auto-deletion for old files
+   - Configured versioning for important data
+   - Created access policies
+   - Then exited (this is normal!)
+3. Storage is now ready to use!
+
+## Check If It's Working ✅
+
+```bash
+docker ps
+
+# You should see:
+# didi-storage (healthy) ← This is your storage server
+```
+
+**Note**: You might also see `didi-storage-init (Exited)` - that's the helper that set everything up. It's supposed to exit!
+
+## Access the Web Console 🖥️
+
+1. Open your browser
+2. Go to: **http://localhost:9003**
+3. Login:
+   - Username: `minioadmin`
+   - Password: `minio123`
+4. You'll see all your buckets ready!
+
+## Connection Info for Your Apps 📡
+
+```python
+# Python example
+from minio import Minio
+
+client = Minio(
+    "localhost:9002",        # API port
+    access_key="minioadmin",
+    secret_key="minio123",
+    secure=False
+)
+```
+
+## The 7 Auto-Created Buckets 🗂️
+
+| Bucket Name | What Goes Here | Auto-Delete After |
+|------------|----------------|-------------------|
+| `text-files` | Text documents, CSVs | 30 days |
+| `image-files` | JPG, PNG, GIF | 60 days |
+| `audio-files` | MP3, WAV, M4A | 30 days |
+| `video-files` | MP4, AVI, MOV | 30 days |
+| `document-files` | PDF, Word, Excel | Never |
+| `pipeline-artifacts` | Analysis results | Never (versioned) |
+| `backups` | System backups | Never (versioned) |
+
+## Quick Test - Upload a File 📤
+
+```bash
+# Create a test file
+echo "Hello Storage!" > test.txt
+
+# Upload it (using docker)
+docker exec didi-storage sh -c "echo 'Test' > /tmp/test.txt && mc cp /tmp/test.txt local/text-files/"
+
+# Check it's there
+docker exec didi-storage mc ls local/text-files/
+```
+
+## Common Tasks 🛠️
+
+### Start Storage
+```bash
+docker compose up -d
+# That's it! Everything auto-configures
+```
+
+### Stop Storage
+```bash
+docker compose down
+# Data is preserved
+```
+
+### View Logs
+```bash
+docker compose logs -f didiStorage
+```
+
+### Check Storage Usage
+```bash
+docker exec didi-storage mc du local/
+```
+
+### List All Files
+```bash
+docker exec didi-storage mc ls --recursive local/
+```
+
+### Complete Fresh Start (WARNING: Deletes Everything!)
+```bash
+docker compose down -v
+rm -rf data/ config/
+docker compose up -d
+```
+
+## What's Special About This Setup? ✨
+
+### 1. **Zero Configuration**
+You don't need to:
+- Create buckets manually
+- Set up policies
+- Configure expiry rules
+- Enable versioning
+
+It's ALL done automatically!
+
+### 2. **Smart File Management**
+- Old files auto-delete (saves space)
+- Important files keep versions (never lose data)
+- Each service gets its own bucket
+
+### 3. **Ready for Production**
+- Passwords in .env file (change them!)
+- Resource limits configured
+- Health checks included
+- Logging configured
+
+## Troubleshooting 🔧
+
+### "Port already in use"
+Someone else is using port 9002 or 9003. Fix:
+1. Edit `.env`
+2. Change `MINIO_API_PORT=9004`
+3. Change `MINIO_CONSOLE_PORT=9005`
+4. Run `docker compose up -d`
+
+### "Can't access console"
+1. Make sure you use `http://` not `https://`
+2. Check container is running: `docker ps`
+3. Try: http://localhost:9003
+
+### "Buckets not created"
+Check the init container logs:
+```bash
+docker logs didi-storage-init
+```
+It should show "Initialization Complete!"
+
+### "Storage full"
+Check usage:
+```bash
+docker exec didi-storage mc du local/
+```
+Files auto-delete after their expiry time!
+
+## For Your Services 🔌
+
+### Python Upload Example
+```python
+from minio import Minio
+
+# Connect
+client = Minio("localhost:9002", 
+               access_key="minioadmin",
+               secret_key="minio123", 
+               secure=False)
+
+# Upload image
+client.fput_object("image-files", "photo.jpg", "/path/to/photo.jpg")
+
+# Upload with metadata
+client.fput_object(
+    "document-files", 
+    "report.pdf",
+    "/path/to/report.pdf",
+    metadata={"pipeline": "text-analysis", "user": "john"}
+)
+```
+
+### Node.js Example
+```javascript
+const Minio = require('minio')
+
+const client = new Minio.Client({
+    endPoint: 'localhost',
+    port: 9002,
+    useSSL: false,
+    accessKey: 'minioadmin',
+    secretKey: 'minio123'
+})
+
+// Upload
+client.fPutObject('text-files', 'data.txt', '/path/to/data.txt')
+```
+
+## How DIDI Platform Uses This 📊
+
+```
+User uploads file → Goes to appropriate bucket
+                 ↓
+Pipeline processes it → Results go to pipeline-artifacts
+                      ↓
+After 30-60 days → Media files auto-delete
+                 ↓
+Artifacts & backups → Keep forever with versions
+```
+
+## Security Notes 🔒
+
+**For Production:**
+1. Change `minioadmin` username in .env
+2. Change `minio123` password in .env  
+3. Use HTTPS (put behind nginx)
+4. Restrict network access
+5. Enable encryption
+
+## Part of the Data Layer 🏗️
+
+```
+📁 data-layer/
+  ├── 📁 didiDatabase/  ✅ PostgreSQL
+  ├── 📁 didiCache/     ✅ Redis
+  ├── 📁 didiStorage/   ✅ MinIO (You are here!)
+  └── 📁 didiQueue/     ⏳ RabbitMQ (Coming next!)
+```
+
+## Summary - Why This Rocks 🎸
+
+1. **One Command**: `docker compose up -d`
+2. **Zero Config**: Everything auto-setup
+3. **Smart Storage**: Auto-expiry, versioning
+4. **Production Ready**: Just change passwords
+5. **Developer Friendly**: Web console included
+
+---
+**That's it! Your storage is ready! 📦**
+
+*No complex setup. No manual configuration. Just works!*
+
+*Version: 1.0.0 | MinIO RELEASE.2024-08-29*