clossers/didi-lot1-ai

Fork 0

Dezvoltari Evotech 42ff22bf85 LOT 1 - Optimizare script build -Instalare mono comanda

2026-06-27 06:42:02 -07:00

7.4 KiB

Raw Blame History

Audio Transcription

Speech-to-text transcription service using faster-whisper for optimized inference.

Features

High-Performance Transcription: Uses faster-whisper (4x faster than original Whisper)
Multiple Model Sizes: Support for tiny, base, small, medium, large-v2, large-v3, and large-v3-turbo
GPU Acceleration: Optimized CUDA inference with int8 quantization
OpenAI-Compatible API: Drop-in replacement for OpenAI's transcription endpoint
Multiple Output Formats: JSON, text, and verbose JSON with segments
Language Detection: Automatic language detection for 99+ languages
VAD Filtering: Voice Activity Detection for improved accuracy

Prerequisites

Required (all modules):

All global prerequisites (see main README.md)
NVIDIA GPU with CUDA support (recommended for production)
~6GB VRAM for large-v3-turbo with int8 quantization

Optional:

CPU-only mode available (slower but no GPU required)

Note: The audio module runs on GPU 0 (shared with Qwen3.5-35B-A3B).

Installation

cd modules/audio

# Install dependencies
uv sync

# Install with dev dependencies
uv sync --extra dev

Quick Start

As an API Server

cd deploy/

# Create deployment env file
cp ../.env.example .env
# Edit deploy/.env with your configuration

# Start the server
./deploy.sh --profile api --detach

# Or with nginx reverse proxy
./deploy.sh --profile api-nginx --detach

API Endpoints

Endpoint	Method	Description
`/health`	GET	Health check
`/v1/models`	GET	List available models
`/v1/audio/transcriptions`	POST	Transcribe audio (OpenAI-compatible)
`/v1/info`	GET	Service catalog metadata (used by catalog-api)

Example API Request

# Health check
curl http://localhost:54300/health

# Transcribe audio file
AUDIO="/path/to/audio.mp3"
curl -X POST "http://localhost:54300/v1/audio/transcriptions" \
  -F "file=@${AUDIO}" \
  -F "model=large-v3-turbo" \
  -F "response_format=json"

# With language specification and verbose output
curl -X POST "http://localhost:54300/v1/audio/transcriptions" \
  -F "file=@${AUDIO}" \
  -F "language=en" \
  -F "response_format=verbose_json"

Example Response

JSON format:

{
  "text": "This is the full transcription of your audio file.",
  "language": "en",
  "duration": 45.5
}

Verbose JSON format:

{
  "text": "This is the full transcription.",
  "language": "en",
  "duration": 45.5,
  "segments": [
    {
      "id": 0,
      "start": 0.0,
      "end": 3.5,
      "text": "This is the full transcription.",
      "tokens": [123, 456, 789],
      "temperature": 0.0,
      "avg_logprob": -0.25,
      "compression_ratio": 1.5,
      "no_speech_prob": 0.01
    }
  ]
}

Configuration

Required Environment Variables

Configured via environment variables (prefix: AUDIO_). These are typically set in deploy/.env:

Variable	Description
`AUDIO_MODEL`	Whisper model name (e.g., `large-v3-turbo`)
`AUDIO_DEVICE`	Device: `cuda` or `cpu`
`AUDIO_CACHE_DIR`	Model cache directory

Optional Environment Variables

Variable	Default	Description
`AUDIO_COMPUTE_TYPE`	`int8`	Compute type: `float16`, `int8`, `int8_float16`
`AUDIO_BEAM_SIZE`	`5`	Beam size for decoding (1-10)
`AUDIO_BEST_OF`	`5`	Number of candidates when sampling
`AUDIO_TEMPERATURE`	`0.0`	Sampling temperature (0.0-1.0)
`AUDIO_LOG_LEVEL`	`INFO`	Log level
`AUDIO_MAX_FILE_SIZE_MB`	`500`	Max upload file size in MB

Available Models

Model	Parameters	VRAM (int8)	Speed	Quality
`tiny`	39M	~1GB	10x	Basic
`base`	74M	~1GB	7x	Good
`small`	244M	~2GB	4x	Better
`medium`	769M	~3-4GB	2x	Very Good
`large-v3`	1550M	~6-8GB	1x	Excellent
`large-v3-turbo`	809M	~6GB	8x	Excellent

Recommendation: Use large-v3-turbo for best balance of speed and accuracy.

Deployment

cd deploy/

# Create deployment env file (REQUIRED)
cp ../.env.example .env
# Edit deploy/.env with your settings

# API only
./deploy.sh --profile api --detach

# API with nginx reverse proxy
./deploy.sh --profile api-nginx --detach

# View logs
./deploy.sh --profile api --logs

# Stop services
./deploy.sh --profile api --down

Common Docker Commands

cd deploy/

# Restart the API service
docker compose restart audio-api

# Rebuild after code changes
docker compose build audio-api
docker compose up -d audio-api

# Check logs
docker compose logs -f audio-api

Port Allocation

Port	Service
`54300`	Audio API (Dev + AI + Audio) — primary
`8200`	Audio API (legacy default)

Development

# Install dev dependencies
uv sync --extra dev

# Run tests
uv run pytest

# Run tests with coverage
uv run pytest --cov=src/audio --cov-report=term-missing

# Lint and format
uv run ruff check .
uv run ruff format .

Architecture

modules/audio/
├── deploy/
│   ├── deploy.sh             # Deployment script
│   ├── docker-compose.yml    # Docker services
│   ├── Dockerfile            # Container image
│   └── nginx.conf            # Nginx reverse proxy config (optional)
├── src/audio/
│   ├── __init__.py
│   ├── app.py                # FastAPI application
│   ├── transcriber.py        # faster-whisper wrapper
│   ├── schemas.py            # Response schemas
│   └── settings.py           # Configuration
├── tests/
├── .env.example              # Environment template
├── API.md                    # API documentation
├── pyproject.toml            # Dependencies
└── README.md                 # This file

Supported Audio Formats

faster-whisper (via FFmpeg) supports:

MP3, WAV, FLAC, OGG, M4A, AAC, WMA
MP4, AVI, MKV (audio track extraction)
And many more formats supported by FFmpeg

Performance Tips

Use int8 quantization for GPU inference (40% memory savings, minimal accuracy loss)
Use large-v3-turbo for best speed/accuracy tradeoff (8x faster than large-v3)
Enable VAD filtering (enabled by default) to skip silence
Batch processing: Process multiple files sequentially for better GPU utilization
Language specification: Specify language code when known for faster processing

Troubleshooting

Issue: Out of Memory

Solution: Use a smaller model or reduce compute type:

# Use medium model instead
AUDIO_MODEL=medium

# Or use float16 instead of int8 (uses more VRAM but may work better)
AUDIO_COMPUTE_TYPE=float16

Issue: Slow Transcription

Solution:

Ensure GPU is being used (AUDIO_DEVICE=cuda)
Use int8 compute type for faster inference
Use large-v3-turbo instead of large-v3
Check GPU utilization with nvidia-smi

Issue: Model Download Fails

Solution: Check network connectivity and cache directory permissions:

# Verify cache directory exists and is writable
ls -la /cai2_ds_storage/hf_cache

# Or change to local directory
AUDIO_CACHE_DIR=/home/user/.cache/huggingface

7.4 KiB Raw Blame History