vGPU и виртуализация GPU для LLM

Введение: GPU для LLM

Проблема

LLM deployment challenges:
  - GPUs are expensive ($10K-$40K per A100)
  - Single model rarely uses full GPU
  - Multiple teams need GPU access
  - Bursty traffic → wasted capacity
  
Solution: GPU virtualization

Факт: GPU virtualization can improve utilization from 15% to 60%+ (cloud benchmark, 2025).

Что такое vGPU?

Core Concepts

vGPU = virtual GPU

Types:
1. MIG (Multi-Instance GPU) - NVIDIA
2. vGPU - NVIDIA vGPU Manager
3. MDEV - Intel GVT
4. SR-IOV - Hardware virtualization

Key idea:
  One physical GPU → Multiple virtual GPUs
  Each VM gets isolated GPU access

MIG (Multi-Instance GPU)

MIG splits one GPU into isolated slices:

A100-80GB:
  ┌─────────────────────────────────────┐
  │  A100-80GB                           │
  │  ┌────┬────┬────┬────┬────┬────┬──┐ │
  │  │MIG │MIG │MIG │MIG │MIG │MIG │  │ │
  │  │ 0  │ 1  │ 2  │ 3  │ 4  │ 5  │  │ │
  │  └────┴────┴────┴────┴────┴────┴──┘ │
  └─────────────────────────────────────┘

MIG configurations:
  - 7x MIG 1c (1 CPU core, 1/7 GPU)
  - 4x MIG 2c (2 CPU cores, 2/7 GPU)
  - 3x MIG 3c (3 CPU cores, 3/7 GPU)
  - 1x MIG 7c (7 CPU cores, full)

vGPU vs MIG vs SR-IOV

Feature          MIG        vGPU      SR-IOV
─────────────────────────────────────────────────
Isolation        Hardware   Software  Hardware
Live migration   ❌         ✅        ✅
GPU types        A100/H100 All       Specific
Management       nvidia-mgr VMWare   NIC-dependent
Overhead         ~0%        ~5-10%    ~2-5%

NVIDIA MIG Deep Dive

MIG Configuration

# Enable MIG mode
sudo nvidia-smi mig -ci  # Compute Instance mode
sudo nvidia-smi mig -gmi # Global mode

# Create MIG instances
sudo nvidia-smi mig -cci -g 0 -m "1g.5gb"  # 1 GPU, 5GB
sudo nvidia-smi mig -cci -g 0 -m "2g.10gb" # 2 GPU, 10GB
sudo nvidia-smi mig -cci -g 0 -m "3g.20gb" # 3 GPU, 20GB

# List MIG instances
nvidia-smi mig -lgi

# Create GPU instance
nvidia-smi mig -cgi -g 0 -gi 1

# Destroy MIG instances
sudo nvidia-smi mig -cdi
sudo nvidia-smi mig -cgmi

MIG Profiles

Profile: 1g.5gb
  GPU: 1/7
  Memory: 5 GB
  CUDA Cores: ~1000
  Tensor Cores: ~64
  Max Devices: 7 per A100

Profile: 2g.10gb
  GPU: 2/7
  Memory: 10 GB
  CUDA Cores: ~2000
  Tensor Cores: ~128
  Max Devices: 4 per A100

Profile: 3g.20gb
  GPU: 3/7
  Memory: 20 GB
  CUDA Cores: ~3000
  Tensor Cores: ~192
  Max Devices: 3 per A100

Profile: 7g.40gb
  GPU: 7/7
  Memory: 40 GB
  CUDA Cores: ~6912
  Tensor Cores: ~432
  Max Devices: 1 per A100

Profile: 7g.80gb (A100-80GB only)
  GPU: 7/7
  Memory: 80 GB
  CUDA Cores: ~6912
  Tensor Cores: ~432
  Max Devices: 1 per A100

Using MIG with Docker

# Run container on MIG instance
docker run -d \
  --gpus '"device=MIG-GUID-12345"' \
  --name llm-instance \
  nvidia/cuda:12.0-base

# Verify MIG device
docker exec llm-instance nvidia-smi

# Docker Compose with MIG
version: '3.8'
services:
  llm-server:
    image: vllm/vllm-openai
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              device_ids: ["MIG-GUID-12345"]
              capabilities: [gpu]

NVIDIA vGPU Manager

vGPU Manager Architecture

Physical GPU (A100/V100/T4)
        │
        ▼
┌──────────────────┐
│  vGPU Manager    │
│  (Host)          │
├──────────────────┤
│  vGPU Type:      │
│  a100-24q        │  ← 24GB slice
│  a100-48q        │  ← 48GB slice
└──────────────────┘
        │
        ├── VM 1 → vGPU-A (24GB)
        ├── VM 2 → vGPU-B (24GB)
        └── VM 3 → vGPU-C (24GB)

vGPU Types

For A100:
  a100-4q:   4GB VRAM
  a100-8q:   8GB VRAM
  a100-16q:  16GB VRAM
  a100-24q:  24GB VRAM
  a100-40q:  40GB VRAM
  a100-48q:  48GB VRAM

For V100:
  v100-1q:   1GB VRAM
  v100-2q:   2GB VRAM
  v100-4q:   4GB VRAM
  v100-8q:   8GB VRAM
  v100-16q:  16GB VRAM

For T4:
  t4-1q:     1GB VRAM
  t4-2q:     2GB VRAM
  t4-4q:     4GB VRAM
  t4-8q:     8GB VRAM
  t4-16q:    16GB VRAM

Licensing

NVIDIA vGPU requires license server:

License tiers:
  - VirtualPC: Basic desktop vGPU
  - VirtualWorkstation: Professional desktop
  - VirtualXR: Extended reality
  - VirtualAI/Compute: AI/ML workloads

For AI/ML:
  NVIDIA AI Enterprise license required
  Cost: ~$500-1000 per GPU/month

Kubernetes GPU Virtualization

Device Plugin

# Kubernetes GPU device plugin
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: nvidia-device-plugin
spec:
  template:
    spec:
      containers:
        - name: nvidia-plugin
          image: nvcr.io/nvidia/k8s/dcgm-exporter:latest
          env:
            - name: NVIDIA_VISIBLE_DEVICES
              value: "all"
            - name: NVIDIA_DRIVER_CAPABILITIES
              value: "compute,utility"

GPU Sharing with KubeEdge

# GPU resource requests
apiVersion: v1
kind: Pod
metadata:
  name: llm-pod
spec:
  containers:
    - name: llm-container
      image: vllm/vllm-openai
      resources:
        limits:
          nvidia.com/gpu: 1
          # Request specific MIG device
          nvidia.com/mig-device: "MIG-GUID-12345"
        requests:
          nvidia.com/gpu-memory: 20Gi

GPU Monitoring

# Prometheus GPU monitoring
apiVersion: apps/v1
kind: Deployment
metadata:
  name: gpu-monitoring
spec:
  template:
    spec:
      containers:
        - name: dcgm-exporter
          image: nvcr.io/nvidia/k8s/dcgm-exporter:2.3.2-2.6.2-ubuntu20.04
          ports:
            - containerPort: 9400
          env:
            - name: DCGM_EXPORTER_LISTEN
              value: ":9400"

LLM Deployment on vGPU

Model Sizing

Model parameters → VRAM requirements:

Model size (FP16):
  7B:   ~14GB
  13B:  ~26GB
  30B:  ~60GB
  70B:  ~140GB
  405B: ~810GB

With quantization:
  70B (INT4): ~35GB
  70B (INT8): ~70GB

MIG-compatible sizes:
  7B:   MIG 1g.5gb or 2g.10gb ✓
  13B:  MIG 2g.10gb ✓
  30B:  MIG 3g.20gb ✓
  70B:  MIG 7g.40gb or 7g.80gb ✓

Multi-Instance LLM Serving

# Deploy multiple models on MIG instances
import subprocess

def deploy_on_mig(mig_guid, model, port):
    """Deploy LLM on specific MIG instance"""
    cmd = f"""
    docker run -d \\
      --gpus '"device={mig_guid}"' \\
      --name {model}-server \\
      -p {port}:8000 \\
      vllm/vllm-openai \\
      --model {model} \\
      --port 8000
    """
    subprocess.run(cmd, shell=True)

# Deploy 3 models on A100 MIG
deploy_on_mig("MIG-12345", "meta-llama/Llama-3-8B", 8000)
deploy_on_mig("MIG-67890", "mistralai/Mistral-7B", 8001)
deploy_on_mig("MIG-11111", "google/gemma-7b", 8002)

Load Balancing

# Nginx load balancer for multiple LLM instances
upstream llm_backends {
    server 127.0.0.1:8000;  # Llama-3-8B on MIG 1
    server 127.0.0.1:8001;  # Mistral-7B on MIG 2
    server 127.0.0.1:8002;  # Gemma-7B on MIG 3
}

server {
    listen 80;
    
    location /api/llama {
        proxy_pass http://127.0.0.1:8000;
    }
    
    location /api/mistral {
        proxy_pass http://127.0.0.1:8001;
    }
    
    location /api/gemma {
        proxy_pass http://127.0.0.1:8002;
    }
}

Performance Overhead

MIG Performance

Metric              Overhead
─────────────────────────────────────
Compute             ~0% (hardware isolation)
Memory bandwidth    ~0% (hardware isolation)
NVLink              ~0% (per-GPU)
PCIe                ~0% (per-GPU)
Context switching   N/A (static partitioning)

Conclusion: MIG has negligible overhead

vGPU Performance

Metric              Overhead
─────────────────────────────────────
Compute             ~3-5%
Memory bandwidth    ~5-8%
Context switching   ~10-15%
Driver overhead     ~2-3%

Conclusion: vGPU has small but acceptable overhead

Benchmark Results

A100-80GB benchmarks:

Configuration    Llama-70B    Throughput
──────────────────────────────────────────
Physical GPU     100%         100 tok/s
MIG 7g.40gb      98%          98 tok/s
MIG 7g.80gb      99%          99 tok/s
vGPU 48q         92%          92 tok/s

Quality impact:
  All configurations: identical output
  (deterministic generation)

Advanced Topics

GPU Passthrough

# Direct GPU passthrough (no virtualization)
apiVersion: v1
kind: Pod
metadata:
  name: gpu-passthrough
spec:
  containers:
    - name: container
      image: nvidia/cuda:12.0
      resources:
        limits:
          nvidia.com/gpu: 1  # Full GPU
          nvidia.com/gpu-shares: 1

GPU Topology Awareness

# GPU topology matters for multi-GPU
import torch

def get_gpu_topology():
    """Get GPU NVLink topology"""
    for i in range(torch.cuda.device_count()):
        for j in range(i + 1, torch.cuda.device_count()):
            try:
                link = torch.cuda.nvlink_link_type(i, j)
                print(f"GPU {i} ↔ GPU {j}: {link}")
            except:
                print(f"GPU {i} ↔ GPU {j}: PCIe only")

# Use topology-aware scheduling
# Place tensor parallel workers on NVLink-connected GPUs

GPU Overcommitment

Overcommitment ratio:
  Safe: 1.5-2x (for inference)
  Risky: 3x+ (for training)

Strategy:
  - Inference: bursty, can overcommit
  - Training: steady, avoid overcommit
  
Implementation:
  - Use MIG for hard isolation
  - Use vGPU for soft isolation
  - Monitor and adjust dynamically

Заключение

GPU virtualization is essential for cost-effective LLM deployment:

Best practices:

Use MIG for hard isolation and zero overhead
Use vGPU for flexibility and live migration
Monitor GPU utilization and adjust sizing
Consider quantization for smaller MIG slices

Recommendations:

Small models (7B-13B): MIG 1g-2g slices
Medium models (30B): MIG 3g slice
Large models (70B): MIG 7g or full GPU
Multi-model serving: MIG + load balancer

Введение: GPU для LLM

Проблема

Что такое vGPU?

Core Concepts

MIG (Multi-Instance GPU)

vGPU vs MIG vs SR-IOV

NVIDIA MIG Deep Dive

MIG Configuration

MIG Profiles

Using MIG with Docker

NVIDIA vGPU Manager

vGPU Manager Architecture

vGPU Types

Licensing

Kubernetes GPU Virtualization

Device Plugin

GPU Sharing with KubeEdge

GPU Monitoring

LLM Deployment on vGPU

Model Sizing

Multi-Instance LLM Serving

Load Balancing

Performance Overhead

MIG Performance

vGPU Performance

Benchmark Results

Advanced Topics

GPU Passthrough

GPU Topology Awareness

GPU Overcommitment

Заключение

Ресурсы