LLM Guardrails: Безопасность и фильтрация контента

Введение

LLM Guardrails — это набор механизмов, которые обеспечивают безопасное и корректное поведение LLM-систем. Они защищают от токсичного контента, утечек данных, prompt injection и других угроз.

---

Threat Model

Threat Category          | Risk Level | Impact
-------------------------|------------|--------
Prompt Injection         | Critical   | System compromise
Data Leakage             | Critical   | PII exposure
Jailbreaking             | High       | Bypass safety
Toxic Output             | High       | Brand damage
Hallucination Spread     | Medium     | Misinformation
Denial of Service        | Medium     | Resource abuse
Model Extraction         | Medium     | IP theft

---

Input Filtering

Prompt injection detection

import re
from typing import Optional

class PromptInjectionDetector:
    """Detect and prevent prompt injection attacks"""
    
    INJECTION_PATTERNS = [
        # System prompt overrides
        r"(?i)(ignore\s+previous|disregard\s+all|forget\s+that)",
        r"(?i)(you\s+are\s+now|from\s+now\s+on\s+you)",
        r"(?i)(system:\s*|SYSTEM:\s*|<system>\s*)",
        
        # Instruction injection
        r"(?i)(new\s+instruction|new\s+rule|new\s+directive)",
        r"(?i)(do\s+this\s+instead|replace\s+your\s+instructions)",
        
        # Format exploitation
        r"(?i)(</?s(?:ystem|ystem)>|<\!--|-->)",
        r"(?i)(```(?:python|json|yaml)\n)",
        
        # Encoding tricks
        r"(?i)(%69%67%6e%6f%72%65|%75%73%65%72%20%69%6e%73%74%72%75%63%74%69%6f%6e)",
        r"(?i)(\\x[0-9a-f]{2}.*\\x[0-9a-f]{2})",
        
        # Role-playing attacks
        r"(?i