Prompt DSL：规则组合与模板引擎

# 原始方式：硬编码 Prompt
def generate_code(task: str) -> str:
    prompt = f"""你是一个专业的 Python 程序员。
    请编写一个 {task} 相关的 Python 函数。
    要求：
    1. 代码简洁高效
    2. 包含完整的错误处理
    3. 添加必要的注释
    """
    return openai.ChatCompletion.create(
        model="gpt-4",
        messages=[{"role": "user", "content": prompt}]
    )

┌─────────────────────────────────────────────────────────────┐
│                    Prompt DSL 在 AI 技术栈中的位置              │
├─────────────────────────────────────────────────────────────┤
│  ┌─────────────────────────────────────────────────────┐   │
│  │              AI 应用层 (Application Layer)             │   │
│  │    LangChain / LlamaIndex / AutoGen / CrewAI          │   │
│  └─────────────────────────────────────────────────────┘   │
│                            ↓                                │
│  ┌─────────────────────────────────────────────────────┐   │
│  │           Prompt 管理层 (Prompt Management Layer)      │   │
│  │    Prompt DSL Compiler / Template Engine / Registry   │   │
│  └─────────────────────────────────────────────────────┘   │
│                            ↓                                │
│  ┌─────────────────────────────────────────────────────┐   │
│  │           模型调用层 (Model Interface Layer)           │   │
│  │    OpenAI / Anthropic / Google / Local Models         │   │
│  └─────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────┘

第1节：引言 - 为什么需要 Prompt DSL（已完成）
    ↓
第2节：DSL 设计原则 - 简洁性、一致性、可组合性
    ↓
第3节：模板语法 - 插值、条件、循环、继承
    ↓
第4节：变量系统 - 类型系统、作用域、默认值
    ↓
第5节：规则组合 - 模块化规则库与动态加载
    ↓
第6节：版本管理与 A/B 测试
    ↓
第7节：调试工具 - 预览、Token 估算、输出验证
    ↓
第8节：实践 - 实现一个轻量级 Prompt DSL 解释器
    ↓
第9节：总结与展望

# 反面示例：过度设计的 DSL
prompt = compile("""
    @template(name="code_generator", version="2.1.0")
    @role(system="python_expert")
    @constraint(max_tokens=2000, temperature=0.7)
    @input(type="CodeTask")
    @output(format="markdown", schema="code_block")
    
    Generate code for: ${task_name}
    
    @foreach(item: ${requirements})
        - ${item.description}
    @endfor
    
    @if ${is_critical}
        @include("safety_checklist")
    @endif
""")

# 正面示例：简洁清晰的 DSL
prompt = template("code_generator") {
    role: "python_expert"
    task: ${task_name}
    requirements: ${requirements}
    safety_check: include("safety_checklist") if is_critical
}

正交性良好的 DSL 结构：

┌──────────────────────────────────────────┐
│              Prompt DSL 构造              │
├────────────┬────────────┬────────────────┤
│  模板构造   │  变量系统   │  规则构造       │
│  - 插值    │  - 类型    │  - 条件        │
│  - 条件    │  - 作用域  │  - 循环        │
│  - 循环    │  - 默认值  │  - 继承        │
│  - 继承    │  - 校验    │  - mixin       │
└────────────┴────────────┴────────────────┘
        ↓                ↓              ↓
    独立演进         独立演进         独立演进
        ↓                ↓              ↓
    ┌─────────────────────────────────────┐
    │           组合使用时的语义            │
    │    模板 + 变量 + 规则 = Prompt       │
    └─────────────────────────────────────┘

# 一致性示例：统一的变量引用语法
# 变量引用：${var_name}
# 环境变量：${env:var_name}
# 全局变量：${global:var_name}
# 上下文字段：${context.field_name}

prompt = template("example") {
    # 统一使用 ${} 语法
    user: "Hello ${user_name}, your balance is ${context.account.balance}"
    
    # 条件表达式也使用相同语法
    system: "You are helping ${global:assistant_name}" if ${env:IS_PREMIUM} else "You are helping a standard user"
}

可组合性的层次结构：

Level 0: 原子构造（Atomic Constructs）
    ├── 变量引用 ${x}
    ├── 字符串字面量 "text"
    ├── 数字字面量 42
    └── 布尔字面量 true/false

Level 1: 表达式构造（Expression Constructs）
    ├── 条件表达式 ${x > 0 ? "positive" : "negative"}
    ├── 列表构造 [${item1}, ${item2}]
    └── 对象构造 {key: ${value}}

Level 2: 语句构造（Statement Constructs）
    ├── 变量声明 let name = "value"
    ├── 条件块 if ${condition} { ... }
    ├── 循环块 foreach ${items} { ... }
    └── 规则引用 include("rule_name")

Level 3: 模板构造（Template Constructs）
    ├── 模板定义 template "name" { ... }
    ├── 模板继承 extends "parent_template"
    └── 模板组合 mixin "rule_set"

Level 4: 模块构造（Module Constructs）
    ├── 模块定义 module "name" { ... }
    ├── 模块导入 import "module_path"
    └── 模块导出 export template "name"

P = 模板
R = 规则集
V = 变量绑定
C = 上下文

组合操作 ⊗ 定义为：
P₁ ⊗ P₂ = merge(P₁, P₂)  // 模板合并
R₁ ⊕ R₂ = union(R₁, R₂)  // 规则并集
V ⊗ P = P with V filled   // 变量填充
C ⊧ P = P evaluated in C  // 上下文求值

# Prompt DSL 基础插值示例

# 简单变量插值
template "greeting" {
    "Hello ${name}, welcome to ${service}!"
}

# 表达式插值
template "greeting_expressive" {
    "Hello ${name}, today is ${date.now()}, your balance is ${balance + interest}!"
}

# 嵌套属性访问
template "user_profile" {
    "User: ${user.name}, Email: ${user.contact.email}"
}

# Prompt DSL 过滤器语法示例

template "product_description" {
    """
    Product: ${product.name \| title}
    Price: $${product.price \| currency}
    Description: ${product.desc \| truncate(100) \| capitalize}
    
    Features:
    ${product.features \| map(f => "- " + f) \| join("\n")}
    
    Reviews (${reviews.count} total):
    ${reviews.items \| filter(r => r.rating >= 4) \| map(r => r.text) \| join("\n---\n")}
    """
}

插值作用域解析链：

┌─────────────────────────────────────────────────────────────┐
│                      作用域解析顺序                           │
├─────────────────────────────────────────────────────────────┤
│  1. 本地作用域 (Local Scope)                                 │
│     ├── 模板参数                                             │
│     ├── let 变量                                             │
│     └── 循环变量                                             │
│                              ↓                               │
│  2. 父作用域链 (Parent Scope Chain)                          │
│     ├── 当前模板的父模板参数                                  │
│     ├── 父模板的 let 变量                                    │
│     └── ...（递归向上）                                      │
│                              ↓                               │
│  3. 上下文字典 (Context Dictionary)                          │
│     ├── 全局配置                                             │
│     ├── 用户会话数据                                         │
│     └── 外部数据源                                           │
│                              ↓                               │
│  4. 环境变量 (Environment Variables)                         │
│     └── ${env:VAR_NAME}                                     │
│                              ↓                               │
│  5. 全局对象 (Global Objects)                                │
│     ├── ${global:now}                                       │
│     ├── ${global:version}                                   │
│     └── ${global:random}                                    │
└─────────────────────────────────────────────────────────────┘

# Prompt DSL 条件块示例

template "user_onboarding" {
    system: """
    ${if is_premium}
        You are a premium user. Provide detailed, comprehensive assistance.
        You have access to:
        - Advanced analytics
        - Priority support
        - Custom integrations
    ${else}
        You are a standard user. Provide efficient, friendly assistance.
        Core features are available.
    ${endif}
    """
    
    user: """
    ${if has_existing_data}
        We noticed you have existing data from ${last_login_date}.
        Would you like to:
        1. Import your existing settings
        2. Start fresh with recommended defaults
        3. Merge with new data
    ${else}
        Welcome! Let's set up your account.
        Please provide the following information:
        - Your name
        - Your preferred language
        - Your use case
    ${endif}
    """
}

# 条件表达式：单行，简洁
template "greeting" {
    "Hello ${name}! ${is_vip ? "Thank you for your premium membership." : "Consider upgrading for more features."}"
}

# 条件块：多行，复杂逻辑
template "detailed_response" {
    ${if response_length == "short"}
        "Brief answer: ${answer}"
    ${else if response_length == "medium"}
        """
        Answer: ${answer}
        
        Key points:
        ${key_points \| map(p => "- " + p) \| join("\n")}
        
        Related topics: ${related_topics \| join(", ")}
        """
    ${else}
        """
        Comprehensive Analysis
        ======================
        
        Executive Summary: ${summary}
        
        Detailed Answer: ${answer}
        
        Supporting Evidence:
        ${evidence \| enumerate}
        
        Implications:
        ${implications \| map((i, idx) => (idx+1) + ". " + i) \| join("\n")}
        
        Recommendations:
        ${recommendations \| map((r, idx) => "[" + (idx+1) + "] " + r) \| join("\n")}
        
        References: ${references \| join(", ")}
        """
    ${endif}
}

# Prompt DSL match 表达式
template "error_response" {
    """
    Error Analysis for Code: ${error_code}
    
    ${match error_type
        when "syntax" then
            "A syntax error was detected. Please check your code for:
            - Missing parentheses or brackets
            - Incorrect indentation
            - Typos in keywords"
        when "runtime" then
            "A runtime error occurred at line ${error_line}.
            Error message: ${error_message}
            Stack trace:
            ${stack_trace}"
        when "logical" then
            "A logical error was detected. The code runs but produces incorrect results.
            Expected output: ${expected}
            Actual output: ${actual}"
        when "performance" then
            "A performance issue was identified:
            - Slow operation at ${slow_operation}
            - Estimated complexity: ${complexity}
            Suggestions for optimization: ${suggestions}"
        default
            "An unknown error occurred. Please contact support with error code ${error_code}."
    }
    """
}

# Prompt DSL foreach 循环示例

template "code_review" {
    system: "You are an expert code reviewer. Provide constructive feedback."
    
    user: """
    Please review the following code changes:
    
    ${foreach file in changed_files}
        File: ${file.path}
        Lines changed: ${file.additions} additions, ${file.deletions} deletions
        
        Diff:
        ${foreach hunk in file.hunks}
            @@ ${hunk.range} @@
            ${foreach line in hunk.lines}
                ${if line.is_addition}+${line.content}${endif}
                ${if line.is_deletion}-${line.content}${endif}
                ${if line.is_context} ${line.content}${endif}
            ${endfor}
        ${endfor}
        
        Review comments:
        ${foreach comment in file.comments}
            [Line ${comment.line}] ${comment.author}: ${comment.text}
        ${endfor}
        
        ---
    ${endfor}
    
    Summary: ${changed_files.length} files changed, ${total_additions} additions, ${total_deletions} deletions
    """
}

# 内置循环变量示例
template "numbered_list" {
    ${foreach item in items}
        ${index1}. ${item.name} ${if last}(Last item)${endif}
          - Index: ${index} (0-based)
          - Position: ${index1} of ${length}
          - ${if odd}Odd${else}Even${endif} iteration
    ${endfor}
}

# Prompt DSL 嵌套循环示例

template "matrix_operation" {
    """
    Matrix Operation: ${operation_type}
    
    Input Matrix A (${rows_A}x${cols_A}):
    ${foreach row_idx in range(rows_A)}
        ${foreach col_idx in range(cols_A)}
            ${A[row_idx][col_idx]} ${if col_idx < cols_A - 1},${endif}
        ${endfor}
        ${if row_idx < rows_A - 1}\n${endif}
    ${endfor}
    
    ${if operation_type == "multiply"}
        Input Matrix B (${rows_B}x${cols_B}):
        ${foreach row_idx in range(rows_B)}
            ${foreach col_idx in range(cols_B)}
                ${B[row_idx][col_idx]} ${if col_idx < cols_B - 1},${endif}
            ${endfor}
            ${if row_idx < rows_B - 1}\n${endif}
        ${endfor}
        
        Result Matrix C (${rows_A}x${cols_B}):
        ${foreach row_idx in range(rows_A)}
            ${foreach col_idx in range(cols_B)}
                ${C[row_idx][col_idx]} ${if col_idx < cols_B - 1},${endif}
            ${endfor}
            ${if row_idx < rows_A - 1}\n${endif}
        ${endfor}
    ${endif}
    """
}

# Prompt DSL 模板继承示例

# 基模板
template "base_conversation" {
    system: """
    ${system_prompt}
    
    Guidelines:
    - Be helpful and professional
    - Follow content policies
    - Admit uncertainty when applicable
    """
    
    user: "${user_message}"
    
    context:
        max_tokens: 2000
        temperature: 0.7
        top_p: 0.9
}

# 继承模板
template "code_generation" extends "base_conversation" {
    system_prompt: """
    You are an expert ${language} programmer.
    Write clean, efficient, well-documented code.
    """
    
    user_message: """
    Task: ${task_description}
    
    Requirements:
    ${foreach req in requirements}
    - ${req}
    ${endfor}
    
    ${if constraints}
    Constraints:
    ${foreach c in constraints}
    - ${c}
    ${endfor}
    ${endif}
    """
    
    # 覆盖父模板的 context
    context:
        max_tokens: 4000
        temperature: 0.5
        top_p: 0.95
}

# 再次继承
template "python_code_generation" extends "code_generation" {
    system_prompt: """
    You are a Python expert specializing in:
    - Clean code with type hints
    - PEP 8 compliance
    - Modern Python idioms (3.10+)
    """
    
    language: "Python"
}

# Prompt DSL mixin 示例

# 定义可复用的 mixin
mixin "safety_guidelines" {
    system_addition: """
    IMPORTANT SAFETY GUIDELINES:
    - Never suggest actions that could cause harm
    - Highlight potential security risks
    - Recommend consulting experts for dangerous operations
    """
}

mixin "formatting_code" {
    code_style:
        indent: "    "  // 4 spaces
        line_length: 100
        include_comments: true
}

mixin "error_handling" {
    system_addition: """
    Error Handling Requirements:
    - Always validate inputs
    - Provide meaningful error messages
    - Suggest recovery strategies
    """
}

# 使用 mixin
template "safe_code_generator" {
    role: "code_expert"
    
    // 组合多个 mixin
    mixins:
        - safety_guidelines
        - formatting_code
        - error_handling
    
    task: ${task_description}
}

# 带参数和条件的 mixin
mixin "conditional_format" {
    ${if format == "markdown"}
        output_format: "markdown"
        code_block: "```${language}\n${code}\n```"
    ${else if format == "html"}
        output_format: "html"
        code_block: "<pre><code class=\"${language}\">${code}</code></pre>"
    ${else}
        output_format: "plain"
        code_block: code
    ${endif}
}

# 继承 vs 组合的实际对比

# 继承场景：明确的层级关系
template "animal" {
    behavior: "Basic animal behavior"
}

template "dog" extends "animal" {
    behavior: "Bark and fetch"
}

template "guide_dog" extends "dog" {
    behavior: "${super} + Guide humans"
}

# 组合场景：跨维度功能组合
mixin "flying" { behavior_addition: "Can fly" }
mixin "swimming" { behavior_addition: "Can swim" }
mixin "speaking" { behavior_addition: "Can speak" }

template "duck" {
    base_behavior: "Quack"
    mixins: [flying, swimming]
}

template "parrot" {
    base_behavior: "Squawk"
    mixins: [flying, speaking]
}

template "human" {
    base_behavior: "Walk and run"
    mixins: [swimming, speaking]
}

# Prompt DSL 类型系统示例

# 基础类型
type String = "hello" | "world" | ...
type Number = 42 | 3.14 | ...
type Boolean = true | false
type Null = null

# 复合类型
type Array<T> = [T, T, ...]
type Object = { key: value, ... }
type Union = String | Number
type Optional = T | null

# 特殊类型
type Regex = /pattern/flags
type Template = template "name" { ... }
type Rule = rule "name" { ... }
type Any = any

# 示例类型定义
type User = {
    id: String,
    name: String,
    email: String,
    role: "admin" | "user" | "guest",
    preferences: {
        theme: "light" | "dark",
        language: String,
        notifications: Boolean
    } | null
}

type CodeTask = {
    description: String,
    language: String,
    requirements: [String],
    constraints: [String] | null,
    test_cases: [{
        input: String,
        expected_output: String,
        is_hidden: Boolean
    }]
}

# Prompt DSL 类型推导示例

// 手动类型标注
template "manual_typed" {
    let name: String = "Alice"
    let age: Number = 30
    let user: User = { id: "123", name: "Alice", email: "alice@example.com", role: "user" }
}

// 自动类型推导
template "auto_typed" {
    let name = "Alice"          // 推导为 String
    let age = 30                // 推导为 Number
    let is_active = true        // 推导为 Boolean
    let items = ["a", "b"]      // 推导为 Array<String>
    let config = {             // 推导为 Object
        timeout: 5000,
        retries: 3
    }
    
    // 类型会在编译时检查
    name = 123   // ❌ 错误：类型不匹配
}

# Prompt DSL 作用域示例

// 全局作用域（Global Scope）
// 定义在整个 DSL 文件顶部，所有模板可见
global {
    app_name: "My Application"
    version: "2.0.0"
    max_retries: 3
}

// 模块作用域（Module Scope）
module "auth" {
    let session_timeout = 3600    // 模块内可见
    let refresh_token_expiry = 604800
    
    template "login" {
        // 模板内局部作用域
        let credentials = {
            username: ${input.username},
            password: ${input.password}
        }
        
        // 错误：不能在局部作用域访问模块变量
        // let x = session_timeout  // 错误！
        
        system: "Authenticate user ${credentials.username}"
    }
}

// 上下文字典（Context Dictionary）
context {
    current_user: User,
    request_id: String,
    timestamp: Number,
    environment: "development" | "staging" | "production"
}

template "greeting" {
    // 上下文变量通过 ${} 语法访问
    "Hello ${current_user.name}, running in ${environment} mode"
    
    // 可以在模板级别覆盖上下文变量
    context:
        environment: "production"  // 覆盖
}

# 作用域规则
template "scope_demo" {
    let x = 1
    
    {
        // 块级作用域
        let x = 2    // 遮蔽外层 x
        let y = 3
        
        // 内层可以访问外层
        "x = ${x}, y = ${y}"  // x=2, y=3
    }
    
    // 外层无法访问内层变量
    // "y = ${y}"  // 错误！y 未定义
    
    // 恢复为外层 x
    "x = ${x}"  // x=1
}

作用域解析优先级（从高到低）：

1. 局部作用域（Local）      ${let x = 1}
   ↓
2. 块级作用域（Block）       ${if true { let y = 2 }}
   ↓
3. 模板参数（Template Args） ${template "t" { arg: ${x} }}
   ↓
4. mixin 作用域（Mixin）     ${mixin "m" { ... }}
   ↓
5. 父模板作用域（Parent）    ${extends "parent"}}
   ↓
6. 上下文作用域（Context）   ${context { ... }}
   ↓
7. 模块作用域（Module）      ${module "m" { ... }}
   ↓
8. 全局作用域（Global）      ${global { ... }}

# Prompt DSL 默认值示例

# 基本默认值
template "user_profile" {
    // 带默认值的参数
    param name: String = "Anonymous"
    param age: Number = 0
    param email: String? = null  // 可选，默认为 null
    param theme: "light" | "dark" = "light"
    param notifications: Boolean = true
    
    // 参数解构默认值
    param settings: {
        timeout: Number = 5000,
        retries: Number = 3,
        debug: Boolean = false
    } = {
        timeout: 5000,
        retries: 3,
        debug: false
    }
}

// 条件默认值
template "response" {
    param tone: "formal" | "casual" | "friendly" = "friendly"
    param detail_level: "brief" | "standard" | "detailed" = "standard"
    
    // 基于其他参数的条件默认值
    param response_format: String = tone == "formal" ? "markdown" : "plain"
    param include_examples: Boolean = detail_level == "detailed"
    
    // 依赖上下文的默认值
    param greeting: String = context.user.preferred_greeting ?? "Hello"
}

// 复杂对象默认值
template "api_request" {
    param config: ApiConfig = {
        method: "GET",
        headers: {
            "Content-Type": "application/json",
            "Accept": "application/json"
        },
        timeout: 30000,
        retries: 3
    }
    
    // 允许部分覆盖
    param method_override: String? = null
    param effective_method = method_override ?? config.method
}

# Prompt DSL 变量校验示例

# 内置校验器
template "validated_registration" {
    param email: String {
        // 内置校验规则
        required: true
        pattern: /^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$/
        message: "Invalid email format"
    }
    
    param age: Number {
        required: true
        min: 13
        max: 150
        message: "Age must be between 13 and 150"
    }
    
    param password: String {
        required: true
        min_length: 8
        max_length: 128
        pattern: /^(?=.*[a-z])(?=.*[A-Z])(?=.*\d)(?=.*[@$!%*?&])[A-Za-z\d@$!%*?&]{8,}$/
        message: "Password must contain uppercase, lowercase, digit, and special character"
    }
    
    param username: String {
        required: true
        min_length: 3
        max_length: 20
        pattern: /^[a-zA-Z0-9_]+$/
        message: "Username must be alphanumeric with underscores"
    }
}

// 自定义校验器
validator "strong_password" {
    params: [password: String]
    
    rules:
        - length(password) >= 8: "Password too short"
        - length(password) <= 128: "Password too long"
        - contains(password, uppercase): "Missing uppercase letter"
        - contains(password, lowercase): "Missing lowercase letter"
        - contains(password, digit): "Missing digit"
        - contains(password, special_chars): "Missing special character"
        - not contains(password, common_passwords): "Password is too common"
}

validator "valid_date_range" {
    params: [start: Date, end: Date]
    
    rules:
        - start < end: "Start date must be before end date"
        - start > today(): "Start date cannot be in the past"
        - end < add_years(today(), 1): "End date must be within one year"
}

template "scheduled_task" {
    param start_date: Date { validator: valid_date_range }
    param end_date: Date
    
    // 自定义校验消息
    param rate: Number {
        validator: range(0, 100)
        message: "Rate must be between 0 and 100"
    }
}

# Prompt DSL 模块化规则库示例

# 定义规则模块
module "code_generation_rules" {
    // 通用代码生成规则
    rule "general_best_practices" {
        description: "通用代码最佳实践"
        priority: 100
        
        directives:
            - "Use clear, descriptive variable names"
            - "Follow language-specific style guides"
            - "Add appropriate comments for complex logic"
            - "Handle errors gracefully"
            - "Consider performance implications"
    }
    
    // 安全规则
    rule "security_guidelines" {
        description: "安全编码指南"
        priority: 200  // 更高优先级
        
        directives:
            - "Never expose sensitive data in code"
            - "Validate and sanitize all user inputs"
            - "Use parameterized queries for database access"
            - "Implement proper authentication/authorization"
            - "Log security-relevant events"
    }
    
    // 性能规则
    rule "performance_guidelines" {
        description: "性能优化指南"
        priority: 150
        
        directives:
            - "Avoid unnecessary memory allocations"
            - "Use appropriate data structures"
            - "Minimize I/O operations"
            - "Consider caching strategies"
            - "Profile before optimizing"
    }
    
    // 导出规则集
    export rules: [general_best_practices, security_guidelines, performance_guidelines]
}

# 使用规则模块
template "secure_code_generator" {
    // 导入规则模块
    import "code_generation_rules"
    
    // 选择性应用规则
    apply_rules:
        - general_best_practices
        - security_guidelines
        - performance_guidelines
    
    // 自定义规则
    custom_rules:
        rule "project_naming" {
            description: "项目特定的命名规范"
            priority: 120
            directives:
                - "Use 'hos_' prefix for internal utilities"
                - "Follow 'HOS-XXX' ticket reference format"
        }
}

# Prompt DSL 动态加载示例

# 规则注册表
registry "prompt_rules" {
    // 静态注册
    register "security" from "modules/security_rules.prompt"
    register "formatting" from "modules/formatting_rules.prompt"
    register "best_practices" from "modules/best_practices.prompt"
    
    // 动态注册（基于条件）
    ${if environment == "production"}
        register "prod_security" from "modules/prod_security_rules.prompt"
    ${endif}
    
    // 动态注册（基于版本）
    ${if api_version >= 2.0}
        register "v2_rules" from "modules/v2_rules.prompt"
    ${endif}
    
    // 注册版本化规则
    register "code_gen_v1" from "modules/code_gen_v1.prompt" version: "1.0.0"
    register "code_gen_v2" from "modules/code_gen_v2.prompt" version: "2.0.0"
}

// 运行时规则加载
template "dynamic_rule_loader" {
    param rules_to_load: [String]
    
    // 动态加载规则
    dynamic_rules: ${rules_to_load.map(name => registry.get(name))}
    
    // 应用加载的规则
    apply_rules: ${dynamic_rules}
    
    // 规则链式调用
    rule_chain:
        - name: "validate_input"
          filter: ${input => input.is_valid}
          transform: ${input => input.sanitize}
        - name: "apply_context"
          context: ${context}
        - name: "generate_output"
          template: "output_template"
}

// 条件规则加载器
loader "conditional_rule_loader" {
    conditions:
        - if: "user.role == 'admin'"
          then: load_rules(["admin_rules", "audit_rules"])
        - if: "user.role == 'developer'"
          then: load_rules(["dev_rules", "code_style_rules"])
        - if: "project.type == 'security_critical'"
          then: load_rules(["security_rules", "audit_rules"])
          priority: 300  // 高优先级
        default: load_rules(["basic_rules"])
}

# Prompt DSL 规则优先级示例

# 规则优先级定义
rule "base_behavior" {
    priority: 0
    directives: ["Be helpful"]
}

rule "safety_first" {
    priority: 1000  // 高优先级，安全规则
    directives: ["Never suggest harmful actions"]
}

rule "user_preference" {
    priority: 500
    directives: ["Respect user preferences"]
}

// 冲突解决策略
conflict_resolver "priority_based" {
    strategy: "highest_priority_wins"
    // 同优先级规则的处理
    tiebreaker: "last_definition_wins"
}

conflict_resolver "merge_based" {
    strategy: "merge_all"
    // 合并冲突的 directive
    merge_strategy: "append_unique"  // 只添加不重复的
}

conflict_resolver "contextual" {
    strategy: "most_specific_wins"
    specificity_rules:
        - "More specific context overrides general context"
        - "User-level rules override project-level rules"
        - "Task-specific rules override domain rules"
}

// 规则应用示例
template "conflict_resolution_demo" {
    // 规则冲突场景
    rule_a: "Use 2-space indentation"
    rule_b: "Use 4-space indentation"
    
    // 应用冲突解决策略
    conflict_resolver: priority_based
    
    // 明确指定优先级
    priorities:
        rule_a: 100
        rule_b: 50
    
    // 最终应用：rule_a 会胜出
    applied_rule: "Use 2-space indentation"
}

// 规则组合示例
template "combined_rules" {
    // 组合多个规则
    apply:
        - security_rules   // 优先级 200
        - formatting_rules // 优先级 100
        - custom_rules     // 优先级 150
    
    // 最终结果
    result: """
        Applied rules in order:
        1. security_rules (priority: 200) - Applied first
        2. custom_rules (priority: 150) - Merged
        3. formatting_rules (priority: 100) - Merged
        
        Conflicts resolved by priority.
    """
}

# Prompt DSL 规则组合操作符

// 规则 A 和 规则 B 的并集
rule "combined_ab" = rule_a + rule_b

// 规则 A 减去 规则 B 的差集
rule "a_without_b" = rule_a - rule_b

// 规则 A 和 规则 B 的交集
rule "common" = rule_a & rule_b

// 规则 A 和 规则 B 的对称差集
rule "exclusive" = rule_a ^ rule_b

// 实际应用示例
template "rule_operations" {
    // 定义基础规则
    let base_rules = rule "base" {
        directives: ["Be helpful", "Be accurate", "Be timely"]
    }
    
    let security_rules = rule "security" {
        directives: ["Validate input", "Protect privacy", "No harm"]
    }
    
    let ml_rules = rule "ml_focus" {
        directives: ["Explain uncertainty", "Cite sources", "Show confidence"]
    }
    
    // 组合规则
    let combined = base_rules + security_rules + ml_rules
    
    // 派生规则
    let ml_security = ml_rules & security_rules  // 交集
    let public_ml = ml_rules - security_rules     // 差集
    
    // 应用规则
    apply: ${combined}
}

# Prompt DSL 版本管理示例

// 语义化版本 (Semantic Versioning)
template "code_assistant" version "2.1.3" {
    // version format: major.minor.patch
    // major: 破坏性变更
    // minor: 新功能（向后兼容）
    // patch: 补丁（向后兼容）
    
    meta:
        author: "HOS Team"
        created: "2024-01-15"
        modified: "2024-03-20"
        deprecated: null
        replaced_by: null
}

// 版本历史记录
version_history "code_assistant" {
    entries:
        - version: "2.1.3"
          date: "2024-03-20"
          changes: [
            "Added support for Python 3.12 syntax",
            "Fixed bug in type hint generation",
            "Improved error message clarity"
          ]
        
        - version: "2.1.0"
          date: "2024-02-01"
          changes: [
            "Added context-aware code completion",
            "Introduced multi-file editing support"
          ]
          breaking: false
        
        - version: "2.0.0"
          date: "2024-01-01"
          changes: [
            "Complete rewrite of generation engine",
            "Changed from GPT-3.5 to GPT-4",
            "New template syntax"
          ]
          breaking: true
          migration_guide: """
            1. Update all template references to new syntax
            2. Review temperature settings (now default 0.5)
            3. Test all prompts with new version
          """
}

// 版本约束
template "flexible_version" {
    // 支持版本范围
    constraints:
        min_version: "2.0.0"
        max_version: "<3.0.0"
        compatible: ["2.1.x", "2.2.x"]
    
    // 版本降级处理
    fallback:
        if_version: ">=2.0.0 <2.1.0"
        use_template: "legacy_v2"
        notify: "Please upgrade to latest version"
}

// Prompt DSL 版本兼容性示例

// 定义版本兼容性矩阵
compatibility_matrix "prompt_versions" {
    current: "2.1.0"
    
    matrix:
        // 模板版本 -> API 版本
        "1.0.0": ["api_v1", "api_v2"]
        "1.5.0": ["api_v2", "api_v3"]
        "2.0.0": ["api_v3", "api_v4"]
        "2.1.0": ["api_v4"]  // 最新
    
    // 自动转换规则
    migrations:
        from: "1.0.0"
        to: "2.0.0"
        transforms:
            - type: "syntax"
              find: "\${var}"
              replace: "\${context.var}"
            - type: "semantic"
              description: "Context variable access changed"
}

// 渐进式迁移
migration_strategy "gradual_migration" {
    phases:
        - phase: 1
          percentage: 10
          duration: "7 days"
          criteria:
            - error_rate < 1%
            - user_satisfaction > 4.0
        
        - phase: 2
          percentage: 50
          duration: "7 days"
          criteria:
            - error_rate < 0.5%
            - latency_p95 < 500ms
        
        - phase: 3
          percentage: 100
          duration: "3 days"
          criteria:
            - error_rate < 0.1%
    
    // 回滚条件
    rollback:
        trigger: "error_rate > 5%"
        action: "immediately_revert_to_previous_version"
}

// 废弃警告
template "deprecated_warning" {
    meta:
        deprecated: true
        deprecated_since: "2.0.0"
        will_be_removed: "3.0.0"
       替代方案: "migrate_to_new_template"
    
    // 自动添加废弃警告
    prepend: """
        ⚠️ DEPRECATED WARNING
        
        This template is deprecated and will be removed in version 3.0.0.
        Please migrate to the new template.
        
        Migration guide: https://docs.example.com/migration
    """
}

// Prompt DSL A/B 测试框架

// 定义实验
experiment "tone_experiment" {
    name: "Response Tone Optimization"
    hypothesis: "A friendlier tone will increase user satisfaction"
    
    // 变体定义
    variants:
        control:
            name: "Current Professional Tone"
            template: "response_professional"
            weight: 50  // 50% 流量
        
        treatment_a:
            name: "Friendly Tone"
            template: "response_friendly"
            weight: 25  // 25% 流量
        
        treatment_b:
            name: "Concise Tone"
            template: "response_concise"
            weight: 25  // 25% 流量
    
    // 目标指标
    metrics:
        primary:
            name: "user_satisfaction"
            type: "continuous"
            direction: "higher_is_better"
            threshold: 4.0
        
        secondary:
            - name: "response_time"
              type: "duration"
              direction: "lower_is_better"
            - name: "task_completion_rate"
              type: "rate"
              direction: "higher_is_better"
            - name: "follow_up_questions"
              type: "count"
              direction: "lower_is_better"
    
    // 流量分配
    allocation:
        strategy: "random"
        seed: 42  // 可重现性
        user_stickiness: true  // 同一用户始终看到同一变体
    
    // 样本量计算
    power_analysis:
        minimum_detectable_effect: 0.1
        statistical_power: 0.8
        significance_level: 0.05
        estimated_sample_size: 1000
    
    // 持续时间
    duration:
        minimum: "7 days"
        maximum: "30 days"
        stop_early_if: "p_value < 0.01 with 500+ samples"
}

// 统计检验
statistical_test "tone_comparison" {
    test_type: "mann_whitney_u"  // 非参数检验
    alpha: 0.05
    correction: "bonferroni"  // 多重比较校正
    
    results:
        control_vs_treatment_a:
            statistic: 12345
            p_value: 0.023
            significant: true
            effect_size: 0.45  // Cohen's d
        
        control_vs_treatment_b:
            statistic: 11234
            p_value: 0.089
            significant: false
            effect_size: 0.22
    
    conclusion: """
        Treatment A (Friendly Tone) shows statistically significant 
        improvement over control (p=0.023).
        
        Recommendation: Roll out Treatment A to 100% of users.
    """
}

// 动态流量调整
template "adaptive_experiment" {
    experiment "bandit_experiment" {
        strategy: "multi_armed_bandit"
        epsilon: 0.1  // 10% 探索，90% 利用
        
        variants:
            - name: "control"
              current_allocation: 0.5
              success_rate: 0.75
            - name: "treatment"
              current_allocation: 0.5
              success_rate: 0.82
        
        // Thompson Sampling 更新
        update_frequency: "hourly"
        prior: "beta(1, 1)"
    }
}

// Prompt DSL 版本与实验集成示例

// 生产环境配置
production_config {
    // 默认使用稳定版本
    default_template: "code_assistant_v2.1.3"
    
    // 实验覆盖
    experiments:
        - name: "tone_experiment"
          enabled: true
          overrides:
            # 实验中的用户使用实验变体
            user_segment: "all"
        
        - name: "format_experiment"
          enabled: true
          overrides:
            # 仅付费用户参与
            user_segment: "premium"
    
    // 版本回滚配置
    rollback:
        enabled: true
        trigger: "error_rate > 1% OR satisfaction < 3.5"
        target_version: "2.1.2"
        notification: ["pagerduty", "slack:#alerts"]
}

// 灰度发布
template "gradual_rollout" {
    rollout_strategy:
        stages:
            - stage: 1
              name: "Canary"
              percentage: 5
              target: "internal_users"
              duration: "24 hours"
              criteria:
                  - error_rate < 2%
                  - latency_p99 < 2000ms
            
            - stage: 2
              name: "Beta"
              percentage: 20
              target: "beta_users"
              duration: "48 hours"
              criteria:
                  - error_rate < 1%
                  - satisfaction > 4.0
            
            - stage: 3
              name: "General Availability"
              percentage: 100
              target: "all_users"
              duration: "permanent"
              criteria:
                  - all_previous_criteria_passed
        
        auto_rollback:
            enabled: true
            metrics:
                - error_rate > 5%
                - latency_p99 > 5000ms
                - satisfaction < 3.0
}

// Prompt DSL 预览工具示例

// 模板预览命令
preview template "code_assistant" {
    // 提供示例输入
    inputs:
        language: "Python"
        task: "sort a list"
        context:
            user_level: "intermediate"
            project_type: "data_analysis"
    
    // 渲染选项
    options:
        show_variables: true      // 显示变量占位符
        show_line_numbers: true  // 显示行号
        highlight_sections: true  // 高亮不同部分
        token_count: true         // 显示 token 估计
    
    // 输出格式
    output_format: "rich"  // rich | plain | json
}

// 预览输出示例
/*
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Template: code_assistant (v2.1.0)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

[SYSTEM] ┌─────────────────────────────────────────────────────
          │ You are an expert Python programmer specializing in
          │ data analysis.
          │ 
          │ Code Style Requirements:
          │ - Use type hints (PEP 484)
          │ - Add docstrings (Google style)
          │ - Follow PEP 8
          └─────────────────────────────────────────────────────

[USER]   ┌─────────────────────────────────────────────────────
          │ Please write a Python function to sort a list.
          │ 
          │ Requirements:
          │ - Sort a list of numbers
          │ - Return a new list (don't modify original)
          │
          │ User Level: intermediate
          │ Project Type: data_analysis
          └─────────────────────────────────────────────────────

[METADATA]
          │ Token Estimate: ~150
          │ Variables: 3 filled, 0 remaining
          │ Rules Applied: 6
          └─────────────────────────────────────────────────────
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
*/

// Prompt DSL Token 估算器

// Token 估算配置
token_estimator {
    // 模型编码配置
    model: "gpt-4"  // or "gpt-3.5-turbo", "claude-3", etc.
    
    // 编码参数
    encoding: "cl100k_base"  // GPT-4 encoding
    
    // 估算精度
    precision: "token"  // token | character | word
    
    // 缓存估算结果
    cache_enabled: true
    cache_ttl: 3600
}

// Token 估算函数实现
function estimate_tokens(text: String, model: String) -> Number {
    // 使用 tiktoken 估算
    let encoding = get_encoding(model)
    let tokens = encoding.encode(text).length
    return tokens
}

// 模板 Token 预算
template_with_budget "code_generator" {
    meta:
        name: "Code Generator"
        max_tokens: 2000
        budget_warning_threshold: 0.8  // 80% 警告
        budget_error_threshold: 0.95   // 95% 错误
    
    // Token 分配
    allocations:
        system_prompt: 500    // 固定
        user_input: 300      // 固定
        generation: 1200     // 剩余用于生成
    
    // 实际估算
    estimate:
        system_prompt_tokens: 487
        user_input_tokens: 156
        available_for_generation: 1357
        
        warnings: []
        errors: []
}

// Token 使用报告
report "token_usage" {
    template: "code_generator"
    
    breakdown:
        - section: "System Prompt"
          tokens: 487
          percentage: 24.4
          char_count: 2100
        
        - section: "User Input"
          tokens: 156
          percentage: 7.8
          char_count: 680
        
        - section: "Context填充"
          tokens: 89
          percentage: 4.5
        
        - section: "Available for Generation"
          tokens: 1357
          percentage: 67.9
    
    total:
        tokens: 1989
        char_count: 8432
        within_budget: true
        headroom: 11 tokens
}

// Prompt DSL 输出验证器

// 定义输出 Schema
output_schema "code_response" {
    fields:
        - name: "code"
          type: "string"
          required: true
          description: "The generated code"
        
        - name: "language"
          type: "string"
          required: true
          enum: ["python", "javascript", "java", "go", "rust"]
        
        - name: "explanation"
          type: "string"
          required: false
        
        - name: "complexity"
          type: "string"
          enum: ["low", "medium", "high"]
        
        - name: "test_cases"
          type: "array"
          items:
              type: "object"
              fields:
                  - name: "input"
                    type: "string"
                  - name: "expected_output"
                    type: "string"
                  - name: "description"
                    type: "string"
        
        - name: "warnings"
          type: "array"
          items:
              type: "string"
    
    constraints:
        - "code must not be empty"
        - "code must have valid syntax for language"
        - "test_cases should cover edge cases"
}

// 验证器实现
validator "code_response_validator" {
    // JSON Schema 验证
    json_schema: output_schema
    
    // 自定义验证规则
    custom_rules:
        - name: "syntax_check"
          check: "validate_syntax(code, language)"
          message: "Generated code has syntax errors"
        
        - name: "security_check"
          check: "scan_security_issues(code)"
          message: "Potential security vulnerabilities detected"
        
        - name: "style_check"
          check: "validate_style(code, language)"
          message: "Code style violations detected"
        
        - name: "test_coverage"
          check: "test_cases.length >= 3"
          message: "At least 3 test cases required"
}

// 验证结果
validation_result "example_validation" {
    valid: false
    errors:
        - field: "code"
          message: "Missing return statement"
          severity: "error"
        - field: "test_cases"
          message: "Only 2 test cases provided, minimum 3 required"
          severity: "error"
    
    warnings:
        - field: "complexity"
          message: "Complexity rated as 'high', consider simplifying"
          severity: "warning"
        - field: "explanation"
          message: "No explanation provided"
          severity: "warning"
    
    metadata:
        validation_time_ms: 45
        schema_version: "1.0"
}

// Prompt DSL 调试日志系统

// 调试配置
debug_config {
    // 日志级别
    level: "verbose"  // off | error | warn | info | verbose | debug
    
    // 日志目标
    outputs:
        - type: "console"
          colorize: true
        - type: "file"
          path: "./logs/prompt_debug.log"
          rotation: "daily"
        - type: "remote"
          endpoint: "https://logs.example.com/prompt"
          batch_size: 100
    
    // 日志包含的信息
    include:
        - template_name
        - template_version
        - input_variables
        - rendered_prompt
        - token_count
        - render_time_ms
        - validation_results
        - generated_output
        - model_response
        - end_to_end_latency
    
    // 敏感信息过滤
    redact:
        - "password"
        - "api_key"
        - "token"
        - "secret"
        - pattern: "Bearer [A-Za-z0-9]+"
}

// 调试日志示例
log_entry "render_trace" {
    timestamp: "2024-03-20T14:32:15.123Z"
    level: "debug"
    
    template: "code_assistant"
    version: "2.1.0"
    
    render:
        input_variables:
            language: "Python"
            task: "implement quicksort"
            user_level: "intermediate"
        
        render_steps:
            - step: 1
              name: "resolve_variables"
              duration_ms: 0.5
              result: "All variables resolved"
            
            - step: 2
              name: "apply_rules"
              duration_ms: 2.3
              rules_applied: ["security", "formatting", "best_practices"]
            
            - step: 3
              name: "inherit_parent"
              duration_ms: 1.1
              parent: "base_conversation"
            
            - step: 4
              name: "final_render"
              duration_ms: 0.8
              output_length: 1234
        
        total_render_time_ms: 4.7
    
    output:
        token_estimate: 234
        sections:
            - name: "system"
              tokens: 156
            - name: "user"
              tokens: 78
    
    validation:
        passed: true
        warnings: []
}

Prompt DSL 解释器架构详解：

┌────────────────────────────────────────────────────────────────┐
│                      解释器（Interpreter）                       │
├────────────────────────────────────────────────────────────────┤
│                                                                │
│  ┌──────────────┐     ┌──────────────┐     ┌──────────────┐   │
│  │    Lexer     │ ──▶ │    Parser    │ ──▶ │  Evaluator   │   │
│  │  词法分析器   │     │  语法分析器   │     │   求值器     │   │
│  └──────────────┘     └──────────────┘     └──────────────┘   │
│         │                   │                    │            │
│         ▼                   ▼                    ▼            │
│  ┌──────────────┐     ┌──────────────┐     ┌──────────────┐   │
│  │  Token 列表   │     │    AST       │     │   Context    │   │
│  │  (源代码)     │     │  (语法树)     │     │   (上下文)    │   │
│  └──────────────┘     └──────────────┘     └──────────────┘   │
│                                                                │
├────────────────────────────────────────────────────────────────┤
│                        核心组件                                  │
├────────────────────────────────────────────────────────────────┤
│                                                                │
│  1. Token 类型定义                                              │
│  2. 词法规则（正则表达式）                                       │
│  3. AST 节点类型                                                │
│  4. 作用域管理                                                  │
│  5. 内置函数库                                                  │
│  6. 错误处理机制                                                │
│                                                                │
└────────────────────────────────────────────────────────────────┘

"""
Prompt DSL 解释器 - 词法分析器 (Lexer)
文件: prompt_dsl/lexer.py

本模块实现 Prompt DSL 的词法分析器，将源代码字符串转换为 Token 流。
"""

from enum import Enum, auto
from dataclasses import dataclass
from typing import List, Optional, Tuple
import re


class TokenType(Enum):
    """Prompt DSL Token 类型"""
    # 字面量
    STRING = auto()           # "hello" 或 'hello'
    NUMBER = auto()          # 42, 3.14
    BOOLEAN = auto()         # true, false
    NULL = auto()            # null
    
    # 标识符
    IDENTIFIER = auto()      # 变量名、模板名
    VARIABLE = auto()        # ${var_name}
    
    # 模板语法
    TEMPLATE_START = auto()  # template "
    TEMPLATE_END = auto()    # }
    LET = auto()             # let
    INCLUDE = auto()         # include
    IF = auto()              # if
    ELSE = auto()            # else
    ELSE_IF = auto()         # else if
    ENDIF = auto()           # endif
    FOREACH = auto()         # foreach
    IN = auto()              # in
    ENDFOR = auto()          # endfor
    EXTENDS = auto()         # extends
    MIXIN = auto()           # mixin
    IMPORT = auto()          # import
    VERSION = auto()         # version
    
    # 操作符
    PLUS = auto()            # +
    MINUS = auto()           # -
    STAR = auto()            # *
    SLASH = auto()           # /
    EQ = auto()              # =
    EQEQ = auto()            # ==
    NEQ = auto()            # !=
    LT = auto()              # <
    GT = auto()              # >
    LTE = auto()            # <=
    GTE = auto()            # >=
    QUESTION = auto()        # ?
    COLON = auto()           # :
    PIPE = auto()            # |
    
    # 分隔符
    LPAREN = auto()          # (
    RPAREN = auto()          # )
    LBRACE = auto()          # {
    RBRACE = auto()          # }
    LBRACKET = auto()        # [
    RBRACKET = auto()        # ]
    COMMA = auto()           # ,
    DOT = auto()             # .
    NEWLINE = auto()         # \n
    
    # 注释
    COMMENT = auto()          # // 单行注释
    BLOCK_COMMENT = auto()   # /* 多行注释 */
    
    # 文件结束
    EOF = auto()
    
    # 错误
    ERROR = auto()


@dataclass
class Token:
    """Token 数据结构"""
    type: TokenType
    value: any
    line: int
    column: int
    
    def __repr__(self):
        return f"Token({self.type.name}, {repr(self.value)}, {self.line}:{self.column})"


class LexerError(Exception):
    """词法分析错误"""
    def __init__(self, message: str, line: int, column: int):
        self.message = message
        self.line = line
        self.column = column
        super().__init__(f"Lexer Error at {line}:{column}: {message}")


class Lexer:
    """
    Prompt DSL 词法分析器
    
    词法规则定义：
    - 标识符: [a-zA-Z_][a-zA-Z0-9_]*
    - 数字: [0-9]+(\.[0-9]+)?
    - 字符串: "..." 或 '...'
    - 变量: ${identifier} 或 ${expression}
    - 模板名: "template_name"
    
    示例输入:
        template "hello" {
            "Hello ${name}!"
        }
    """
    
    # 关键字映射
    KEYWORDS = {
        'template': TokenType.TEMPLATE_START,
        'let': TokenType.LET,
        'include': TokenType.INCLUDE,
        'if': TokenType.IF,
        'else': TokenType.ELSE,
        'endif': TokenType.ENDIF,
        'foreach': TokenType.FOREACH,
        'in': TokenType.IN,
        'endfor': TokenType.ENDFOR,
        'extends': TokenType.EXTENDS,
        'mixin': TokenType.MIXIN,
        'import': TokenType.IMPORT,
        'version': TokenType.VERSION,
        'true': TokenType.BOOLEAN,
        'false': TokenType.BOOLEAN,
        'null': TokenType.NULL,
    }
    
    # 正则表达式规则
    PATTERNS = [
        (r'[a-zA-Z_][a-zA-Z0-9_]*', 'IDENTIFIER'),
        (r'"[^"]*"', 'STRING'),
        (r"'[^']*'", 'STRING'),
        (r'\$\{[^{}]*\}', 'VARIABLE'),
        (r'[0-9]+\.[0-9]+', 'NUMBER'),
        (r'[0-9]+', 'NUMBER'),
        (r'\+\+', 'PLUS'),
        (r'--', 'MINUS'),
        (r'==', 'EQEQ'),
        (r'!=', 'NEQ'),
        (r'<=', 'LTE'),
        (r'>=', 'GTE'),
        (r'=>', 'ARROW'),
        (r'->', 'DARROW'),
    ]
    
    def __init__(self, source: str):
        self.source = source
        self.pos = 0
        self.line = 1
        self.column = 1
        self.tokens: List[Token] = []
        
    def tokenize(self) -> List[Token]:
        """执行词法分析，返回 Token 列表"""
        while self.pos < len(self.source):
            self._scan_token()
        
        self.tokens.append(Token(TokenType.EOF, None, self.line, self.column))
        return self.tokens
    
    def _scan_token(self):
        """扫描单个 Token"""
        # 跳过空白字符（但保留换行）
        if self._skip_whitespace():
            return
        
        # 行号更新
        if self.source[self.pos] == '\n':
            self.line += 1
            self.column = 1
            self.pos += 1
            return
        
        # 尝试匹配各种模式
        for pattern, token_type in self.PATTERNS:
            regex = re.compile(pattern)
            match = regex.match(self.source, self.pos)
            if match:
                value = match.group()
                
                # 处理关键字
                if token_type == 'IDENTIFIER' and value in self.KEYWORDS:
                    token_type = self.KEYWORDS[value]
                
                # 特殊处理 VARIABLE
                if token_type == 'VARIABLE':
                    # 去掉 ${ 和 }
                    value = value[2:-1]
                    token_type = TokenType.VARIABLE
                
                # 特殊处理字符串
                if token_type == 'STRING':
                    # 去掉引号
                    value = value[1:-1]
                
                # 特殊处理数字
                if token_type == 'NUMBER':
                    if '.' in value:
                        value = float(value)
                    else:
                        value = int(value)
                
                # 特殊处理布尔值
                if token_type == 'BOOLEAN':
                    value = value == 'true'
                
                self.tokens.append(Token(
                    TokenType[token_type] if isinstance(token_type, str) else token_type,
                    value,
                    self.line,
                    self.column
                ))
                
                self.pos = match.end()
                self.column += match.end() - match.start()
                return
        
        # 如果没有匹配到任何模式，报告错误
        raise LexerError(
            f"Unexpected character: {repr(self.source[self.pos])}",
            self.line,
            self.column
        )
    
    def _skip_whitespace(self) -> bool:
        """跳过空白字符（不包括换行）"""
        if self.pos < len(self.source) and self.source[self.pos] in ' \t\r':
            self.pos += 1
            self.column += 1
            return True
        return False


def tokenize(source: str) -> List[Token]:
    """便捷函数：执行词法分析"""
    lexer = Lexer(source)
    return lexer.tokenize()

"""
Prompt DSL 解释器 - 语法分析器 (Parser)
文件: prompt_dsl/parser.py

本模块实现 Prompt DSL 的语法分析器，将 Token 流转换为 AST（抽象语法树）。
"""

from typing import List, Optional, Dict, Any
from dataclasses import dataclass, field
from abc import ABC, abstractmethod

from prompt_dsl.lexer import Token, TokenType, LexerError


# =============================================================================
# AST 节点基类
# =============================================================================

class ASTNode(ABC):
    """AST 节点基类"""
    
    @abstractmethod
    def accept(self, visitor: 'ASTVisitor'):
        """接受访问者（Visitor Pattern）"""
        pass


@dataclass
class Program(ASTNode):
    """程序根节点"""
    statements: List[ASTNode] = field(default_factory=list)
    
    def accept(self, visitor: 'ASTVisitor'):
        return visitor.visit_program(self)


# =============================================================================
# 模板相关节点
# =============================================================================

@dataclass
class TemplateNode(ASTNode):
    """模板定义节点"""
    name: str
    version: Optional[str] = None
    body: List[ASTNode] = field(default_factory=list)
    parent: Optional[str] = None
    mixins: List[str] = field(default_factory=list)
    
    def accept(self, visitor: 'ASTVisitor'):
        return visitor.visit_template(self)


@dataclass
class VariableDeclaration(ASTNode):
    """变量声明节点"""
    name: str
    value: ASTNode
    is_param: bool = False
    default_value: Optional[ASTNode] = None
    
    def accept(self, visitor: 'ASTVisitor'):
        return visitor.visit_variable_declaration(self)


@dataclass
class LiteralNode(ASTNode):
    """字面量节点"""
    value: any
    
    def accept(self, visitor: 'ASTVisitor'):
        return visitor.visit_literal(self)


@dataclass
class VariableNode(ASTNode):
    """变量引用节点"""
    name: str
    
    def accept(self, visitor: 'ASTVisitor'):
        return visitor.visit_variable(self)


@dataclass
class StringInterpolation(ASTNode):
    """字符串插值节点"""
    template: str
    expressions: List[ASTNode] = field(default_factory=list)
    
    def accept(self, visitor: 'ASTVisitor'):
        return visitor.visit_string_interpolation(self)


@dataclass
class FilterNode(ASTNode):
    """过滤器节点"""
    value: ASTNode
    filter_name: str
    args: List[ASTNode] = field(default_factory=list)
    
    def accept(self, visitor: 'ASTVisitor'):
        return visitor.visit_filter(self)


# =============================================================================
# 控制流节点
# =============================================================================

@dataclass
class IfNode(ASTNode):
    """条件节点"""
    condition: ASTNode
    then_branch: List[ASTNode] = field(default_factory=list)
    else_if_branches: List[tuple] = field(default_factory=list)  # [(condition, body), ...]
    else_branch: List[ASTNode] = field(default_factory=list)
    
    def accept(self, visitor: 'ASTVisitor'):
        return visitor.visit_if(self)


@dataclass
class ForeachNode(ASTNode):
    """循环节点"""
    variable: str
    iterable: ASTNode
    body: List[ASTNode] = field(default_factory=list)
    
    def accept(self, visitor: 'ASTVisitor'):
        return visitor.visit_foreach(self)


@dataclass
class IncludeNode(ASTNode):
    """包含节点"""
    template_name: str
    params: Optional[Dict[str, ASTNode]] = None
    
    def accept(self, visitor: 'ASTVisitor'):
        return visitor.visit_include(self)


# =============================================================================
# 表达式节点
# =============================================================================

@dataclass
class BinaryOpNode(ASTNode):
    """二元操作符节点"""
    left: ASTNode
    operator: str
    right: ASTNode
    
    def accept(self, visitor: 'ASTVisitor'):
        return visitor.visit_binary_op(self)


@dataclass
class UnaryOpNode(ASTNode):
    """一元操作符节点"""
    operator: str
    operand: ASTNode
    
    def accept(self, visitor: 'ASTVisitor'):
        return visitor.visit_unary_op(self)


@dataclass
class ArrayNode(ASTNode):
    """数组节点"""
    elements: List[ASTNode] = field(default_factory=list)
    
    def accept(self, visitor: 'ASTVisitor'):
        return visitor.visit_array(self)


@dataclass
class ObjectNode(ASTNode):
    """对象节点"""
    properties: Dict[str, ASTNode] = field(default_factory=dict)
    
    def accept(self, visitor: 'ASTVisitor'):
        return visitor.visit_object(self)


# =============================================================================
# AST Visitor 接口
# =============================================================================

class ASTVisitor(ABC):
    """AST Visitor 接口"""
    
    def visit_program(self, node: Program) -> Any: pass
    def visit_template(self, node: TemplateNode) -> Any: pass
    def visit_variable_declaration(self, node: VariableDeclaration) -> Any: pass
    def visit_literal(self, node: LiteralNode) -> Any: pass
    def visit_variable(self, node: VariableNode) -> Any: pass
    def visit_string_interpolation(self, node: StringInterpolation) -> Any: pass
    def visit_filter(self, node: FilterNode) -> Any: pass
    def visit_if(self, node: IfNode) -> Any: pass
    def visit_foreach(self, node: ForeachNode) -> Any: pass
    def visit_include(self, node: IncludeNode) -> Any: pass
    def visit_binary_op(self, node: BinaryOpNode) -> Any: pass
    def visit_unary_op(self, node: UnaryOpNode) -> Any: pass
    def visit_array(self, node: ArrayNode) -> Any: pass
    def visit_object(self, node: ObjectNode) -> Any: pass


# =============================================================================
# 语法分析器
# =============================================================================

class ParserError(Exception):
    """语法分析错误"""
    def __init__(self, message: str, token: Token):
        self.message = message
        self.token = token
        super().__init__(f"Parser Error at {token.line}:{token.column}: {message}")


class Parser:
    """
    Prompt DSL 语法分析器
    
    语法规则 (EBNF):
        program       := statement*
        statement     := template_def | variable_decl | expr
        template_def  := 'template' STRING ('version' STRING)? ('extends' STRING)? '{' body '}'
        body          := statement*
        variable_decl := 'let' IDENTIFIER '=' expr
        expr          := string_interp | binary_op | primary
        string_interp := STRING ('${' expr '}')*
        binary_op     := expr ('+' | '-' | '*' | '/' | '==' | '!=' | '<' | '>' | '<=' | '>=') expr
        primary       := IDENTIFIER | STRING | NUMBER | BOOLEAN | NULL | array | object
        array         := '[' expr (',' expr)* ']'
        object        := '{' (IDENTIFIER ':' expr (',' IDENTIFIER ':' expr)*)? '}'
        if_stmt       := 'if' expr '{' body '}' ('else' 'if' expr '{' body '}')* ('else' '{' body '}')? 'endif'
        foreach_stmt  := 'foreach' IDENTIFIER 'in' expr '{' body '}' 'endfor'
        include_stmt  := 'include' STRING ('with' props)?
    """
    
    def __init__(self, tokens: List[Token]):
        self.tokens = tokens
        self.pos = 0
        self.current = tokens[0] if tokens else None
        
    def _advance(self):
        """移动到下一个 Token"""
        self.pos += 1
        if self.pos < len(self.tokens):
            self.current = self.tokens[self.pos]
    
    def _peek(self, offset: int = 1) -> Optional[Token]:
        """查看未来的 Token"""
        idx = self.pos + offset
        if idx < len(self.tokens):
            return self.tokens[idx]
        return None
    
    def _expect(self, token_type: TokenType, message: str) -> Token:
        """期望指定类型的 Token"""
        if self.current.type != token_type:
            raise ParserError(message, self.current)
        token = self.current
        self._advance()
        return token
    
    def _match(self, *token_types: TokenType) -> bool:
        """检查当前 Token 是否匹配指定类型"""
        if self.current.type in token_types:
            self._advance()
            return True
        return False
    
    def parse(self) -> Program:
        """执行语法分析，返回 AST"""
        program = Program()
        
        while self.current.type != TokenType.EOF:
            stmt = self._parse_statement()
            if stmt:
                program.statements.append(stmt)
        
        return program
    
    def _parse_statement(self) -> Optional[ASTNode]:
        """解析语句"""
        if self.current.type == TokenType.TEMPLATE_START:
            return self._parse_template()
        elif self.current.type == TokenType.LET:
            return self._parse_variable_decl()
        elif self.current.type == TokenType.IF:
            return self._parse_if()
        elif self.current.type == TokenType.FOREACH:
            return self._parse_foreach()
        elif self.current.type == TokenType.INCLUDE:
            return self._parse_include()
        else:
            return self._parse_expression()
    
    def _parse_template(self) -> TemplateNode:
        """解析模板定义"""
        self._advance()  # 跳过 'template'
        
        name = self._expect(TokenType.STRING, "Expected template name").value
        version = None
        
        # 可选版本
        if self._match(TokenType.VERSION):
            version = self._expect(TokenType.STRING, "Expected version string").value
        
        # 可选父模板
        parent = None
        if self._match(TokenType.EXTENDS):
            parent = self._expect(TokenType.STRING, "Expected parent template name").value
        
        self._expect(TokenType.LBRACE, "Expected '{' after template declaration")
        
        body = []
        while self.current.type != TokenType.RBRACE and self.current.type != TokenType.EOF:
            body.append(self._parse_statement())
        
        self._expect(TokenType.RBRACE, "Expected '}' after template body")
        
        return TemplateNode(name=name, version=version, body=body, parent=parent)
    
    def _parse_variable_decl(self) -> VariableDeclaration:
        """解析变量声明"""
        self._advance()  # 跳过 'let'
        
        name = self._expect(TokenType.IDENTIFIER, "Expected variable name").value
        self._expect(TokenType.EQ, "Expected '=' after variable name")
        
        value = self._parse_expression()
        
        return VariableDeclaration(name=name, value=value)
    
    def _parse_if(self) -> IfNode:
        """解析 if 语句"""
        self._advance()  # 跳过 'if'
        
        condition = self._parse_expression()
        self._expect(TokenType.LBRACE, "Expected '{' after if condition")
        
        then_branch = []
        while self.current.type != TokenType.RBRACE and self.current.type != TokenType.EOF:
            then_branch.append(self._parse_statement())
        
        self._expect(TokenType.RBRACE, "Expected '}' after if body")
        
        else_if_branches = []
        else_branch = []
        
        while self._match(TokenType.ELSE):
            if self._match(TokenType.IF):
                # else if
                elif_cond = self._parse_expression()
                self._expect(TokenType.LBRACE, "Expected '{' after else if condition")
                
                elif_body = []
                while self.current.type != TokenType.RBRACE and self.current.type != TokenType.EOF:
                    elif_body.append(self._parse_statement())
                
                self._expect(TokenType.RBRACE, "Expected '}' after else if body")
                else_if_branches.append((elif_cond, elif_body))
            else:
                # else
                self._expect(TokenType.LBRACE, "Expected '{' after else")
                
                while self.current.type != TokenType.RBRACE and self.current.type != TokenType.EOF:
                    else_branch.append(self._parse_statement())
                
                self._expect(TokenType.RBRACE, "Expected '}' after else body")
                break
        
        self._expect(TokenType.ENDIF, "Expected 'endif' after if-else block")
        
        return IfNode(
            condition=condition,
            then_branch=then_branch,
            else_if_branches=else_if_branches,
            else_branch=else_branch
        )
    
    def _parse_foreach(self) -> ForeachNode:
        """解析 foreach 循环"""
        self._advance()  # 跳过 'foreach'
        
        variable = self._expect(TokenType.IDENTIFIER, "Expected loop variable name").value
        self._expect(TokenType.IN, "Expected 'in' after loop variable")
        
        iterable = self._parse_expression()
        self._expect(TokenType.LBRACE, "Expected '{' after iterable expression")
        
        body = []
        while self.current.type != TokenType.RBRACE and self.current.type != TokenType.EOF:
            body.append(self._parse_statement())
        
        self._expect(TokenType.RBRACE, "Expected '}' after foreach body")
        self._expect(TokenType.ENDFOR, "Expected 'endfor' after foreach body")
        
        return ForeachNode(variable=variable, iterable=iterable, body=body)
    
    def _parse_include(self) -> IncludeNode:
        """解析 include 语句"""
        self._advance()  # 跳过 'include'
        
        template_name = self._expect(TokenType.STRING, "Expected template name").value
        
        return IncludeNode(template_name=template_name)
    
    def _parse_expression(self) -> ASTNode:
        """解析表达式"""
        return self._parse_comparison()
    
    def _parse_comparison(self) -> ASTNode:
        """解析比较表达式"""
        left = self._parse_addition()
        
        while self.current.type in (TokenType.EQEQ, TokenType.NEQ, TokenType.LT, TokenType.GT, TokenType.LTE, TokenType.GTE):
            operator = self.current.value
            self._advance()
            right = self._parse_addition()
            left = BinaryOpNode(left=left, operator=operator, right=right)
        
        return left
    
    def _parse_addition(self) -> ASTNode:
        """解析加减表达式"""
        left = self._parse_multiplication()
        
        while self.current.type in (TokenType.PLUS, TokenType.MINUS):
            operator = self.current.value
            self._advance()
            right = self._parse_multiplication()
            left = BinaryOpNode(left=left, operator=operator, right=right)
        
        return left
    
    def _parse_multiplication(self) -> ASTNode:
        """解析乘除表达式"""
        left = self._parse_primary()
        
        while self.current.type in (TokenType.STAR, TokenType.SLASH):
            operator = self.current.value
            self._advance()
            right = self._parse_primary()
            left = BinaryOpNode(left=left, operator=operator, right=right)
        
        return left
    
    def _parse_primary(self) -> ASTNode:
        """解析基本表达式"""
        token = self.current
        
        if token.type == TokenType.STRING:
            self._advance()
            return LiteralNode(value=token.value)
        
        if token.type == TokenType.NUMBER:
            self._advance()
            return LiteralNode(value=token.value)
        
        if token.type == TokenType.BOOLEAN:
            self._advance()
            return LiteralNode(value=token.value)
        
        if token.type == TokenType.NULL:
            self._advance()
            return LiteralNode(value=None)
        
        if token.type == TokenType.IDENTIFIER:
            self._advance()
            return VariableNode(name=token.value)
        
        if token.type == TokenType.LBRACKET:
            return self._parse_array()
        
        if token.type == TokenType.LBRACE:
            return self._parse_object()
        
        if token.type == TokenType.VARIABLE:
            self._advance()
            return VariableNode(name=token.value)
        
        raise ParserError(f"Unexpected token: {token}", token)
    
    def _parse_array(self) -> ArrayNode:
        """解析数组"""
        self._advance()  # 跳过 '['
        
        elements = []
        if self.current.type != TokenType.RBRACKET:
            elements.append(self._parse_expression())
            while self._match(TokenType.COMMA):
                elements.append(self._parse_expression())
        
        self._expect(TokenType.RBRACKET, "Expected ']' after array elements")
        
        return ArrayNode(elements=elements)
    
    def _parse_object(self) -> ObjectNode:
        """解析对象"""
        self._advance()  # 跳过 '{'
        
        properties = {}
        if self.current.type != TokenType.RBRACE:
            key = self._expect(TokenType.IDENTIFIER, "Expected property key").value
            self._expect(TokenType.COLON, "Expected ':' after property key")
            properties[key] = self._parse_expression()
            
            while self._match(TokenType.COMMA):
                key = self._expect(TokenType.IDENTIFIER, "Expected property key").value
                self._expect(TokenType.COLON, "Expected ':' after property key")
                properties[key] = self._parse_expression()
        
        self._expect(TokenType.RBRACE, "Expected '}' after object properties")
        
        return ObjectNode(properties=properties)


def parse(tokens: List[Token]) -> Program:
    """便捷函数：执行语法分析"""
    parser = Parser(tokens)
    return parser.parse()

"""
Prompt DSL 解释器 - 求值器 (Evaluator)
文件: prompt_dsl/evaluator.py

本模块实现 Prompt DSL 的求值器，对 AST 进行求值并生成最终 Prompt。
"""

from typing import Dict, List, Any, Optional, Callable
from dataclasses import dataclass, field

from prompt_dsl.parser import (
    Program, TemplateNode, VariableDeclaration, LiteralNode, VariableNode,
    StringInterpolation, FilterNode, IfNode, ForeachNode, IncludeNode,
    BinaryOpNode, UnaryOpNode, ArrayNode, ObjectNode, ASTVisitor
)


# =============================================================================
# 求值上下文
# =============================================================================

@dataclass
class Scope:
    """作用域"""
    variables: Dict[str, Any] = field(default_factory=dict)
    parent: Optional['Scope'] = None
    
    def get(self, name: str) -> Any:
        """从当前或父作用域获取变量"""
        if name in self.variables:
            return self.variables[name]
        if self.parent:
            return self.parent.get(name)
        raise NameError(f"Variable '{name}' is not defined")
    
    def set(self, name: str, value: Any):
        """在当前作用域设置变量"""
        self.variables[name] = value
    
    def has(self, name: str) -> bool:
        """检查变量是否存在"""
        return name in self.variables or (self.parent and self.parent.has(name))


@dataclass
class Context:
    """求值上下文"""
    scope: Scope = field(default_factory=Scope)
    template_registry: Dict[str, TemplateNode] = field(default_factory=dict)
    parent_template: Optional[str] = None
    loop_vars: Dict[str, Any] = field(default_factory=dict)
    
    def create_child_scope(self) -> 'Context':
        """创建子上下文"""
        child_scope = Scope(parent=self.scope)
        return Context(
            scope=child_scope,
            template_registry=self.template_registry,
            parent_template=self.parent_template
        )


# =============================================================================
# 内置过滤器
# =============================================================================

class BuiltinFilters:
    """内置过滤器库"""
    
    @staticmethod
    def upper(value: str) -> str:
        return str(value).upper()
    
    @staticmethod
    def lower(value: str) -> str:
        return str(value).lower()
    
    @staticmethod
    def title(value: str) -> str:
        return str(value).title()
    
    @staticmethod
    def capitalize(value: str) -> str:
        return str(value).capitalize()
    
    @staticmethod
    def truncate(value: str, length: int = 50) -> str:
        s = str(value)
        if len(s) <= length:
            return s
        return s[:length] + "..."
    
    @staticmethod
    def join(value: List, separator: str = ", ") -> str:
        return separator.join(str(v) for v in value)
    
    @staticmethod
    def length(value: Any) -> int:
        if isinstance(value, (list, str)):
            return len(value)
        return 1
    
    @staticmethod
    def default(value: Any, default_val: Any) -> Any:
        return value if value is not None else default_val
    
    @staticmethod
    def json(value: Any) -> str:
        import json
        return json.dumps(value)
    
    @staticmethod
    def int(value: Any) -> int:
        return int(value)
    
    @staticmethod
    def float(value: Any) -> float:
        return float(value)
    
    @staticmethod
    def string(value: Any) -> str:
        return str(value)


FILTERS: Dict[str, Callable] = {
    name: getattr(BuiltinFilters, name)
    for name in dir(BuiltinFilters)
    if not name.startswith('_')
}


# =============================================================================
# 求值器
# =============================================================================

class Evaluator(ASTVisitor):
    """
    Prompt DSL 求值器
    
    求值策略：
    - 模板定义：注册到模板注册表，不立即求值
    - 变量声明：在作用域中绑定变量
    - 字面量：直接返回值
    - 变量引用：从作用域解析
    - 二元运算：递归求值后计算
    - 条件：基于条件值选择分支
    - 循环：迭代求值
    """
    
    def __init__(self):
        self.context = Context()
        self.output_parts: List[str] = []
    
    def evaluate(self, ast: Program, context: Optional[Context] = None) -> str:
        """对 AST 进行求值，返回最终 Prompt 字符串"""
        if context:
            self.context = context
        
        for stmt in ast.statements:
            self.visit(stmt)
        
        return ''.join(self.output_parts)
    
    def visit(self, node) -> Any:
        """访问 AST 节点"""
        return node.accept(self)
    
    def visit_program(self, node: Program) -> Any:
        """求值程序节点"""
        for stmt in node.statements:
            self.visit(stmt)
        return None
    
    def visit_template(self, node: TemplateNode) -> Any:
        """求值模板定义节点"""
        # 注册模板到注册表
        self.context.template_registry[node.name] = node
        
        # 如果是当前要渲染的模板
        if hasattr(self, '_current_template') and self._current_template == node.name:
            # 创建子作用域
            child_context = self.context.create_child_scope()
            child_context.parent_template = node.parent
            
            # 处理继承
            if node.parent and node.parent in self.context.template_registry:
                parent_template = self.context.template_registry[node.parent]
                self._render_template_body(parent_template.body, child_context)
            
            # 处理 mixin
            for mixin_name in node.mixins:
                if mixin_name in self.context.template_registry:
                    mixin_template = self.context.template_registry[mixin_name]
                    self._render_template_body(mixin_template.body, child_context)
            
            # 渲染自身内容
            self._render_template_body(node.body, child_context)
        
        return None
    
    def _render_template_body(self, body: List, context: Context):
        """渲染模板体"""
        old_context = self.context
        old_output = self.output_parts
        
        self.context = context
        self.output_parts = []
        
        for stmt in body:
            self.visit(stmt)
        
        # 将输出添加到父输出
        old_output.extend(self.output_parts)
        self.output_parts = old_output
        self.context = old_context
    
    def visit_variable_declaration(self, node: VariableDeclaration) -> Any:
        """求值变量声明节点"""
        value = self.visit(node.value)
        self.context.scope.set(node.name, value)
        return value
    
    def visit_literal(self, node: LiteralNode) -> Any:
        """求值字面量节点"""
        return node.value
    
    def visit_variable(self, node: VariableNode) -> Any:
        """求值变量引用节点"""
        # 首先检查循环变量
        if node.name in self.context.loop_vars:
            return self.context.loop_vars[node.name]
        
        # 然后检查作用域
        try:
            return self.context.scope.get(node.name)
        except NameError:
            # 如果未定义，返回变量引用本身（用于字符串插值）
            return f"${{{node.name}}}"
    
    def visit_string_interpolation(self, node: StringInterpolation) -> Any:
        """求值字符串插值节点"""
        result = node.template
        for expr in node.expressions:
            value = self.visit(expr)
            result = result.replace(f"${{{expr}}}", str(value))
        return result
    
    def visit_filter(self, node: FilterNode) -> Any:
        """求值过滤器节点"""
        value = self.visit(node.value)
        
        # 获取过滤器函数
        if node.filter_name in FILTERS:
            filter_func = FILTERS[node.filter_name]
            args = [self.visit(arg) for arg in node.args]
            return filter_func(value, *args)
        
        raise NameError(f"Unknown filter: {node.filter_name}")
    
    def visit_if(self, node: IfNode) -> Any:
        """求值条件节点"""
        condition = self.visit(node.condition)
        
        if self._is_truthy(condition):
            for stmt in node.then_branch:
                self.visit(stmt)
        else:
            # 检查 else if 分支
            for elif_cond, elif_body in node.else_if_branches:
                if self._is_truthy(self.visit(elif_cond)):
                    for stmt in elif_body:
                        self.visit(stmt)
                    return None
            
            # 执行 else 分支
            for stmt in node.else_branch:
                self.visit(stmt)
        
        return None
    
    def visit_foreach(self, node: ForeachNode) -> Any:
        """求值循环节点"""
        iterable = self.visit(node.iterable)
        
        if not isinstance(iterable, (list, tuple)):
            iterable = [iterable]
        
        for item in iterable:
            # 设置循环变量
            old_loop_vars = self.context.loop_vars.copy()
            self.context.loop_vars[node.variable] = item
            
            for stmt in node.body:
                self.visit(stmt)
            
            self.context.loop_vars = old_loop_vars
        
        return None
    
    def visit_include(self, node: IncludeNode) -> Any:
        """求值包含节点"""
        template_name = node.template_name
        
        if template_name not in self.context.template_registry:
            raise NameError(f"Template '{template_name}' is not defined")
        
        template = self.context.template_registry[template_name]
        self._render_template_body(template.body, self.context)
        
        return None
    
    def visit_binary_op(self, node: BinaryOpNode) -> Any:
        """求值二元操作符节点"""
        left = self.visit(node.left)
        right = self.visit(node.right)
        
        ops = {
            '+': lambda a, b: a + b,
            '-': lambda a, b: a - b,
            '*': lambda a, b: a * b,
            '/': lambda a, b: a / b,
            '==': lambda a, b: a == b,
            '!=': lambda a, b: a != b,
            '<': lambda a, b: a < b,
            '>': lambda a, b: a > b,
            '<=': lambda a, b: a <= b,
            '>=': lambda a, b: a >= b,
        }
        
        if node.operator not in ops:
            raise ValueError(f"Unknown operator: {node.operator}")
        
        return ops[node.operator](left, right)
    
    def visit_unary_op(self, node: UnaryOpNode) -> Any:
        """求值一元操作符节点"""
        operand = self.visit(node.operand)
        
        if node.operator == '!':
            return not operand
        elif node.operator == '-':
            return -operand
        
        raise ValueError(f"Unknown unary operator: {node.operator}")
    
    def visit_array(self, node: ArrayNode) -> Any:
        """求值数组节点"""
        return [self.visit(elem) for elem in node.elements]
    
    def visit_object(self, node: ObjectNode) -> Any:
        """求值对象节点"""
        return {key: self.visit(val) for key, val in node.properties.items}
    
    def _is_truthy(self, value: Any) -> bool:
        """判断值是否为真"""
        if value is None:
            return False
        if isinstance(value, bool):
            return value
        if isinstance(value, (int, float)) and value == 0:
            return False
        if isinstance(value, str) and value == "":
            return False
        if isinstance(value, (list, tuple)) and len(value) == 0:
            return False
        return True


# =============================================================================
# 主函数
# =============================================================================

def evaluate(ast: Program, context: Optional[Context] = None) -> str:
    """便捷函数：求值 AST"""
    evaluator = Evaluator()
    return evaluator.evaluate(ast, context)


def render_template(template_name: str, ast: Program, variables: Dict[str, Any]) -> str:
    """渲染指定模板"""
    # 找到模板定义
    evaluator = Evaluator()
    
    # 首先处理所有顶级语句，注册模板
    for stmt in ast.statements:
        if isinstance(stmt, TemplateNode):
            evaluator.visit(stmt)
    
    # 设置当前模板并创建上下文
    evaluator._current_template = template_name
    context = Context()
    context.scope.variables.update(variables)
    
    # 重新求值以渲染目标模板
    evaluator.output_parts = []
    for stmt in ast.statements:
        if isinstance(stmt, TemplateNode) and stmt.name == template_name:
            evaluator.visit(stmt)
    
    return ''.join(evaluator.output_parts)

Prompt DSL 成熟度模型：

Level 1: 手工阶段
├── 硬编码 Prompt
├── 无版本管理
└── 难以测试
    ↓
Level 2: 模板阶段
├── 基础模板语法
├── 简单变量替换
└── 手动版本记录
    ↓
Level 3: 工程化阶段
├── 完整的 DSL 语法
├── 类型系统与校验
├── 规则模块化
└── 基础调试工具
    ↓
Level 4: 工业化阶段
├── 版本化管理
├── A/B 测试框架
├── 性能优化
├── 团队协作支持
└── 企业级可靠性
    ↓
Level 5: 智能化阶段
├── AI 辅助 Prompt 优化
├── 自动版本选择
├── 智能规则推荐
└── 端到端质量保障

引入路径建议：

Week 1-2: 评估与试点
├── 识别高频使用的 Prompt 场景
├── 选择一个中等复杂度的场景进行试点
└── 使用现有模板引擎（如 Jinja2）快速验证概念

Week 3-4: 核心实现
├── 实现基础的 DSL 语法
├── 建立模板注册表与版本管理
└── 开发基础的预览和调试工具

Week 5-8: 完善功能
├── 实现完整的变量类型系统
├── 构建规则组合框架
└── 开发 Token 估算和输出验证工具

Week 9-12: 生产就绪
├── 实现 A/B 测试框架
├── 建立监控和告警系统
├── 编写团队使用文档
└── 培训团队成员

prompt_dsl/
├── __init__.py
├── lexer.py          # 词法分析器
├── parser.py         # 语法分析器
├── ast.py            # AST 节点定义
├── evaluator.py      # 求值器
├── context.py        # 上下文管理
├── filters.py        # 内置过滤器
├── exceptions.py     # 异常定义
├── utils.py          # 工具函数
└── main.py           # 入口文件

"""
Prompt DSL - Prompt 领域特定语言解释器

一个轻量级的 Prompt 领域特定语言，支持模板语法、条件逻辑、
循环、继承、变量类型系统、规则组合等高级特性。

快速开始:
    >>> from prompt_dsl import compile, render
    >>> 
    >>> source = '''
    ... template "greeting" {
    ...     "Hello ${name}!"
    ... }
    ... '''
    >>> 
    >>> ast = compile(source)
    >>> result = render(ast, "greeting", {"name": "World"})
    >>> print(result)
    Hello World!
"""

from prompt_dsl.lexer import Lexer, Token, TokenType, LexerError
from prompt_dsl.parser import Parser, Program, ParserError
from prompt_dsl.evaluator import Evaluator, Context, Scope
from prompt_dsl.exceptions import PromptDSLError, ValidationError

__version__ = "0.1.0"
__all__ = [
    "Lexer", "Token", "TokenType", "LexerError",
    "Parser", "Program", "ParserError",
    "Evaluator", "Context", "Scope",
    "PromptDSLError", "ValidationError",
    "compile", "render", "tokenize", "parse"
]


def tokenize(source: str):
    """词法分析源代码，返回 Token 列表"""
    lexer = Lexer(source)
    return lexer.tokenize()


def parse(tokens):
    """语法分析 Token 列表，返回 AST"""
    parser = Parser(tokens)
    return parser.parse()


def compile(source: str):
    """编译源代码字符串，返回 AST"""
    tokens = tokenize(source)
    return parse(tokens)


def render(ast: Program, template_name: str, variables: dict = None):
    """渲染指定模板，返回最终 Prompt 字符串"""
    from prompt_dsl.evaluator import render_template as _render
    return _render(template_name, ast, variables or {})

"""
Prompt DSL 异常定义
"""


class PromptDSLError(Exception):
    """Prompt DSL 基础异常"""
    pass


class LexerError(PromptDSLError):
    """词法分析错误"""
    def __init__(self, message: str, line: int = 0, column: int = 0):
        self.message = message
        self.line = line
        self.column = column
        super().__init__(f"Lexer Error at {line}:{column}: {message}")


class ParserError(PromptDSLError):
    """语法分析错误"""
    def __init__(self, message: str, line: int = 0, column: int = 0, token=None):
        self.message = message
        self.line = line
        self.column = column
        self.token = token
        if token:
            super().__init__(f"Parser Error at {token.line}:{token.column}: {message}")
        else:
            super().__init__(f"Parser Error: {message}")


class EvaluationError(PromptDSLError):
    """求值错误"""
    def __init__(self, message: str, node=None):
        self.message = message
        self.node = node
        super().__init__(f"Evaluation Error: {message}")


class ValidationError(PromptDSLError):
    """验证错误"""
    def __init__(self, message: str, field: str = None):
        self.message = message
        self.field = field
        super().__init__(f"Validation Error{f' in field {field}' if field else ''}: {message}")


class TemplateNotFoundError(PromptDSLError):
    """模板未找到错误"""
    def __init__(self, template_name: str):
        self.template_name = template_name
        super().__init__(f"Template '{template_name}' is not defined")


class TypeError(PromptDSLError):
    """类型错误"""
    def __init__(self, expected: str, actual: str, field: str = None):
        self.expected = expected
        self.actual = actual
        self.field = field
        super().__init__(
            f"Type Error{f' in field {field}' if field else ''}: "
            f"expected {expected}, got {actual}"
        )

"""
Prompt DSL 命令行工具
"""

import sys
import argparse
from pathlib import Path

from prompt_dsl import compile, render, __version__
from prompt_dsl.exceptions import PromptDSLError


def main():
    parser = argparse.ArgumentParser(
        description="Prompt DSL - Prompt 领域特定语言解释器",
        formatter_class=argparse.RawDescriptionHelpFormatter,
        epilog="""
示例:
  prompt_dsl run greet.prompt --template greeting --vars name=World
  prompt_dsl compile hello.prompt
  prompt_dsl tokens "template \"test\" { \"Hello ${name}!\" }"
        """
    )
    
    parser.add_argument('--version', action='version', version=f'prompt_dsl {__version__}')
    
    subparsers = parser.add_subparsers(dest='command', help='可用命令')
    
    # run 命令
    run_parser = subparsers.add_parser('run', help='运行 Prompt 模板')
    run_parser.add_argument('file', type=Path, help='模板文件路径')
    run_parser.add_argument('--template', '-t', required=True, help='模板名称')
    run_parser.add_argument('--vars', '-v', nargs='+', help='变量列表，格式: key=value')
    run_parser.add_argument('--pretty', '-p', action='store_true', help='格式化输出')
    
    # compile 命令
    compile_parser = subparsers.add_parser('compile', help='编译模板文件')
    compile_parser.add_argument('file', type=Path, help='模板文件路径')
    compile_parser.add_argument('--output', '-o', type=Path, help='输出 AST JSON 文件')
    
    # tokens 命令
    tokens_parser = subparsers.add_parser('tokens', help='词法分析源代码')
    tokens_parser.add_argument('source', help='源代码字符串')
    
    # 解析参数
    args = parser.parse_args()
    
    if args.command == 'run':
        run_template(args)
    elif args.command == 'compile':
        compile_template(args)
    elif args.command == 'tokens':
        tokenize_source(args)
    else:
        parser.print_help()


def run_template(args):
    """运行模板"""
    source = args.file.read_text(encoding='utf-8')
    
    try:
        ast = compile(source)
        
        # 解析变量
        variables = {}
        if args.vars:
            for var in args.vars:
                if '=' in var:
                    key, value = var.split('=', 1)
                    variables[key] = value
        
        result = render(ast, args.template, variables)
        
        if args.pretty:
            print("=" * 60)
            print(f"Template: {args.template}")
            print("=" * 60)
            print(result)
            print("=" * 60)
        else:
            print(result)
            
    except PromptDSLError as e:
        print(f"Error: {e}", file=sys.stderr)
        sys.exit(1)


def compile_template(args):
    """编译模板"""
    source = args.file.read_text(encoding='utf-8')
    
    try:
        ast = compile(source)
        
        if args.output:
            import json
            # 简单序列化 AST
            output = serialize_ast(ast)
            args.output.write_text(json.dumps(output, indent=2), encoding='utf-8')
            print(f"Compiled to {args.output}")
        else:
            print(ast)
            
    except PromptDSLError as e:
        print(f"Error: {e}", file=sys.stderr)
        sys.exit(1)


def tokenize_source(args):
    """词法分析源代码"""
    from prompt_dsl import tokenize
    
    try:
        tokens = tokenize(args.source)
        
        print(f"Tokens ({len(tokens)}):")
        for token in tokens:
            print(f"  {token}")
            
    except PromptDSLError as e:
        print(f"Error: {e}", file=sys.stderr)
        sys.exit(1)


def serialize_ast(ast):
    """简单序列化 AST"""
    # 这是一个简化的序列化实现
    result = {"type": "Program", "statements": []}
    
    def serialize_node(node):
        if hasattr(node, '__dataclass_fields__'):
            fields = {}
            for name, field in node.__dataclass_fields__.items():
                value = getattr(node, name)
                if isinstance(value, list):
                    fields[name] = [serialize_node(v) for v in value]
                elif hasattr(value, '__dataclass_fields__'):
                    fields[name] = serialize_node(value)
                else:
                    fields[name] = value
            return {"type": type(node).__name__, **fields}
        return str(node)
    
    for stmt in ast.statements:
        result["statements"].append(serialize_node(stmt))
    
    return result


if __name__ == '__main__':
    main()

# prompt_dsl 使用示例

from prompt_dsl import compile, render

# 定义 Prompt 模板
source = '''
template "code_reviewer" version "1.0.0" {
    system: "You are an expert code reviewer."
    
    let language = "Python"
    let max_issues = 5
    
    "Review the following ${language} code for:
    1. Performance issues
    2. Security vulnerabilities
    3. Code style violations
    4. Potential bugs
    
    Priority: Show top ${max_issues} most critical issues.
    
    ${if is_security_focused}
        IMPORTANT: Pay extra attention to security concerns including:
        - SQL injection
        - XSS vulnerabilities
        - Authentication/Authorization issues
        - Data encryption
    ${endif}
    
    Code to review:
    ${code}
    
    ${foreach category in categories}
        ## ${category.name}
        
        ${foreach issue in category.issues}
            - [${issue.severity}] Line ${issue.line}: ${issue.description}
              Suggestion: ${issue.suggestion}
        ${endfor}
    ${endfor}
}
'''

# 编译模板
ast = compile(source)

# 定义变量
variables = {
    "code": '''
def get_user(user_id):
    query = f"SELECT * FROM users WHERE id = {user_id}"
    return execute_query(query)
''',
    "is_security_focused": True,
    "categories": [
        {
            "name": "Security",
            "issues": [
                {"severity": "HIGH", "line": 2, "description": "SQL Injection vulnerability", "suggestion": "Use parameterized queries"},
                {"severity": "MEDIUM", "line": 3, "description": "Missing error handling", "suggestion": "Add try-except block"}
            ]
        },
        {
            "name": "Style",
            "issues": [
                {"severity": "LOW", "line": 2, "description": "Missing type hints", "suggestion": "Add function type annotations"}
            ]
        }
    ]
}

# 渲染模板
result = render(ast, "code_reviewer", variables)

print(result)

Review the following Python code for:
1. Performance issues
2. Security vulnerabilities
3. Code style violations
4. Potential bugs

Priority: Show top 5 most critical issues.

IMPORTANT: Pay extra attention to security concerns including:
- SQL injection
- XSS vulnerabilities
- Authentication/Authorization issues
- Data encryption

Code to review:

def get_user(user_id):
    query = f"SELECT * FROM users WHERE id = {user_id}"
    return execute_query(query)
    

    ## Security
    
        - [HIGH] Line 2: SQL Injection vulnerability
          Suggestion: Use parameterized queries
        - [MEDIUM] Line 3: Missing error handling
          Suggestion: Add try-except block
    
    ## Style
    
        - [LOW] Line 2: Missing type hints
          Suggestion: Add function type annotations