feat(llm): provider-abstract gateway with cost and failure controls

DeepSeek (OpenAI-compatible) as primary with a 智谱 GLM slot that is a real
routing path but raises until the 算法备案 model list is confirmed. Adds scene
routing, failover, a per-provider circuit breaker, structured output with
retry-on-validation-failure, prompts loaded by id+version from prompts/, and
cost attribution written to ob_ai_call for every attempt -- successes,
retries and failures alike.

Two controls exist because they are cheap now and expensive later: a
cache-prefix validator that rejects timestamps, uuids and nicknames in the
cached system prefix (a volatile prefix silently multiplies input cost), and
a hard platform-wide daily budget that rejects non-critical scenes instead of
merely alerting, since a retry storm outruns any human on-call.

packages/llm/README.md

@@ -0,0 +1,54 @@
+# `packages/llm/` — LLM 网关
+
+架构 §4.3、§8.1。V1 是进程内包，V2 拆成独立服务。
+**因此接口必须按远程调用来设计**：无共享内存假设、明确超时、可序列化的入参出参。
+
+## 唯一入口
+
+所有 LLM 调用都走 `LLMGateway.complete()`。业务代码不许自己 `httpx.post` 到任何
+模型供应商 —— 那样会绕过成本归因、预算闸门、故障转移和 Prompt 版本记录，而这四件事
+一旦有缺口就补不回来（历史数据是补不出来的）。
+
+## 已实现
+
+| 能力 | 位置 |
+| --- | --- |
+| 场景化路由（`scene → provider/model/温度/超时/预算`，全配置化） | `routing.py` |
+| 故障转移 + 熔断（超时/5xx/限流切备用；连续失败熔断，冷却后半开探测） | `gateway.py` `circuit.py` |
+| 结构化输出（Pydantic Schema 约束 + 校验失败重试） | `gateway.py` |
+| Prompt 版本（文件化，按 id+version 加载，版本随每次调用落库） | `prompts.py` `/prompts` |
+| **缓存前缀纪律**（固定前缀含时间戳/随机 ID/昵称 → 拒绝或告警） | `prompts.py` |
+| 成本归因（每次 provider 调用写一行 `ob_ai_call`，含缓存命中 token） | `cost.py` `recording.py` |
+| **全局日成本硬熔断**（超预算即拒绝非关键场景，不是只告警） | `budget.py` |
+
+## 未实现（按 §8.1 还差）
+
+- **流式输出 SSE**（FR-AGT-01 要求首字 p95 < 2s，非流式做不到）
+- **审核联动**：输入输出审核应在网关内统一完成，业务方无需关心（CR-06）
+- **用户级频控**：目前只有平台级日预算与业务侧配额
+- **A/B 分流**：按用户哈希选择 prompt 版本
+- **智谱 GLM provider**：仅结构占位，见 `providers/zhipu.py`
+
+## 加一个新场景
+
+1. 在 `types.Scene` 加枚举值。
+2. 在 `routing._DEFAULTS` 加默认配置，想清楚 `critical` 该不该为真
+   —— `critical=True` 意味着日预算耗尽时它仍然会花钱。
+3. 在 `prompts/<scene>/v1.txt` 写模板。**`[system]` 段不许有任何变量**。
+4. 跑 `pytest tests/unit/test_prompt_cache_prefix.py`，它会扫描仓库里全部
+   prompt 的固定前缀。
+
+## 加一个新 provider
+
+1. 实现 `providers/base.LLMProvider` 协议（`name` / `default_model` /
+   `complete` / `aclose`）。
+2. HTTP 错误必须映射到 `errors.py` 的异常族，`retryable` 决定要不要切备用。
+   映射错了会导致「明明该切备用却没切」或者「明明是我们请求写错了却把备用也打一遍」。
+3. usage 字段必须映射出 **缓存命中 token 数**，否则 NFR-COST-03 的缓存命中率
+   看板是空的。
+4. 在 `apps/api/deps.build_llm_gateway()` 注册。
+
+## 费率
+
+`RateTable` 来自配置（`LLM_COST_RATES_JSON`），不是常量。未知模型走 default 费率
+而不是 0 —— 成本上的盲区比不精确更危险。**上线前必须用供应商实际价目表校准。**

packages/llm/__init__.py

@@ -0,0 +1,64 @@
+"""LLM gateway — 多供应商路由、Prompt 版本、结构化输出、成本计量、预算保护.
+
+架构文档 §4.3: V1 是进程内包，V2 拆为独立服务。接口按远程调用设计，拆分零成本。
+"""
+
+from packages.llm.budget import DailyBudgetGuard, InMemoryCostCounter, RedisCostCounter
+from packages.llm.circuit import CircuitBreaker, CircuitBreakerRegistry, CircuitState
+from packages.llm.cost import (
+    AICallRecord,
+    AICallRecorder,
+    CompositeAICallRecorder,
+    InMemoryAICallRecorder,
+    ModelRate,
+    NullAICallRecorder,
+    RateTable,
+    compute_cost_cents,
+)
+from packages.llm.errors import (
+    AllProvidersFailedError,
+    CachePrefixViolationError,
+    DailyBudgetExceededError,
+    LLMError,
+    ProviderError,
+    StructuredOutputError,
+)
+from packages.llm.gateway import LLMGateway
+from packages.llm.prompts import PromptRegistry, PromptTemplate, validate_cache_prefix
+from packages.llm.routing import SceneRouter, build_scene_configs
+from packages.llm.types import Completion, Message, PromptRef, Scene, SceneConfig, Usage
+
+__all__ = [
+    "AICallRecord",
+    "AICallRecorder",
+    "AllProvidersFailedError",
+    "CachePrefixViolationError",
+    "CircuitBreaker",
+    "CircuitBreakerRegistry",
+    "CircuitState",
+    "Completion",
+    "CompositeAICallRecorder",
+    "DailyBudgetExceededError",
+    "DailyBudgetGuard",
+    "InMemoryAICallRecorder",
+    "InMemoryCostCounter",
+    "LLMError",
+    "LLMGateway",
+    "Message",
+    "ModelRate",
+    "NullAICallRecorder",
+    "PromptRef",
+    "PromptRegistry",
+    "PromptTemplate",
+    "ProviderError",
+    "RateTable",
+    "RedisCostCounter",
+    "Scene",
+    "SceneConfig",
+    "SceneRouter",
+    "StructuredOutputError",
+    "Usage",
+    "build_scene_configs",
+    "compute_cost_cents",
+    "validate_cache_prefix",
+]

packages/llm/budget.py

@@ -0,0 +1,120 @@
+"""Global daily cost circuit breaker (架构文档 §13「必须设置的三道成本闸门」).
+
+Gate 1 in §13 is described as "某功能超预算自动降级，并告警". Alerting alone is
+not enough: the failure modes that blow up AI spend (retry storms, a cache-key
+bug, someone scripting the API) all produce cost faster than a human can react
+to a page. So this is an *enforcement* point, not a notification: once the
+platform-wide daily spend crosses the threshold, non-critical scenes are
+rejected outright.
+
+Critical scenes (e.g. content moderation follow-ups, paid-path completions)
+opt out via ``SceneConfig.critical`` so that a budget overrun degrades the
+product instead of breaking it.
+"""
+
+from __future__ import annotations
+
+from datetime import UTC, datetime, timedelta, timezone
+from decimal import Decimal
+from typing import Protocol, runtime_checkable
+
+from packages.llm.errors import DailyBudgetExceededError
+
+_DAY_SECONDS = 60 * 60 * 24
+
+
+@runtime_checkable
+class CostCounter(Protocol):
+    """Atomic accumulating counter. Redis in production, dict in tests."""
+
+    async def incr(self, key: str, amount: Decimal, ttl_seconds: int) -> Decimal: ...
+
+    async def get(self, key: str) -> Decimal: ...
+
+
+class InMemoryCostCounter:
+    """Process-local counter. Correct only for a single process — tests only."""
+
+    def __init__(self) -> None:
+        self._values: dict[str, Decimal] = {}
+
+    async def incr(self, key: str, amount: Decimal, ttl_seconds: int) -> Decimal:
+        self._values[key] = self._values.get(key, Decimal("0")) + amount
+        return self._values[key]
+
+    async def get(self, key: str) -> Decimal:
+        return self._values.get(key, Decimal("0"))
+
+
+class RedisCostCounter:
+    """Cross-process counter backed by ``INCRBYFLOAT`` + a rolling TTL."""
+
+    def __init__(self, redis: object) -> None:
+        self._redis = redis
+
+    async def incr(self, key: str, amount: Decimal, ttl_seconds: int) -> Decimal:
+        pipe = self._redis.pipeline()  # type: ignore[attr-defined]
+        pipe.incrbyfloat(key, float(amount))
+        pipe.expire(key, ttl_seconds)
+        results = await pipe.execute()
+        return Decimal(str(results[0]))
+
+    async def get(self, key: str) -> Decimal:
+        value = await self._redis.get(key)  # type: ignore[attr-defined]
+        return Decimal(str(value)) if value is not None else Decimal("0")
+
+
+class DailyBudgetGuard:
+    """Hard platform-wide daily AI budget."""
+
+    def __init__(
+        self,
+        counter: CostCounter,
+        *,
+        daily_budget_cents: Decimal,
+        enabled: bool = True,
+        key_prefix: str = "llm:cost:daily",
+        tz_offset_hours: int = 8,
+        clock: object | None = None,
+    ) -> None:
+        self._counter = counter
+        self._budget = daily_budget_cents
+        self._enabled = enabled
+        self._key_prefix = key_prefix
+        self._tz = timezone(timedelta(hours=tz_offset_hours))
+        self._clock = clock
+
+    @property
+    def enabled(self) -> bool:
+        return self._enabled
+
+    @property
+    def budget_cents(self) -> Decimal:
+        return self._budget
+
+    def _now(self) -> datetime:
+        if self._clock is not None:
+            return self._clock()  # type: ignore[operator,no-any-return]
+        return datetime.now(UTC)
+
+    def current_key(self) -> str:
+        """Budget day boundary follows UTC+8, consistent with FR-BILL-01."""
+        local = self._now().astimezone(self._tz)
+        return f"{self._key_prefix}:{local:%Y-%m-%d}"
+
+    async def spent_today(self) -> Decimal:
+        return await self._counter.get(self.current_key())
+
+    async def ensure_within_budget(self, *, scene: str, critical: bool = False) -> None:
+        """Raise :class:`DailyBudgetExceededError` for non-critical scenes over budget."""
+        if not self._enabled or critical:
+            return
+        spent = await self.spent_today()
+        if spent >= self._budget:
+            raise DailyBudgetExceededError(str(spent), str(self._budget), scene)
+
+    async def add_cost(self, cost_cents: Decimal) -> Decimal:
+        """Accumulate spend. Called after every provider attempt, success or not."""
+        if not self._enabled or cost_cents <= 0:
+            return await self.spent_today() if self._enabled else Decimal("0")
+        return await self._counter.incr(self.current_key(), cost_cents, _DAY_SECONDS)

packages/llm/circuit.py

@@ -0,0 +1,136 @@
+"""Circuit breaker for provider failover (架构文档 §8.1 故障转移).
+
+主供应商超时/5xx/限流 → 自动切备用；连续失败触发熔断，冷却后半开探测.
+
+The clock is injectable so tests do not sleep.
+"""
+
+from __future__ import annotations
+
+import time
+from collections.abc import Callable
+from enum import StrEnum
+
+
+class CircuitState(StrEnum):
+    CLOSED = "closed"
+    OPEN = "open"
+    HALF_OPEN = "half_open"
+
+
+class CircuitBreaker:
+    """Per-provider breaker.
+
+    closed  → normal traffic; ``failure_threshold`` consecutive failures open it.
+    open    → all calls rejected until ``cooldown_seconds`` elapse.
+    half_open → a limited number of probe calls; one success closes it, one
+                failure re-opens it and restarts the cooldown.
+    """
+
+    __slots__ = (
+        "_clock",
+        "_cooldown",
+        "_failures",
+        "_half_open_calls",
+        "_half_open_max",
+        "_opened_at",
+        "_state",
+        "_threshold",
+        "name",
+    )
+
+    def __init__(
+        self,
+        name: str,
+        *,
+        failure_threshold: int = 3,
+        cooldown_seconds: float = 30.0,
+        half_open_max_calls: int = 1,
+        clock: Callable[[], float] = time.monotonic,
+    ) -> None:
+        if failure_threshold < 1:
+            msg = "failure_threshold must be >= 1"
+            raise ValueError(msg)
+        self.name = name
+        self._threshold = failure_threshold
+        self._cooldown = cooldown_seconds
+        self._half_open_max = half_open_max_calls
+        self._clock = clock
+        self._state = CircuitState.CLOSED
+        self._failures = 0
+        self._opened_at = 0.0
+        self._half_open_calls = 0
+
+    @property
+    def state(self) -> CircuitState:
+        return self._state
+
+    @property
+    def consecutive_failures(self) -> int:
+        return self._failures
+
+    def allow(self) -> bool:
+        """Whether a call may be attempted right now. Mutates state on cooldown expiry."""
+        if self._state is CircuitState.CLOSED:
+            return True
+        if self._state is CircuitState.OPEN:
+            if self._clock() - self._opened_at >= self._cooldown:
+                self._state = CircuitState.HALF_OPEN
+                self._half_open_calls = 1
+                return True
+            return False
+        # HALF_OPEN
+        if self._half_open_calls < self._half_open_max:
+            self._half_open_calls += 1
+            return True
+        return False
+
+    def record_success(self) -> None:
+        self._state = CircuitState.CLOSED
+        self._failures = 0
+        self._half_open_calls = 0
+
+    def record_failure(self) -> None:
+        if self._state is CircuitState.HALF_OPEN:
+            self._trip()
+            return
+        self._failures += 1
+        if self._failures >= self._threshold:
+            self._trip()
+
+    def _trip(self) -> None:
+        self._state = CircuitState.OPEN
+        self._opened_at = self._clock()
+        self._half_open_calls = 0
+        self._failures = self._threshold
+
+
+class CircuitBreakerRegistry:
+    """One breaker per provider, created on first use."""
+
+    def __init__(
+        self,
+        *,
+        failure_threshold: int = 3,
+        cooldown_seconds: float = 30.0,
+        clock: Callable[[], float] = time.monotonic,
+    ) -> None:
+        self._breakers: dict[str, CircuitBreaker] = {}
+        self._failure_threshold = failure_threshold
+        self._cooldown_seconds = cooldown_seconds
+        self._clock = clock
+
+    def get(self, provider: str) -> CircuitBreaker:
+        breaker = self._breakers.get(provider)
+        if breaker is None:
+            breaker = CircuitBreaker(
+                provider,
+                failure_threshold=self._failure_threshold,
+                cooldown_seconds=self._cooldown_seconds,
+                clock=self._clock,
+            )
+            self._breakers[provider] = breaker
+        return breaker
+
+    def snapshot(self) -> dict[str, str]:
+        return {name: breaker.state.value for name, breaker in self._breakers.items()}

packages/llm/cost.py

@@ -0,0 +1,130 @@
+"""Cost attribution (架构文档 §13, FR-BILL-08, NFR-COST-01).
+
+**每一次付费调用都必须落 ``ob_ai_call``，并可归因到用户与功能。**
+V1 不收费让这张表的地位更高，不是更低 —— 它是 V2 定价的唯一数据来源，
+不允许「先上线后补埋点」。
+
+Rates are configuration, never constants at a call site: providers re-price
+without warning and V1's whole cost model depends on these numbers being
+correct at the time of the call.
+"""
+
+from __future__ import annotations
+
+from collections.abc import Mapping
+from dataclasses import dataclass, replace
+from datetime import UTC, datetime
+from decimal import ROUND_HALF_UP, Decimal
+from typing import Protocol, runtime_checkable
+
+from packages.llm.types import Usage
+
+_MILLION = Decimal("1000000")
+_QUANT = Decimal("0.000001")
+
+
+@dataclass(frozen=True, slots=True)
+class ModelRate:
+    """Cents per 1M tokens. Cached input is billed at a lower rate by DeepSeek."""
+
+    input_cents_per_mtok: Decimal
+    cached_input_cents_per_mtok: Decimal
+    output_cents_per_mtok: Decimal
+
+    @classmethod
+    def from_mapping(cls, data: Mapping[str, Decimal | str | int | float]) -> ModelRate:
+        return cls(
+            input_cents_per_mtok=Decimal(str(data["input_cents_per_mtok"])),
+            cached_input_cents_per_mtok=Decimal(str(data["cached_input_cents_per_mtok"])),
+            output_cents_per_mtok=Decimal(str(data["output_cents_per_mtok"])),
+        )
+
+
+class RateTable:
+    """model → :class:`ModelRate`, with an explicit fallback so an unknown model
+    is still costed (a zero-cost blind spot is worse than an approximate one)."""
+
+    def __init__(self, rates: dict[str, ModelRate], default: ModelRate) -> None:
+        self._rates = dict(rates)
+        self._default = default
+
+    def for_model(self, model: str) -> ModelRate:
+        return self._rates.get(model, self._default)
+
+    def known_models(self) -> tuple[str, ...]:
+        return tuple(sorted(self._rates))
+
+
+def compute_cost_cents(usage: Usage, rate: ModelRate) -> Decimal:
+    """Cost of one call, in cents, with cache-hit tokens billed separately."""
+    billable_input = Decimal(usage.billable_input) * rate.input_cents_per_mtok
+    cached_input = Decimal(usage.tokens_cached) * rate.cached_input_cents_per_mtok
+    output = Decimal(usage.tokens_out) * rate.output_cents_per_mtok
+    total = (billable_input + cached_input + output) / _MILLION
+    return total.quantize(_QUANT, rounding=ROUND_HALF_UP)
+
+
+@dataclass(frozen=True, slots=True)
+class AICallRecord:
+    """One row of ``ob_ai_call``. Written for successes *and* failures."""
+
+    call_id: int
+    scene: str
+    provider: str
+    model: str
+    success: bool
+    usage: Usage
+    latency_ms: int
+    cost_cents: Decimal
+    user_id: int | None = None
+    prompt_id: str | None = None
+    prompt_version: str | None = None
+    error_code: str | None = None
+    error_message: str | None = None
+    ref_type: str | None = None
+    ref_id: str | None = None
+    created_at: datetime | None = None
+
+    def with_defaults(self) -> AICallRecord:
+        if self.created_at is not None:
+            return self
+        return replace(self, created_at=datetime.now(UTC))
+
+
+@runtime_checkable
+class AICallRecorder(Protocol):
+    """Persistence port. The gateway never touches the ORM directly."""
+
+    async def record(self, record: AICallRecord) -> None: ...
+
+
+class NullAICallRecorder:
+    """Explicitly opts out of cost attribution. Only valid in unit tests."""
+
+    async def record(self, record: AICallRecord) -> None:
+        return None
+
+
+class InMemoryAICallRecorder:
+    """Test double that keeps every record for assertions."""
+
+    def __init__(self) -> None:
+        self.records: list[AICallRecord] = []
+
+    async def record(self, record: AICallRecord) -> None:
+        self.records.append(record)
+
+
+class CompositeAICallRecorder:
+    """Fan out to several recorders (e.g. PostgreSQL + Prometheus).
+
+    A failure in one recorder must not lose the others, and must never fail the
+    user's request — cost bookkeeping is mandatory but not user-facing.
+    """
+
+    def __init__(self, *recorders: AICallRecorder) -> None:
+        self._recorders = recorders
+
+    async def record(self, record: AICallRecord) -> None:
+        for recorder in self._recorders:
+            await recorder.record(record)

packages/llm/errors.py

@@ -0,0 +1,146 @@
+"""LLM gateway error taxonomy.
+
+``retryable`` means "worth failing over to the backup provider and counting
+towards the circuit breaker". A malformed request is *not* retryable — sending
+it to the backup would just burn money twice.
+"""
+
+from __future__ import annotations
+
+
+class LLMError(Exception):
+    """Base class for every gateway-level failure."""
+
+    code = "llm_error"
+
+
+class ProviderError(LLMError):
+    """A single provider call failed."""
+
+    code = "llm_provider_error"
+    retryable = True
+
+    def __init__(
+        self,
+        provider: str,
+        message: str,
+        *,
+        status_code: int | None = None,
+    ) -> None:
+        self.provider = provider
+        self.status_code = status_code
+        super().__init__(f"{provider}: {message}")
+
+
+class ProviderTimeoutError(ProviderError):
+    code = "llm_timeout"
+    retryable = True
+
+
+class ProviderRateLimitedError(ProviderError):
+    code = "llm_rate_limited"
+    retryable = True
+
+
+class ProviderServerError(ProviderError):
+    """5xx from the provider."""
+
+    code = "llm_server_error"
+    retryable = True
+
+
+class ProviderAuthError(ProviderError):
+    """401/403 — the key for this provider is wrong; the backup may still work."""
+
+    code = "llm_auth_error"
+    retryable = True
+
+
+class ProviderBadRequestError(ProviderError):
+    """4xx that the backup provider would reject too. Do not fail over."""
+
+    code = "llm_bad_request"
+    retryable = False
+
+
+class ProviderNotImplementedError(ProviderError, NotImplementedError):
+    """Structural placeholder for a provider slot that has no implementation yet."""
+
+    code = "llm_provider_not_implemented"
+    retryable = True
+
+
+class CircuitOpenError(ProviderError):
+    """The breaker for this provider is open; the call was not attempted."""
+
+    code = "llm_circuit_open"
+    retryable = True
+
+
+class StructuredOutputError(LLMError):
+    """The model could not produce output matching the requested schema."""
+
+    code = "llm_structured_output_failed"
+
+    def __init__(self, provider: str, schema_name: str, attempts: int, detail: str) -> None:
+        self.provider = provider
+        self.schema_name = schema_name
+        self.attempts = attempts
+        super().__init__(
+            f"{provider}: failed to satisfy schema {schema_name} "
+            f"after {attempts} attempts: {detail}"
+        )
+
+
+class AllProvidersFailedError(LLMError):
+    """Every provider in the scene's routing chain failed."""
+
+    code = "llm_all_providers_failed"
+
+    def __init__(self, scene: str, failures: list[Exception]) -> None:
+        self.scene = scene
+        self.failures = failures
+        detail = "; ".join(f"{type(e).__name__}: {e}" for e in failures) or "no providers routed"
+        super().__init__(f"scene={scene} all providers failed: {detail}")
+
+
+class CachePrefixViolationError(LLMError):
+    """The cacheable system prefix contains volatile tokens.
+
+    架构文档 §7.2 缓存前缀纪律 / TD-04: 系统提示词含时间戳、随机 ID、用户昵称
+    会让 prompt 缓存永远不命中，输入成本翻数倍。
+    """
+
+    code = "llm_prompt_cache_violation"
+
+    def __init__(self, prompt_ref: str, violations: list[str]) -> None:
+        self.prompt_ref = prompt_ref
+        self.violations = violations
+        super().__init__(f"{prompt_ref}: volatile tokens in cached prefix: {', '.join(violations)}")
+
+
+class DailyBudgetExceededError(LLMError):
+    """Platform-wide daily AI spend crossed the hard threshold.
+
+    架构文档 §13 三道成本闸门. Non-critical scenes are rejected/degraded here
+    rather than merely alerted — an alert does not stop a retry storm.
+    """
+
+    code = "llm_daily_budget_exceeded"
+
+    def __init__(self, spent_cents: str, budget_cents: str, scene: str) -> None:
+        self.spent_cents = spent_cents
+        self.budget_cents = budget_cents
+        self.scene = scene
+        super().__init__(
+            f"daily AI budget exhausted (spent={spent_cents} budget={budget_cents}); "
+            f"non-critical scene '{scene}' rejected"
+        )
+
+
+class PromptNotFoundError(LLMError):
+    code = "llm_prompt_not_found"
+
+
+class SceneNotRoutedError(LLMError):
+    code = "llm_scene_not_routed"

packages/llm/gateway.py

@@ -0,0 +1,355 @@
+"""The LLM gateway (架构文档 §8.1).
+
+Built-in capabilities, all exercised on every call:
+
+===================  =========================================================
+场景化路由            ``scene → (provider, model, temperature, max_tokens, timeout)``
+故障转移 + 熔断        primary 超时/5xx/限流 → 自动切备用；连续失败熔断，冷却后半开
+结构化输出            传入 Pydantic Schema → JSON 约束 + 解析失败重试 + 校验
+Prompt 版本           模板文件按 id+version 加载，版本号随每次调用落库
+缓存前缀纪律          固定前缀含时间戳/随机 ID/昵称 → 拒绝或告警 (§7.2, TD-04)
+成本计量              每次 provider 调用（含失败与重试）写一行 ``ob_ai_call``
+预算保护              平台级日成本硬熔断：超限即拒绝非关键场景 (§13)
+===================  =========================================================
+
+Deliberately not here yet: streaming (SSE), 审核联动 (input/output moderation),
+per-user rate limiting, A/B prompt split by user hash. See packages/llm/README.md.
+"""
+
+from __future__ import annotations
+
+import re
+import time
+from collections.abc import Callable, Mapping, Sequence
+from decimal import Decimal
+
+from pydantic import BaseModel, ValidationError
+
+from packages.domain.ids import next_id
+from packages.llm.budget import DailyBudgetGuard
+from packages.llm.circuit import CircuitBreakerRegistry
+from packages.llm.cost import AICallRecord, AICallRecorder, RateTable, compute_cost_cents
+from packages.llm.errors import (
+    AllProvidersFailedError,
+    CircuitOpenError,
+    ProviderError,
+    SceneNotRoutedError,
+    StructuredOutputError,
+)
+from packages.llm.prompts import PromptRegistry, validate_cache_prefix
+from packages.llm.providers.base import LLMProvider
+from packages.llm.routing import SceneRouter
+from packages.llm.types import (
+    Completion,
+    Message,
+    ProviderRequest,
+    Scene,
+    SceneConfig,
+    Usage,
+)
+from packages.shared.logging import get_logger
+
+logger = get_logger(__name__)
+
+_JSON_FENCE_RE = re.compile(r"```(?:json)?\s*(.*?)```", re.DOTALL)
+_STRUCTURED_INVALID = "llm_structured_output_invalid"
+
+
+def extract_json(text: str) -> str:
+    """Pull the JSON object out of a model reply that may be fenced or chatty."""
+    match = _JSON_FENCE_RE.search(text)
+    candidate = (match.group(1) if match else text).strip()
+    start, end = candidate.find("{"), candidate.rfind("}")
+    if start != -1 and end > start:
+        return candidate[start : end + 1]
+    return candidate
+
+
+class LLMGateway:
+    """Single entry point for every LLM call in the platform."""
+
+    def __init__(
+        self,
+        *,
+        providers: Mapping[str, LLMProvider],
+        router: SceneRouter,
+        prompts: PromptRegistry,
+        rates: RateTable,
+        recorder: AICallRecorder,
+        budget: DailyBudgetGuard,
+        breakers: CircuitBreakerRegistry | None = None,
+        id_factory: Callable[[], int] = next_id,
+        timer: Callable[[], float] = time.perf_counter,
+    ) -> None:
+        self._providers = dict(providers)
+        self._router = router
+        self._prompts = prompts
+        self._rates = rates
+        self._recorder = recorder
+        self._budget = budget
+        self._breakers = breakers or CircuitBreakerRegistry()
+        self._id_factory = id_factory
+        self._timer = timer
+
+    # -- public API ---------------------------------------------------------
+
+    async def complete(
+        self,
+        *,
+        scene: Scene,
+        messages: Sequence[Message] = (),
+        prompt_vars: Mapping[str, str] | None = None,
+        schema: type[BaseModel] | None = None,
+        user_id: int | None = None,
+        ref_type: str | None = None,
+        ref_id: str | None = None,
+    ) -> Completion:
+        config = self._router.resolve(scene)
+
+        # Gate 1: platform-wide daily cost breaker, checked before any spend.
+        await self._budget.ensure_within_budget(scene=scene.value, critical=config.critical)
+
+        template = self._prompts.get(config.prompt_id, config.prompt_version)
+        violations = validate_cache_prefix(
+            template.system_prefix, mode=self._prompts.cache_prefix_mode, ref=template.ref
+        )
+        if violations:
+            logger.warning(
+                "llm_prompt_cache_prefix_violation",
+                prompt=template.ref,
+                violations=[str(v) for v in violations],
+            )
+
+        composed = self._compose(
+            system_prefix=template.system_prefix,
+            user_text=template.render_user(dict(prompt_vars or {})),
+            messages=messages,
+            schema=schema,
+        )
+
+        failures: list[Exception] = []
+        for provider_name in config.providers:
+            provider = self._providers.get(provider_name)
+            if provider is None:
+                failures.append(
+                    SceneNotRoutedError(f"provider '{provider_name}' is not registered")
+                )
+                continue
+
+            breaker = self._breakers.get(provider_name)
+            if not breaker.allow():
+                logger.warning(
+                    "llm_circuit_open", provider=provider_name, state=breaker.state.value
+                )
+                failures.append(CircuitOpenError(provider_name, f"circuit {breaker.state.value}"))
+                continue
+
+            try:
+                completion = await self._call_provider(
+                    provider=provider,
+                    config=config,
+                    messages=composed,
+                    schema=schema,
+                    user_id=user_id,
+                    prompt_id=template.prompt_id,
+                    prompt_version=template.version,
+                    ref_type=ref_type,
+                    ref_id=ref_id,
+                )
+            except ProviderError as exc:
+                if not exc.retryable:
+                    # A malformed request would fail on the backup too.
+                    raise
+                breaker.record_failure()
+                failures.append(exc)
+                logger.warning(
+                    "llm_provider_failed",
+                    provider=provider_name,
+                    scene=scene.value,
+                    error_code=exc.code,
+                    circuit=breaker.state.value,
+                )
+                continue
+            except StructuredOutputError as exc:
+                breaker.record_failure()
+                failures.append(exc)
+                continue
+
+            breaker.record_success()
+            return completion
+
+        raise AllProvidersFailedError(scene.value, failures)
+
+    def breaker_states(self) -> dict[str, str]:
+        return self._breakers.snapshot()
+
+    async def aclose(self) -> None:
+        for provider in self._providers.values():
+            await provider.aclose()
+
+    # -- internals ----------------------------------------------------------
+
+    @staticmethod
+    def _compose(
+        *,
+        system_prefix: str,
+        user_text: str,
+        messages: Sequence[Message],
+        schema: type[BaseModel] | None,
+    ) -> list[Message]:
+        """架构文档 §7.2 ③: 固定前缀放最前以命中缓存，变量部分一律排在后面。"""
+        composed: list[Message] = [Message(role="system", content=system_prefix)]
+        composed.extend(messages)
+        if user_text:
+            composed.append(Message(role="user", content=user_text))
+        if schema is not None:
+            composed.append(Message(role="user", content=_schema_instruction(schema)))
+        return composed
+
+    async def _call_provider(
+        self,
+        *,
+        provider: LLMProvider,
+        config: SceneConfig,
+        messages: Sequence[Message],
+        schema: type[BaseModel] | None,
+        user_id: int | None,
+        prompt_id: str,
+        prompt_version: str,
+        ref_type: str | None,
+        ref_id: str | None,
+    ) -> Completion:
+        model = config.model_for(provider.name, provider.default_model)
+        conversation: list[Message] = list(messages)
+        max_attempts = config.structured_max_retries + 1 if schema is not None else 1
+        schema_name = schema.__name__ if schema is not None else "<none>"
+        last_detail = "no attempt was made"
+
+        for attempt in range(1, max_attempts + 1):
+            request = ProviderRequest(
+                model=model,
+                messages=tuple(conversation),
+                temperature=config.temperature,
+                max_tokens=config.max_tokens,
+                timeout_seconds=config.timeout_seconds,
+                json_mode=schema is not None,
+            )
+            call_id = self._id_factory()
+            started = self._timer()
+
+            try:
+                response = await provider.complete(request)
+            except ProviderError as exc:
+                await self._write_record(
+                    AICallRecord(
+                        call_id=call_id,
+                        scene=config.scene.value,
+                        provider=provider.name,
+                        model=model,
+                        success=False,
+                        usage=Usage(),
+                        latency_ms=self._elapsed_ms(started),
+                        cost_cents=Decimal("0"),
+                        user_id=user_id,
+                        prompt_id=prompt_id,
+                        prompt_version=prompt_version,
+                        error_code=exc.code,
+                        error_message=str(exc)[:1000],
+                        ref_type=ref_type,
+                        ref_id=ref_id,
+                    )
+                )
+                raise
+
+            latency_ms = self._elapsed_ms(started)
+            cost = compute_cost_cents(response.usage, self._rates.for_model(response.model))
+
+            parsed: BaseModel | None = None
+            success = True
+            error_code: str | None = None
+            if schema is not None:
+                try:
+                    parsed = schema.model_validate_json(extract_json(response.text))
+                except (ValidationError, ValueError) as exc:
+                    success = False
+                    error_code = _STRUCTURED_INVALID
+                    last_detail = str(exc)[:400]
+
+            await self._write_record(
+                AICallRecord(
+                    call_id=call_id,
+                    scene=config.scene.value,
+                    provider=provider.name,
+                    model=response.model,
+                    success=success,
+                    usage=response.usage,
+                    latency_ms=latency_ms,
+                    cost_cents=cost,
+                    user_id=user_id,
+                    prompt_id=prompt_id,
+                    prompt_version=prompt_version,
+                    error_code=error_code,
+                    error_message=last_detail if not success else None,
+                    ref_type=ref_type,
+                    ref_id=ref_id,
+                )
+            )
+            # Budget accrues on every attempt, including ones we then discard —
+            # a validation-retry loop costs real money and must be visible.
+            await self._budget.add_cost(cost)
+
+            if success:
+                return Completion(
+                    text=response.text,
+                    scene=config.scene,
+                    provider=provider.name,
+                    model=response.model,
+                    prompt_id=prompt_id,
+                    prompt_version=prompt_version,
+                    usage=response.usage,
+                    latency_ms=latency_ms,
+                    cost_cents=cost,
+                    call_id=call_id,
+                    parsed=parsed,
+                    attempts=attempt,
+                )
+
+            logger.warning(
+                "llm_structured_output_invalid",
+                provider=provider.name,
+                scene=config.scene.value,
+                schema=schema_name,
+                attempt=attempt,
+            )
+            conversation.append(Message(role="assistant", content=response.text))
+            conversation.append(Message(role="user", content=_repair_instruction(last_detail)))
+
+        raise StructuredOutputError(provider.name, schema_name, max_attempts, last_detail)
+
+    def _elapsed_ms(self, started: float) -> int:
+        return max(int((self._timer() - started) * 1000), 0)
+
+    async def _write_record(self, record: AICallRecord) -> None:
+        """Cost attribution is mandatory but must never break the user's request."""
+        try:
+            await self._recorder.record(record.with_defaults())
+        except Exception:
+            logger.exception("ai_call_record_failed", call_id=record.call_id, scene=record.scene)
+
+
+def _schema_instruction(schema: type[BaseModel]) -> str:
+    import json
+
+    return (
+        "请只输出一个 JSON 对象，不要包含解释文字或代码块标记。"
+        "该对象必须严格符合以下 JSON Schema：\n"
+        f"{json.dumps(schema.model_json_schema(), ensure_ascii=False)}"
+    )
+
+
+def _repair_instruction(detail: str) -> str:
+    return (
+        "上一次输出不符合要求的 JSON Schema，校验错误如下：\n"
+        f"{detail}\n"
+        "请重新只输出修正后的 JSON 对象，不要包含任何其他文字。"
+    )

packages/llm/prompts.py

@@ -0,0 +1,185 @@
+"""Versioned prompt templates + prompt-cache prefix discipline.
+
+架构文档 §8.1 Prompt 版本: Prompt 存文件（随代码版本化）+ 场景绑定版本号.
+架构文档 §7.2 缓存前缀纪律 / TD-04:
+
+    系统提示词与稳定知识片段必须放在 prompt 头部且**不含时间戳、随机 ID、
+    用户昵称等变量**，否则缓存永远不命中，输入成本翻数倍。
+
+That rule is unenforceable by review alone — every prompt edit is a chance to
+break it, and the damage (a silent 5-10x input-cost increase) is invisible in
+functional tests. So it is enforced in code: :func:`validate_cache_prefix` runs
+on every gateway call.
+
+File layout::
+
+    prompts/<prompt_id>/<version>.txt
+
+    [system]
+    ...fixed, cacheable prefix — no variables at all...
+
+    [user]
+    ...template rendered with str.format(**vars)...
+"""
+
+from __future__ import annotations
+
+import re
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Literal
+
+from packages.llm.errors import CachePrefixViolationError, PromptNotFoundError
+
+CachePrefixMode = Literal["strict", "warn", "off"]
+
+_SECTION_RE = re.compile(r"^\[(system|user)\]\s*$", re.MULTILINE)
+
+#: Patterns whose presence in a cached prefix destroys the provider prompt cache.
+_VOLATILE_PATTERNS: tuple[tuple[str, re.Pattern[str]], ...] = (
+    ("iso-date", re.compile(r"\b\d{4}-\d{2}-\d{2}\b")),
+    ("iso-time", re.compile(r"\b\d{2}:\d{2}:\d{2}\b")),
+    ("cn-date", re.compile(r"\b\d{4}\s*年\s*\d{1,2}\s*月")),
+    ("epoch-seconds", re.compile(r"\b1[6-9]\d{8}\b")),
+    (
+        "uuid",
+        re.compile(
+            r"\b[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-"
+            r"[0-9a-fA-F]{4}-[0-9a-fA-F]{12}\b"
+        ),
+    ),
+    ("long-hex-id", re.compile(r"\b[0-9a-f]{24,}\b")),
+    (
+        "volatile-placeholder",
+        re.compile(
+            r"\{[^{}]*(?:nick|user|name|now|today|date|time|stamp|uuid|rand|"
+            r"session|request|trace|token|id)[^{}]*\}",
+            re.IGNORECASE,
+        ),
+    ),
+)
+
+
+@dataclass(frozen=True, slots=True)
+class CachePrefixViolation:
+    kind: str
+    sample: str
+
+    def __str__(self) -> str:
+        return f"{self.kind}({self.sample!r})"
+
+
+def find_volatile_tokens(prefix: str) -> list[CachePrefixViolation]:
+    """Return every volatile token found in a would-be cached prefix."""
+    violations: list[CachePrefixViolation] = []
+    for kind, pattern in _VOLATILE_PATTERNS:
+        for match in pattern.finditer(prefix):
+            violations.append(CachePrefixViolation(kind=kind, sample=match.group(0)))
+    return violations
+
+
+def validate_cache_prefix(
+    prefix: str,
+    *,
+    mode: CachePrefixMode = "strict",
+    ref: str = "<inline>",
+    on_warning: object | None = None,
+) -> list[CachePrefixViolation]:
+    """Enforce the cache-prefix discipline.
+
+    ``strict`` raises :class:`CachePrefixViolationError`; ``warn`` returns the
+    violations for the caller to log; ``off`` skips the check entirely.
+    """
+    if mode == "off":
+        return []
+    violations = find_volatile_tokens(prefix)
+    if violations and mode == "strict":
+        raise CachePrefixViolationError(ref, [str(v) for v in violations])
+    return violations
+
+
+@dataclass(frozen=True, slots=True)
+class PromptTemplate:
+    prompt_id: str
+    version: str
+    #: Fixed, cacheable prefix. Placed first in the message list.
+    system_prefix: str
+    #: Optional variable part, rendered per call.
+    user_template: str = ""
+
+    @property
+    def ref(self) -> str:
+        return f"{self.prompt_id}@{self.version}"
+
+    def render_user(self, variables: dict[str, str] | None = None) -> str:
+        if not self.user_template:
+            return ""
+        try:
+            return self.user_template.format(**(variables or {}))
+        except KeyError as exc:
+            msg = f"{self.ref}: missing prompt variable {exc.args[0]!r}"
+            raise PromptNotFoundError(msg) from exc
+
+
+def parse_prompt_file(text: str, *, prompt_id: str, version: str) -> PromptTemplate:
+    sections: dict[str, str] = {}
+    parts = _SECTION_RE.split(text)
+    # parts == [preamble, name, body, name, body, ...]
+    for name, body in zip(parts[1::2], parts[2::2], strict=True):
+        sections[name] = body.strip()
+    if "system" not in sections:
+        msg = f"{prompt_id}@{version}: prompt file must contain a [system] section"
+        raise PromptNotFoundError(msg)
+    return PromptTemplate(
+        prompt_id=prompt_id,
+        version=version,
+        system_prefix=sections["system"],
+        user_template=sections.get("user", ""),
+    )
+
+
+class PromptRegistry:
+    """Loads and caches versioned prompt templates from disk."""
+
+    def __init__(
+        self,
+        root: Path | str,
+        *,
+        cache_prefix_mode: CachePrefixMode = "strict",
+        suffixes: tuple[str, ...] = (".txt", ".md"),
+    ) -> None:
+        self._root = Path(root)
+        self._mode: CachePrefixMode = cache_prefix_mode
+        self._suffixes = suffixes
+        self._cache: dict[tuple[str, str], PromptTemplate] = {}
+
+    @property
+    def cache_prefix_mode(self) -> CachePrefixMode:
+        return self._mode
+
+    def get(self, prompt_id: str, version: str) -> PromptTemplate:
+        key = (prompt_id, version)
+        cached = self._cache.get(key)
+        if cached is not None:
+            return cached
+        template = self._load(prompt_id, version)
+        # Fail fast at load time as well as call time.
+        validate_cache_prefix(template.system_prefix, mode=self._mode, ref=template.ref)
+        self._cache[key] = template
+        return template
+
+    def _load(self, prompt_id: str, version: str) -> PromptTemplate:
+        for suffix in self._suffixes:
+            path = self._root / prompt_id / f"{version}{suffix}"
+            if path.is_file():
+                return parse_prompt_file(
+                    path.read_text(encoding="utf-8"), prompt_id=prompt_id, version=version
+                )
+        msg = f"prompt {prompt_id}@{version} not found under {self._root}"
+        raise PromptNotFoundError(msg)
+
+    def available_versions(self, prompt_id: str) -> list[str]:
+        directory = self._root / prompt_id
+        if not directory.is_dir():
+            return []
+        return sorted(p.stem for p in directory.iterdir() if p.suffix in self._suffixes)

packages/llm/providers/__init__.py

@@ -0,0 +1 @@
+"""LLM 供应商适配器。DeepSeek 为主，智谱 GLM 为备。"""

packages/llm/providers/base.py

@@ -0,0 +1,34 @@
+"""Provider abstraction.
+
+架构文档 §4.3 requires the gateway interface to be designed as if it were a
+remote call, so providers take a fully-specified request (explicit timeout,
+serialisable payload) and return a fully-specified response. No provider is
+allowed to reach back into gateway state.
+"""
+
+from __future__ import annotations
+
+from typing import Protocol, runtime_checkable
+
+from packages.llm.types import ProviderRequest, ProviderResponse
+
+
+@runtime_checkable
+class LLMProvider(Protocol):
+    @property
+    def name(self) -> str:
+        """Stable identifier used in routing config and in ``ob_ai_call``."""
+        ...
+
+    @property
+    def default_model(self) -> str:
+        """Model used when the scene config does not pin one."""
+        ...
+
+    async def complete(self, request: ProviderRequest) -> ProviderResponse:
+        """Perform one completion. Must raise ``packages.llm.errors.ProviderError``."""
+        ...
+
+    async def aclose(self) -> None:
+        """Release connections."""
+        ...

packages/llm/providers/deepseek.py

@@ -0,0 +1,137 @@
+"""DeepSeek provider — primary LLM supplier (TD-04, 架构文档 §5).
+
+DeepSeek exposes an OpenAI-compatible API at ``https://api.deepseek.com/v1``.
+Its usage block reports ``prompt_cache_hit_tokens`` / ``prompt_cache_miss_tokens``,
+which is what makes the cache-prefix discipline in :mod:`packages.llm.prompts`
+measurable (NFR-COST-03).
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+import httpx
+
+from packages.llm.errors import (
+    ProviderAuthError,
+    ProviderBadRequestError,
+    ProviderError,
+    ProviderRateLimitedError,
+    ProviderServerError,
+    ProviderTimeoutError,
+)
+from packages.llm.types import ProviderRequest, ProviderResponse, Usage
+
+
+class DeepSeekProvider:
+    """OpenAI-compatible chat-completions client."""
+
+    def __init__(
+        self,
+        *,
+        api_key: str,
+        base_url: str = "https://api.deepseek.com/v1",
+        default_model: str = "deepseek-v4-flash",
+        client: httpx.AsyncClient | None = None,
+        name: str = "deepseek",
+    ) -> None:
+        self._api_key = api_key
+        self._base_url = base_url.rstrip("/")
+        self._default_model = default_model
+        self._client = client or httpx.AsyncClient(timeout=httpx.Timeout(60.0))
+        self._name = name
+
+    @property
+    def name(self) -> str:
+        return self._name
+
+    @property
+    def default_model(self) -> str:
+        return self._default_model
+
+    def _payload(self, request: ProviderRequest) -> dict[str, Any]:
+        payload: dict[str, Any] = {
+            "model": request.model,
+            "messages": [{"role": m.role, "content": m.content} for m in request.messages],
+            "temperature": request.temperature,
+            "max_tokens": request.max_tokens,
+            "stream": request.stream,
+        }
+        if request.json_mode:
+            payload["response_format"] = {"type": "json_object"}
+        return payload
+
+    async def complete(self, request: ProviderRequest) -> ProviderResponse:
+        url = f"{self._base_url}/chat/completions"
+        headers = {
+            "Authorization": f"Bearer {self._api_key}",
+            "Content-Type": "application/json",
+        }
+        try:
+            response = await self._client.post(
+                url,
+                json=self._payload(request),
+                headers=headers,
+                timeout=request.timeout_seconds,
+            )
+        except httpx.TimeoutException as exc:
+            raise ProviderTimeoutError(
+                self._name, f"request timed out after {request.timeout_seconds}s"
+            ) from exc
+        except httpx.HTTPError as exc:
+            raise ProviderServerError(self._name, f"transport error: {exc}") from exc
+
+        self._raise_for_status(response)
+        return self._parse(response.json(), fallback_model=request.model)
+
+    def _raise_for_status(self, response: httpx.Response) -> None:
+        status = response.status_code
+        if status < 400:
+            return
+        detail = _short_body(response)
+        if status == 429:
+            raise ProviderRateLimitedError(self._name, detail, status_code=status)
+        if status in (401, 403):
+            raise ProviderAuthError(self._name, detail, status_code=status)
+        if status >= 500:
+            raise ProviderServerError(self._name, detail, status_code=status)
+        raise ProviderBadRequestError(self._name, detail, status_code=status)
+
+    def _parse(self, data: dict[str, Any], *, fallback_model: str) -> ProviderResponse:
+        try:
+            choice = data["choices"][0]
+            text = choice["message"]["content"] or ""
+        except (KeyError, IndexError, TypeError) as exc:
+            raise ProviderServerError(self._name, f"malformed response: {data!r}") from exc
+
+        usage_raw: dict[str, Any] = data.get("usage") or {}
+        cached = int(
+            usage_raw.get("prompt_cache_hit_tokens")
+            or (usage_raw.get("prompt_tokens_details") or {}).get("cached_tokens")
+            or 0
+        )
+        usage = Usage(
+            tokens_in=int(usage_raw.get("prompt_tokens", 0)),
+            tokens_out=int(usage_raw.get("completion_tokens", 0)),
+            tokens_cached=cached,
+        )
+        return ProviderResponse(
+            text=text,
+            model=str(data.get("model") or fallback_model),
+            usage=usage,
+            finish_reason=choice.get("finish_reason"),
+        )
+
+    async def aclose(self) -> None:
+        await self._client.aclose()
+
+
+def _short_body(response: httpx.Response, limit: int = 300) -> str:
+    try:
+        body = response.text
+    except (UnicodeDecodeError, httpx.ResponseNotRead):  # pragma: no cover
+        return f"HTTP {response.status_code}"
+    return f"HTTP {response.status_code}: {body[:limit]}"
+
+
+__all__ = ["DeepSeekProvider", "ProviderError"]

packages/llm/providers/zhipu.py

@@ -0,0 +1,57 @@
+"""智谱 GLM provider — backup supplier slot (TD-04, 架构文档 §2 降级表).
+
+**Structure only.** The routing/failover code path is real: the gateway will
+route to this provider, the breaker will count its failures, and ``ob_ai_call``
+will record the attempt. Only the HTTP call is missing.
+
+Before implementing, resolve the open compliance item in 需求规格说明 §5.3:
+「备案覆盖的模型清单核实 —— 若备案未覆盖 DeepSeek + GLM 两家，双路由属超范围」.
+Shipping a working second route ahead of that check is a compliance risk, not a
+resilience win.
+"""
+
+from __future__ import annotations
+
+from packages.llm.errors import ProviderNotImplementedError
+from packages.llm.types import ProviderRequest, ProviderResponse
+
+_NOT_IMPLEMENTED_MESSAGE = (
+    "智谱 GLM provider is a structural placeholder and has no implementation. "
+    "To enable it: (1) confirm the 算法备案 model list covers GLM "
+    "(需求规格说明 §5.3), (2) implement complete() against "
+    "POST {base_url}/chat/completions, (3) map GLM usage fields onto "
+    "packages.llm.types.Usage, (4) set ZHIPU_API_KEY. "
+    "Until then the gateway treats this slot as a permanently failing provider."
+)
+
+
+class ZhipuGLMProvider:
+    def __init__(
+        self,
+        *,
+        api_key: str = "",
+        base_url: str = "https://open.bigmodel.cn/api/paas/v4",
+        default_model: str = "glm-4-flash",
+        name: str = "zhipu",
+    ) -> None:
+        self._api_key = api_key
+        self._base_url = base_url.rstrip("/")
+        self._default_model = default_model
+        self._name = name
+
+    @property
+    def name(self) -> str:
+        return self._name
+
+    @property
+    def default_model(self) -> str:
+        return self._default_model
+
+    async def complete(self, request: ProviderRequest) -> ProviderResponse:
+        raise ProviderNotImplementedError(self._name, _NOT_IMPLEMENTED_MESSAGE)
+
+    async def aclose(self) -> None:
+        return None
+
+
+__all__ = ["ZhipuGLMProvider"]

packages/llm/recording.py

@@ -0,0 +1,61 @@
+"""Adapters that persist :class:`packages.llm.cost.AICallRecord`.
+
+The SQLAlchemy recorder deliberately opens its **own** session instead of
+joining the caller's transaction. If a request later rolls back — quota refund,
+moderation rejection, an unhandled error — the cost still happened and must
+still be attributable. Losing those rows would silently under-report spend,
+which is exactly the blind spot 架构文档 §13 forbids.
+"""
+
+from __future__ import annotations
+
+from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
+
+from packages.llm.cost import AICallRecord
+from packages.shared.models import AiCall
+from packages.shared.observability import LLM_CALLS, LLM_COST_CENTS, LLM_LATENCY, LLM_TOKENS
+
+
+class SqlAlchemyAICallRecorder:
+    """Writes one ``ob_ai_call`` row per provider attempt."""
+
+    def __init__(self, session_factory: async_sessionmaker[AsyncSession]) -> None:
+        self._session_factory = session_factory
+
+    async def record(self, record: AICallRecord) -> None:
+        async with self._session_factory() as session:
+            session.add(
+                AiCall(
+                    call_id=record.call_id,
+                    user_id=record.user_id,
+                    scene=record.scene,
+                    provider=record.provider,
+                    model=record.model,
+                    prompt_id=record.prompt_id,
+                    prompt_version=record.prompt_version,
+                    tokens_in=record.usage.tokens_in,
+                    tokens_out=record.usage.tokens_out,
+                    tokens_cached=record.usage.tokens_cached,
+                    latency_ms=record.latency_ms,
+                    cost_cents=record.cost_cents,
+                    success=record.success,
+                    error_code=record.error_code,
+                    error_message=record.error_message,
+                    ref_type=record.ref_type,
+                    ref_id=record.ref_id,
+                )
+            )
+            await session.commit()
+
+
+class PrometheusAICallRecorder:
+    """Feeds the AI 成本看板 (架构文档 §15) straight from the same records."""
+
+    async def record(self, record: AICallRecord) -> None:
+        labels = (record.scene, record.provider, record.model)
+        LLM_CALLS.labels(*labels, "true" if record.success else "false").inc()
+        LLM_LATENCY.labels(*labels).observe(record.latency_ms / 1000)
+        LLM_COST_CENTS.labels(*labels).inc(float(record.cost_cents))
+        LLM_TOKENS.labels(*labels, "in").inc(record.usage.tokens_in)
+        LLM_TOKENS.labels(*labels, "out").inc(record.usage.tokens_out)
+        LLM_TOKENS.labels(*labels, "cached").inc(record.usage.tokens_cached)

packages/llm/routing.py

@@ -0,0 +1,93 @@
+"""Scene-based routing table (架构文档 §8.1 场景化路由).
+
+``scene → (provider, model, temperature, max_tokens, timeout)`` is configuration,
+so switching models never requires a code change (NFR-MAINT-05). Defaults live
+here; ``LLM_SCENE_OVERRIDES_JSON`` overrides any field per scene at deploy time.
+
+Token budgets follow 架构文档 §7.2: 输入合计 ≤ 3700, 输出 ≤ 800.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from packages.llm.errors import SceneNotRoutedError
+from packages.llm.types import Scene, SceneConfig
+
+_DEFAULTS: dict[Scene, dict[str, Any]] = {
+    Scene.AGENT_CHAT: {
+        "temperature": 0.7,
+        "max_tokens": 800,
+        "timeout_seconds": 30.0,
+        "prompt_id": "agent_chat",
+        "prompt_version": "v1",
+        "critical": False,
+    },
+    Scene.INTENT_CLASSIFY: {
+        "temperature": 0.0,
+        "max_tokens": 128,
+        "timeout_seconds": 10.0,
+        "prompt_id": "intent_classify",
+        "prompt_version": "v1",
+        # Intent classification gates 越界拒答 (FR-AGT-02); degrading it silently
+        # would let out-of-scope questions through, so it is never budget-gated.
+        "critical": True,
+    },
+    Scene.PREFERENCE_EXTRACT: {
+        "temperature": 0.0,
+        "max_tokens": 256,
+        "timeout_seconds": 15.0,
+        "prompt_id": "preference_extract",
+        "prompt_version": "v1",
+        "critical": False,
+    },
+    Scene.REPORT_COPY: {
+        "temperature": 0.6,
+        "max_tokens": 3000,
+        "timeout_seconds": 90.0,
+        "prompt_id": "report_copy",
+        "prompt_version": "v1",
+        "critical": False,
+    },
+}
+
+
+def build_scene_configs(
+    *,
+    primary_provider: str,
+    backup_provider: str | None = None,
+    structured_max_retries: int = 2,
+    models: dict[str, str] | None = None,
+    overrides: dict[str, dict[str, Any]] | None = None,
+) -> dict[Scene, SceneConfig]:
+    """Materialise the routing table from settings."""
+    providers = tuple(p for p in (primary_provider, backup_provider) if p)
+    configs: dict[Scene, SceneConfig] = {}
+    for scene, defaults in _DEFAULTS.items():
+        merged: dict[str, Any] = {
+            **defaults,
+            "structured_max_retries": structured_max_retries,
+            **(overrides or {}).get(scene.value, {}),
+        }
+        configs[scene] = SceneConfig(
+            scene=scene,
+            providers=tuple(merged.pop("providers", providers)),
+            models=dict(merged.pop("models", models or {})),
+            **merged,
+        )
+    return configs
+
+
+class SceneRouter:
+    def __init__(self, configs: dict[Scene, SceneConfig]) -> None:
+        self._configs = dict(configs)
+
+    def resolve(self, scene: Scene) -> SceneConfig:
+        config = self._configs.get(scene)
+        if config is None:
+            msg = f"no routing configured for scene {scene.value}"
+            raise SceneNotRoutedError(msg)
+        return config
+
+    def scenes(self) -> tuple[Scene, ...]:
+        return tuple(self._configs)

packages/llm/types.py

@@ -0,0 +1,123 @@
+"""Wire types for the LLM gateway.
+
+架构文档 §4.3: llm-gateway 在 V1 是进程内包，但**接口必须按远程调用来设计**
+（无共享内存假设、明确超时、可序列化的入参出参），这样 V2 拆成独立服务时零
+成本。That is why everything here is a plain dataclass/pydantic model.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from decimal import Decimal
+from enum import StrEnum
+from typing import Any, Literal
+
+from pydantic import BaseModel, ConfigDict
+
+Role = Literal["system", "user", "assistant"]
+
+
+class Scene(StrEnum):
+    """场景决定模型档位、超时、预算 (架构文档 §8.1)."""
+
+    AGENT_CHAT = "agent_chat"
+    INTENT_CLASSIFY = "intent_classify"
+    PREFERENCE_EXTRACT = "preference_extract"
+    REPORT_COPY = "report_copy"
+
+
+class Message(BaseModel):
+    model_config = ConfigDict(frozen=True)
+
+    role: Role
+    content: str
+
+
+class Usage(BaseModel):
+    model_config = ConfigDict(frozen=True)
+
+    tokens_in: int = 0
+    tokens_out: int = 0
+    #: Subset of ``tokens_in`` served from the provider's prompt cache.
+    tokens_cached: int = 0
+
+    @property
+    def billable_input(self) -> int:
+        return max(self.tokens_in - self.tokens_cached, 0)
+
+
+@dataclass(frozen=True, slots=True)
+class SceneConfig:
+    """``scene → (provider, model, temperature, max_tokens, timeout)``, all configurable.
+
+    改模型不改代码 (架构文档 §8.1, NFR-MAINT-05).
+    """
+
+    scene: Scene
+    providers: tuple[str, ...]
+    models: dict[str, str] = field(default_factory=dict)
+    temperature: float = 0.7
+    max_tokens: int = 800
+    timeout_seconds: float = 30.0
+    prompt_id: str = ""
+    prompt_version: str = "v1"
+    structured_max_retries: int = 2
+    #: Critical scenes bypass the global daily budget breaker (架构文档 §13).
+    critical: bool = False
+
+    def model_for(self, provider: str, default: str) -> str:
+        return self.models.get(provider, default)
+
+
+@dataclass(frozen=True, slots=True)
+class ProviderRequest:
+    model: str
+    messages: tuple[Message, ...]
+    temperature: float
+    max_tokens: int
+    timeout_seconds: float
+    #: When set, the provider must constrain output to JSON.
+    json_mode: bool = False
+    stream: bool = False
+
+
+@dataclass(frozen=True, slots=True)
+class ProviderResponse:
+    text: str
+    model: str
+    usage: Usage
+    finish_reason: str | None = None
+
+
+@dataclass(frozen=True, slots=True)
+class PromptRef:
+    prompt_id: str
+    version: str
+
+    def __str__(self) -> str:
+        return f"{self.prompt_id}@{self.version}"
+
+
+@dataclass(frozen=True, slots=True)
+class Completion:
+    """Result of a successful gateway call."""
+
+    text: str
+    scene: Scene
+    provider: str
+    model: str
+    prompt_id: str | None
+    prompt_version: str | None
+    usage: Usage
+    latency_ms: int
+    cost_cents: Decimal
+    call_id: int
+    #: Populated when a ``schema`` was requested.
+    parsed: BaseModel | None = None
+    attempts: int = 1
+
+    def parsed_as(self, schema: type[Any]) -> Any:
+        if self.parsed is None or not isinstance(self.parsed, schema):
+            msg = f"completion has no parsed {schema.__name__}"
+            raise TypeError(msg)
+        return self.parsed