feat(llm): provider-abstract gateway with cost and failure controls
DeepSeek (OpenAI-compatible) as primary with a 智谱 GLM slot that is a real routing path but raises until the 算法备案 model list is confirmed. Adds scene routing, failover, a per-provider circuit breaker, structured output with retry-on-validation-failure, prompts loaded by id+version from prompts/, and cost attribution written to ob_ai_call for every attempt -- successes, retries and failures alike. Two controls exist because they are cheap now and expensive later: a cache-prefix validator that rejects timestamps, uuids and nicknames in the cached system prefix (a volatile prefix silently multiplies input cost), and a hard platform-wide daily budget that rejects non-critical scenes instead of merely alerting, since a retry storm outruns any human on-call.
This commit is contained in:
Executable
+57
@@ -0,0 +1,57 @@
|
||||
"""智谱 GLM provider — backup supplier slot (TD-04, 架构文档 §2 降级表).
|
||||
|
||||
**Structure only.** The routing/failover code path is real: the gateway will
|
||||
route to this provider, the breaker will count its failures, and ``ob_ai_call``
|
||||
will record the attempt. Only the HTTP call is missing.
|
||||
|
||||
Before implementing, resolve the open compliance item in 需求规格说明 §5.3:
|
||||
「备案覆盖的模型清单核实 —— 若备案未覆盖 DeepSeek + GLM 两家,双路由属超范围」.
|
||||
Shipping a working second route ahead of that check is a compliance risk, not a
|
||||
resilience win.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from packages.llm.errors import ProviderNotImplementedError
|
||||
from packages.llm.types import ProviderRequest, ProviderResponse
|
||||
|
||||
_NOT_IMPLEMENTED_MESSAGE = (
|
||||
"智谱 GLM provider is a structural placeholder and has no implementation. "
|
||||
"To enable it: (1) confirm the 算法备案 model list covers GLM "
|
||||
"(需求规格说明 §5.3), (2) implement complete() against "
|
||||
"POST {base_url}/chat/completions, (3) map GLM usage fields onto "
|
||||
"packages.llm.types.Usage, (4) set ZHIPU_API_KEY. "
|
||||
"Until then the gateway treats this slot as a permanently failing provider."
|
||||
)
|
||||
|
||||
|
||||
class ZhipuGLMProvider:
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
api_key: str = "",
|
||||
base_url: str = "https://open.bigmodel.cn/api/paas/v4",
|
||||
default_model: str = "glm-4-flash",
|
||||
name: str = "zhipu",
|
||||
) -> None:
|
||||
self._api_key = api_key
|
||||
self._base_url = base_url.rstrip("/")
|
||||
self._default_model = default_model
|
||||
self._name = name
|
||||
|
||||
@property
|
||||
def name(self) -> str:
|
||||
return self._name
|
||||
|
||||
@property
|
||||
def default_model(self) -> str:
|
||||
return self._default_model
|
||||
|
||||
async def complete(self, request: ProviderRequest) -> ProviderResponse:
|
||||
raise ProviderNotImplementedError(self._name, _NOT_IMPLEMENTED_MESSAGE)
|
||||
|
||||
async def aclose(self) -> None:
|
||||
return None
|
||||
|
||||
|
||||
__all__ = ["ZhipuGLMProvider"]
|
||||
Reference in New Issue
Block a user