feat(llm): provider-abstract gateway with cost and failure controls
DeepSeek (OpenAI-compatible) as primary with a 智谱 GLM slot that is a real routing path but raises until the 算法备案 model list is confirmed. Adds scene routing, failover, a per-provider circuit breaker, structured output with retry-on-validation-failure, prompts loaded by id+version from prompts/, and cost attribution written to ob_ai_call for every attempt -- successes, retries and failures alike. Two controls exist because they are cheap now and expensive later: a cache-prefix validator that rejects timestamps, uuids and nicknames in the cached system prefix (a volatile prefix silently multiplies input cost), and a hard platform-wide daily budget that rejects non-critical scenes instead of merely alerting, since a retry storm outruns any human on-call.
This commit is contained in:
Executable
+34
@@ -0,0 +1,34 @@
|
||||
"""Provider abstraction.
|
||||
|
||||
架构文档 §4.3 requires the gateway interface to be designed as if it were a
|
||||
remote call, so providers take a fully-specified request (explicit timeout,
|
||||
serialisable payload) and return a fully-specified response. No provider is
|
||||
allowed to reach back into gateway state.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Protocol, runtime_checkable
|
||||
|
||||
from packages.llm.types import ProviderRequest, ProviderResponse
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
class LLMProvider(Protocol):
|
||||
@property
|
||||
def name(self) -> str:
|
||||
"""Stable identifier used in routing config and in ``ob_ai_call``."""
|
||||
...
|
||||
|
||||
@property
|
||||
def default_model(self) -> str:
|
||||
"""Model used when the scene config does not pin one."""
|
||||
...
|
||||
|
||||
async def complete(self, request: ProviderRequest) -> ProviderResponse:
|
||||
"""Perform one completion. Must raise ``packages.llm.errors.ProviderError``."""
|
||||
...
|
||||
|
||||
async def aclose(self) -> None:
|
||||
"""Release connections."""
|
||||
...
|
||||
Reference in New Issue
Block a user