xsl
54dc741811
DeepSeek (OpenAI-compatible) as primary with a 智谱 GLM slot that is a real routing path but raises until the 算法备案 model list is confirmed. Adds scene routing, failover, a per-provider circuit breaker, structured output with retry-on-validation-failure, prompts loaded by id+version from prompts/, and cost attribution written to ob_ai_call for every attempt -- successes, retries and failures alike. Two controls exist because they are cheap now and expensive later: a cache-prefix validator that rejects timestamps, uuids and nicknames in the cached system prefix (a volatile prefix silently multiplies input cost), and a hard platform-wide daily budget that rejects non-critical scenes instead of merely alerting, since a retry storm outruns any human on-call.
35 lines
1.0 KiB
Python
Executable File
35 lines
1.0 KiB
Python
Executable File
"""Provider abstraction.
|
|
|
|
架构文档 §4.3 requires the gateway interface to be designed as if it were a
|
|
remote call, so providers take a fully-specified request (explicit timeout,
|
|
serialisable payload) and return a fully-specified response. No provider is
|
|
allowed to reach back into gateway state.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from typing import Protocol, runtime_checkable
|
|
|
|
from packages.llm.types import ProviderRequest, ProviderResponse
|
|
|
|
|
|
@runtime_checkable
|
|
class LLMProvider(Protocol):
|
|
@property
|
|
def name(self) -> str:
|
|
"""Stable identifier used in routing config and in ``ob_ai_call``."""
|
|
...
|
|
|
|
@property
|
|
def default_model(self) -> str:
|
|
"""Model used when the scene config does not pin one."""
|
|
...
|
|
|
|
async def complete(self, request: ProviderRequest) -> ProviderResponse:
|
|
"""Perform one completion. Must raise ``packages.llm.errors.ProviderError``."""
|
|
...
|
|
|
|
async def aclose(self) -> None:
|
|
"""Release connections."""
|
|
...
|