xsl
f46ecc5eb0
Two defects that only surface against the real DeepSeek API, both of which break the cost model rather than the code: deepseek-v4-flash enables thinking mode by default. The same single-turn question took 13.5s with it on and 1.8s with it off, and the thinking tokens are billed at the output rate. That misses NFR-PERF-02b (15s per turn) and inflates every conversational call. SceneConfig now carries an explicit `thinking` flag, all four V1 scenes set it to false, and the provider always sends the parameter rather than relying on the upstream default. Reasoning tokens are recorded separately in Usage.tokens_reasoning so the cost of ever turning it back on is visible. DeepSeek moves to peak/off-peak pricing on 2026-08-17, with peak (Beijing 09:00-12:00 and 14:00-18:00) at twice the off-peak rate. Reports are generated mostly during peak hours, so a single blended rate would leave ob_ai_call -- the only input to V2 pricing -- off by nearly a factor of two. PeakAwareRateTable resolves the rate at call time; cost is computed and stored then, never recomputed later at whatever the current rate happens to be. Also here: - Replace the live API key in the redaction test with a synthetic one. The test that proves we redact secrets was itself a secret leak waiting for its first commit. - Correct LLM_COST_RATES_JSON, which held placeholder rates. - Make ids.SnowflakeGenerator take an injectable clock so rollover and clock-rollback are testable without patching a slots class. - Rename exceptions to the -Error suffix (N818) and clear the remaining ruff, mypy and import-linter findings, including two dead lines: a duplicate pytestmark assignment and a no-op assert on a None return. Gates: ruff, mypy (96 files incl. tests), 4/4 import contracts, detect-secrets, 166 unit + 3 integration tests, migration up/down round-trip.
71 lines
1.9 KiB
Python
Executable File
71 lines
1.9 KiB
Python
Executable File
"""LLM gateway — 多供应商路由、Prompt 版本、结构化输出、成本计量、预算保护.
|
|
|
|
架构文档 §4.3: V1 是进程内包，V2 拆为独立服务。接口按远程调用设计，拆分零成本。
|
|
"""
|
|
|
|
from packages.llm.budget import DailyBudgetGuard, InMemoryCostCounter, RedisCostCounter
|
|
from packages.llm.circuit import CircuitBreaker, CircuitBreakerRegistry, CircuitState
|
|
from packages.llm.cost import (
|
|
AICallRecord,
|
|
AICallRecorder,
|
|
CompositeAICallRecorder,
|
|
InMemoryAICallRecorder,
|
|
ModelRate,
|
|
NullAICallRecorder,
|
|
PeakAwareRateTable,
|
|
RateResolver,
|
|
RateTable,
|
|
compute_cost_cents,
|
|
is_peak,
|
|
)
|
|
from packages.llm.errors import (
|
|
AllProvidersFailedError,
|
|
CachePrefixViolationError,
|
|
DailyBudgetExceededError,
|
|
LLMError,
|
|
ProviderError,
|
|
StructuredOutputError,
|
|
)
|
|
from packages.llm.gateway import LLMGateway
|
|
from packages.llm.prompts import PromptRegistry, PromptTemplate, validate_cache_prefix
|
|
from packages.llm.routing import SceneRouter, build_scene_configs
|
|
from packages.llm.types import Completion, Message, PromptRef, Scene, SceneConfig, Usage
|
|
|
|
__all__ = [
|
|
"AICallRecord",
|
|
"AICallRecorder",
|
|
"AllProvidersFailedError",
|
|
"CachePrefixViolationError",
|
|
"CircuitBreaker",
|
|
"CircuitBreakerRegistry",
|
|
"CircuitState",
|
|
"Completion",
|
|
"CompositeAICallRecorder",
|
|
"DailyBudgetExceededError",
|
|
"DailyBudgetGuard",
|
|
"InMemoryAICallRecorder",
|
|
"InMemoryCostCounter",
|
|
"LLMError",
|
|
"LLMGateway",
|
|
"Message",
|
|
"ModelRate",
|
|
"NullAICallRecorder",
|
|
"PeakAwareRateTable",
|
|
"PromptRef",
|
|
"PromptRegistry",
|
|
"PromptTemplate",
|
|
"ProviderError",
|
|
"RateResolver",
|
|
"RateTable",
|
|
"RedisCostCounter",
|
|
"Scene",
|
|
"SceneConfig",
|
|
"SceneRouter",
|
|
"StructuredOutputError",
|
|
"Usage",
|
|
"build_scene_configs",
|
|
"compute_cost_cents",
|
|
"is_peak",
|
|
"validate_cache_prefix",
|
|
]
|