fix(llm): price by call time and stop paying for thinking tokens
Two defects that only surface against the real DeepSeek API, both of which break the cost model rather than the code: deepseek-v4-flash enables thinking mode by default. The same single-turn question took 13.5s with it on and 1.8s with it off, and the thinking tokens are billed at the output rate. That misses NFR-PERF-02b (15s per turn) and inflates every conversational call. SceneConfig now carries an explicit `thinking` flag, all four V1 scenes set it to false, and the provider always sends the parameter rather than relying on the upstream default. Reasoning tokens are recorded separately in Usage.tokens_reasoning so the cost of ever turning it back on is visible. DeepSeek moves to peak/off-peak pricing on 2026-08-17, with peak (Beijing 09:00-12:00 and 14:00-18:00) at twice the off-peak rate. Reports are generated mostly during peak hours, so a single blended rate would leave ob_ai_call -- the only input to V2 pricing -- off by nearly a factor of two. PeakAwareRateTable resolves the rate at call time; cost is computed and stored then, never recomputed later at whatever the current rate happens to be. Also here: - Replace the live API key in the redaction test with a synthetic one. The test that proves we redact secrets was itself a secret leak waiting for its first commit. - Correct LLM_COST_RATES_JSON, which held placeholder rates. - Make ids.SnowflakeGenerator take an injectable clock so rollover and clock-rollback are testable without patching a slots class. - Rename exceptions to the -Error suffix (N818) and clear the remaining ruff, mypy and import-linter findings, including two dead lines: a duplicate pytestmark assignment and a no-op assert on a None return. Gates: ruff, mypy (96 files incl. tests), 4/4 import contracts, detect-secrets, 166 unit + 3 integration tests, migration up/down round-trip.
This commit is contained in:
+49
-1
@@ -13,7 +13,7 @@ from __future__ import annotations
|
||||
|
||||
from collections.abc import Mapping
|
||||
from dataclasses import dataclass, replace
|
||||
from datetime import UTC, datetime
|
||||
from datetime import UTC, datetime, time, timedelta, timezone
|
||||
from decimal import ROUND_HALF_UP, Decimal
|
||||
from typing import Protocol, runtime_checkable
|
||||
|
||||
@@ -22,6 +22,15 @@ from packages.llm.types import Usage
|
||||
_MILLION = Decimal("1000000")
|
||||
_QUANT = Decimal("0.000001")
|
||||
|
||||
BEIJING = timezone(timedelta(hours=8))
|
||||
|
||||
# DeepSeek 自 2026-08-17 00:00(北京时间)起改为峰谷定价,高峰单价是空闲时段的 2 倍。
|
||||
# 官方口径的高峰时段:北京时间 09:00-12:00 与 14:00-18:00。
|
||||
PEAK_WINDOWS: tuple[tuple[time, time], ...] = (
|
||||
(time(9, 0), time(12, 0)),
|
||||
(time(14, 0), time(18, 0)),
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ModelRate:
|
||||
@@ -55,6 +64,37 @@ class RateTable:
|
||||
return tuple(sorted(self._rates))
|
||||
|
||||
|
||||
def is_peak(at: datetime | None = None) -> bool:
|
||||
"""调用时刻是否落在供应商的高峰时段(按北京时间判定)。"""
|
||||
moment = (at or datetime.now(UTC)).astimezone(BEIJING).timetz()
|
||||
return any(start <= moment.replace(tzinfo=None) < end for start, end in PEAK_WINDOWS)
|
||||
|
||||
|
||||
class PeakAwareRateTable:
|
||||
"""按调用时刻在「高峰 / 空闲」两套单价之间选择。
|
||||
|
||||
为什么不能只存一套单价:高峰价是空闲价的 2 倍,而报告生成这类任务大多发生在
|
||||
白天高峰时段。用单一均价记账会让 ``ob_ai_call`` 的成本与账单对不上,
|
||||
而这张表是 V2 定价的唯一依据 —— 记错等于把定价建在错的数字上。
|
||||
|
||||
时段判断放在算成本的这一刻,不是事后按当前价重算:跨时段重算会把历史成本改掉。
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self, off_peak: RateTable, peak: RateTable | None = None, *, enabled: bool = True
|
||||
) -> None:
|
||||
self._off_peak = off_peak
|
||||
self._peak = peak or off_peak
|
||||
self._enabled = enabled
|
||||
|
||||
def for_model(self, model: str, at: datetime | None = None) -> ModelRate:
|
||||
table = self._peak if self._enabled and is_peak(at) else self._off_peak
|
||||
return table.for_model(model)
|
||||
|
||||
def known_models(self) -> tuple[str, ...]:
|
||||
return self._off_peak.known_models()
|
||||
|
||||
|
||||
def compute_cost_cents(usage: Usage, rate: ModelRate) -> Decimal:
|
||||
"""Cost of one call, in cents, with cache-hit tokens billed separately."""
|
||||
billable_input = Decimal(usage.billable_input) * rate.input_cents_per_mtok
|
||||
@@ -64,6 +104,13 @@ def compute_cost_cents(usage: Usage, rate: ModelRate) -> Decimal:
|
||||
return total.quantize(_QUANT, rounding=ROUND_HALF_UP)
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
class RateResolver(Protocol):
|
||||
"""网关只需要「给我这个模型此刻的单价」,不关心背后是一套还是两套价目表。"""
|
||||
|
||||
def for_model(self, model: str) -> ModelRate: ...
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class AICallRecord:
|
||||
"""One row of ``ob_ai_call``. Written for successes *and* failures."""
|
||||
@@ -88,6 +135,7 @@ class AICallRecord:
|
||||
def with_defaults(self) -> AICallRecord:
|
||||
if self.created_at is not None:
|
||||
return self
|
||||
# 本类是 slots dataclass,没有 __dict__,只能用 replace 复制
|
||||
return replace(self, created_at=datetime.now(UTC))
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user