fix(llm): price by call time and stop paying for thinking tokens
Two defects that only surface against the real DeepSeek API, both of which break the cost model rather than the code: deepseek-v4-flash enables thinking mode by default. The same single-turn question took 13.5s with it on and 1.8s with it off, and the thinking tokens are billed at the output rate. That misses NFR-PERF-02b (15s per turn) and inflates every conversational call. SceneConfig now carries an explicit `thinking` flag, all four V1 scenes set it to false, and the provider always sends the parameter rather than relying on the upstream default. Reasoning tokens are recorded separately in Usage.tokens_reasoning so the cost of ever turning it back on is visible. DeepSeek moves to peak/off-peak pricing on 2026-08-17, with peak (Beijing 09:00-12:00 and 14:00-18:00) at twice the off-peak rate. Reports are generated mostly during peak hours, so a single blended rate would leave ob_ai_call -- the only input to V2 pricing -- off by nearly a factor of two. PeakAwareRateTable resolves the rate at call time; cost is computed and stored then, never recomputed later at whatever the current rate happens to be. Also here: - Replace the live API key in the redaction test with a synthetic one. The test that proves we redact secrets was itself a secret leak waiting for its first commit. - Correct LLM_COST_RATES_JSON, which held placeholder rates. - Make ids.SnowflakeGenerator take an injectable clock so rollover and clock-rollback are testable without patching a slots class. - Rename exceptions to the -Error suffix (N818) and clear the remaining ruff, mypy and import-linter findings, including two dead lines: a duplicate pytestmark assignment and a no-op assert on a None return. Gates: ruff, mypy (96 files incl. tests), 4/4 import contracts, detect-secrets, 166 unit + 3 integration tests, migration up/down round-trip.
This commit is contained in:
+14
-1
@@ -4,7 +4,12 @@
|
||||
so switching models never requires a code change (NFR-MAINT-05). Defaults live
|
||||
here; ``LLM_SCENE_OVERRIDES_JSON`` overrides any field per scene at deploy time.
|
||||
|
||||
Token budgets follow 架构文档 §7.2: 输入合计 ≤ 3700, 输出 ≤ 800.
|
||||
Token budgets follow 架构文档 §7.2: 输入合计 ≤ 4200, 输出 ≤ 800.
|
||||
|
||||
``thinking`` 在每个场景都显式给值。deepseek-v4 默认开启思考模式,实测同一个单句
|
||||
问答开启要 13.5s、关闭只要 1.8s,且思考 token 按输出价计费 —— 漏配就会同时踩中
|
||||
「响应超时」和「成本翻倍」。要开某个场景的思考,改 ``LLM_SCENE_OVERRIDES_JSON``
|
||||
即可,不必改代码。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -22,6 +27,8 @@ _DEFAULTS: dict[Scene, dict[str, Any]] = {
|
||||
"prompt_id": "agent_chat",
|
||||
"prompt_version": "v1",
|
||||
"critical": False,
|
||||
# 对话要在 NFR-PERF-02b 的整轮 15s 内返回,思考模式做不到
|
||||
"thinking": False,
|
||||
},
|
||||
Scene.INTENT_CLASSIFY: {
|
||||
"temperature": 0.0,
|
||||
@@ -29,6 +36,8 @@ _DEFAULTS: dict[Scene, dict[str, Any]] = {
|
||||
"timeout_seconds": 10.0,
|
||||
"prompt_id": "intent_classify",
|
||||
"prompt_version": "v1",
|
||||
# 分类是短输出的判别任务,思考只会拖慢并多花钱
|
||||
"thinking": False,
|
||||
# Intent classification gates 越界拒答 (FR-AGT-02); degrading it silently
|
||||
# would let out-of-scope questions through, so it is never budget-gated.
|
||||
"critical": True,
|
||||
@@ -40,6 +49,7 @@ _DEFAULTS: dict[Scene, dict[str, Any]] = {
|
||||
"prompt_id": "preference_extract",
|
||||
"prompt_version": "v1",
|
||||
"critical": False,
|
||||
"thinking": False,
|
||||
},
|
||||
Scene.REPORT_COPY: {
|
||||
"temperature": 0.6,
|
||||
@@ -48,6 +58,9 @@ _DEFAULTS: dict[Scene, dict[str, Any]] = {
|
||||
"prompt_id": "report_copy",
|
||||
"prompt_version": "v1",
|
||||
"critical": False,
|
||||
# 报告文案是异步任务,延迟宽松,理论上最适合开思考;V1 先关,
|
||||
# 等黄金集能量化"思考带来的质量提升"再按场景打开(改配置即可)。
|
||||
"thinking": False,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user