Commit Graph
14 Commits
Author SHA1 Message Date
xsl cf9d500bfa feat(identity): phone SMS login with triple rate limiting and anti-abuse
SmsSender Protocol(fake/tencent)+ 验证码生命周期(哈希存储/一次性/
5 次作废)+ 号码-IP-设备三重频控 + 掩码纪律(FR-ACC-04/14,TD-32;
真实发送待签名审核)。

实现:packages/identity/sms_login.py(SmsStore/SmsSender 双 Protocol、
FakeSmsSender 打日志、TencentSmsSender 包官方 SDK 失败映射 61000、
SmsAuthService 验证码生命周期 + UTC+8 时段桶频控);路由 POST
/auth/sms/send、POST /auth/sms/verify(provider="phone" 映射 +
issue_session + TokenPairResponse,与华为通道零耦合)。测试:单测 37 个
(内存 dict 实现 store、注入时钟模拟 TTL;号码规范化/掩码/频控矩阵/
验证码四要素/腾讯通道桩测),集成 7 个(send→verify 双令牌闭环、同号
同 user、脚本化刷量 11 发第 11 次 42911、IP/设备维度、错 5 次作废)。
门禁:ruff/mypy(identity+apps/api)/lint-imports 全绿,单测 275 通过、
集成 23 通过。

偏差说明:
1. SmsSender.send_code 收 (phone, masked, code) 三参而非单一「掩码键」:
   腾讯通道下发必须有真实号码(掩码/哈希无法投递);防泄漏由「实现层
   不得记录 phone」约定 + structlog 全局手机号脱敏兜底。
2. 频控计数按「请求」计(含被拒请求)且在限额检查前自增:否则被拒
   请求不烧额度,攻击者可无限试探;同时这是「连发 11 次第 11 次
   42911」脚本化验收在真实时钟下成立的前提。
3. request_code 对 sender 逃逸的非 AppError 异常兜底包成 61000:
   通道故障不得以 5xxxx 内部错误形式稀释告警(TencentSmsSender 自身
   映射为主,此处仅接漏网)。
4. wire 请求模型(SmsSendRequest/SmsVerifyRequest)定义在 sms_login.py
   内(任务文档允许的两选一,未新建 sms_schemas.py)。
2026-08-14 20:04:46 +08:00
xsl ddec3973ad merge: T1 四档分级降级熔断(T-P1-25b,NFR-COST-02) 2026-08-14 16:28:22 +08:00
xsl a4d59aca30 feat(llm): four-tier daily budget degradation with manual override
把单一日预算熔断升级为架构 §13.3 的四档分级降级(T-P1-25b / NFR-COST-02):

- budget.py:新增 BudgetTier 配置模型与档位判定(current_tier/tier_name/
  blocked_scenes)。阈值相对 LLM_DAILY_BUDGET_CENTS:warn ≥60% 只告警;
  degrade_secondary ≥80% 拒 hd_export/report_regen;degrade_generation ≥90%
  追加拒 hairstyle/tryon;block_all ≥100% 拒全部非 critical、allow 名单
  (默认 agent_chat)放行。高档位 blocked 取「截至当前档的并集」(架构原文
  「追加拒绝」)。critical 场景全档豁免但照常计成本。
- 手动 override:Redis 键 llm:budget:override(值 "1"~"4"),优先于成本水
  位,读不到/非法则忽略;set_override TTL 到当日 UTC+8 24 点,跨预算日必然
  失效。CostCounter 协议相应增加 set/delete。
- 指标:Gauge llm_budget_tier + Counter llm_budget_blocked_total{scene}
  挂到共享 REGISTRY 随 /metrics 暴露;档位上升沿只告警一次
  (budget_tier_raised,实例内记 last_tier)。
- errors.py 新增 SceneDegradedError(tier2/3 拒绝);gateway.py 调用点换成
  ensure_scene_allowed(保留 ensure_within_budget 兼容入口);agent.py 映射
  LLM_SCENE_DEGRADED(60006);100% 档维持 DailyBudgetExceededError/60003 语义。
- 单测:新增 tests/unit/test_tiered_budget.py(27 例:四档边界、各档放行/
  拒绝、critical 豁免、override 加严/放宽/非法值/TTL、配置驱动)。
- 集成:tests/integration/test_vertical_slice.py 新增用例——预算 1 分 +
  block_all allow 清空 + Redis 注入当日成本,POST /agent/chat 返回 60003,
  断言 bl_ledger 中 consume(refunded,-1)+refund(committed,+1) 净和为 0,
  覆盖「reserve 之后才熔断 → refund」路径;fixture 重构为共享
  FakeProvider/fake_client,断言语义未弱化。

门禁:ruff check/format、mypy(packages/llm strict + 其余包)、lint-imports、
单测 196 passed、集成 4 passed。

偏差说明:
1. tests/unit/test_gateway.py 的 test_daily_budget_rejects_non_critical_
   scenes_before_spending 把被测场景 AGENT_CHAT 换成 REPORT_COPY(test_daily_
   budget.py 同用例换 tryon):block_all 档默认 allow=["agent_chat"](架构
   「仅保留纯文本对话」)后 agent_chat 在 100% 档不再被拒,原场景选择与新
   语义冲突;断言本身(进 provider 前被拒)未弱化。
2. 集成测试的 block_all 档配置为 blocked="*"+allow=[](文档示例为
   blocked=["agent_chat"]):后者按档位规则会走 SceneDegradedError/60006,
   前者才能按设计走既有 60003 语义(文档允许二者取其一且"按上面设计应为
   60003")。
2026-08-14 16:27:21 +08:00
xsl fcd7e63ce5 merge: T2 任务奖励领取(POST /quota/tasks/{task_key}/claim,FR-BILL-03) 2026-08-14 16:23:46 +08:00
xsl ecdfbb64c3 feat(billing): task bonus claim endpoint with idempotency and cap
POST /quota/tasks/{task_key}/claim:配置驱动任务注册表、
grant_bonus 纯域函数、ledger 幂等键防重、task_bonus_max 封顶(FR-BILL-03/04)。

实现:packages/domain/billing/quota.py 新增 BonusDecision/grant_bonus(mypy strict,
恰好到顶允许、超一格整体拒绝)、grant_idempotency_key、PERIOD_TYPES;
packages/billing/service.py 新增 claim_task_bonus(幂等快捷路径 → 行锁 → 封顶判定
→ 同事务更新 bonus_granted 并写 system_grant 流水,delta=0 且 balance_after 按
remaining 口径;IntegrityError 并发兜底回滚为 40910);响应模型 TaskBonusClaimResponse
放新建的 packages/billing/schemas.py;路由挂在 apps/api/routers/quota.py,
rules 由 settings.task_bonus_rules 注入(amount 为占位值,产品定稿后改 env)。
测试:新增单测 11 个(封顶边界、amount<=0、不动 used、hard_cap 独立生效、
默认规则形状与 env 覆盖通路);新增集成 4 个(领取+重复 40910+纯 DB 流水断言、
五类任务全部到账、未知任务 40410、amount=11>max=10 → 40310 且无新流水)。
门禁全绿:ruff check / ruff format --check / mypy 两段 / lint-imports;
单测 180 passed,集成 7 passed。

偏差说明:1) 任务文档提到的 BillingService._load_rule 实际名为 _require_rule,
   直接复用现有助手;2) packages/billing/repo.py 无需新增函数(现有
   get_ledger_by_idempotency_key/lock_or_create_quota/add_ledger 已覆盖本任务);
   3) 文档设想的「TASK_BONUS_RULES_JSON 空值容错」经 env 源实测不成立——
   pydantic-settings 对 dict 字段先做 JSON 解码,空白字符串在 Settings 构造时即
   报错(共享校验器/config.py 属禁改,未动);单测改为覆盖受支持的空配置形态 "{}"。
2026-08-14 16:23:14 +08:00
xsl ae53196657 merge: T4 刷新令牌与服务端可撤销会话(FR-ACC-05) 2026-08-14 16:22:47 +08:00
xsl 140a63effa feat(identity): refresh token rotation with server-side revocable sessions
id_session 落地:opaque refresh(token_urlsafe(48))+ sha256 哈希存储 +
一次性轮换(旧行撤销与新行同一事务)+ logout 幂等撤销 + 已撤销重放检测
告警(V1 不做家族连坐);错误码 40103/40104/40105 接入。新增 POST
/auth/refresh、POST /auth/logout;dev-token 扩展为双令牌,旧字段
access_token/token_type/expires_in/user_id 原名保留,新增
refresh_token/refresh_expires_in(smoke.sh 不受影响)。

测试:tests/unit/test_refresh_tokens.py 覆盖 token 长度/字符域/唯一性、
哈希稳定性与不可逆(7 例,不连网不连库);tests/integration/
test_auth_refresh.py 覆盖轮换后新 access 可调 /quota、旧 refresh 二次
使用 40105、logout 后失效 40105、伪造 40103、TTL=0 过期 40104(5 例)。
门禁全绿:ruff check/format、mypy 两段、lint-imports 4 契约、
单测 176 通过、集成 8 通过。

偏差说明:1) SessionIssued 较任务文档字段清单增加 user_id ——
/auth/refresh 换发 access JWT 必须知道用户,opaque refresh 本身不携带
信息,只能随签发结果带回;2) 过期分支「顺手落库 revoked=true」在 HTTP
路径会随 401 的统一回滚一起丢失(仅事务内标记,已注释说明,不影响判定
语义);3) repo 撤销用 SELECT FOR UPDATE + ORM 变更而非 UPDATE
rowcount —— 对齐 billing.repo 现行模式,且 rowcount 在 mypy 下类型
不可用。
2026-08-14 16:21:52 +08:00
xsl d0522cd495 merge: T5 COS 直传预签名(POST /uploads/presign,FR-SYS-03) 2026-08-14 16:18:16 +08:00
xsl 367b610e93 feat(storage): COS presigned PUT URL endpoint for direct upload
POST /uploads/presign:场景/类型/大小三重校验 + 可审计 object_key 布局 +
SDK 隔离在 CosSigner Protocol 后(FR-SYS-03,T-P1-27)。

实现:packages/storage/schemas.py(PresignRequest/PresignResponse)、
packages/storage/service.py(CosSigner Protocol / QcloudCosSigner /
StorageService.build_presign,bucket 四别名映射到 cos_bucket_*,
object_key 写死 {scene}/{user_id}/{yyyyMMdd}/{uuid4hex}.{ext})、
apps/api/routers/uploads.py(POST /presign,CurrentUserId 认证 + ok 信封)。

测试:tests/unit/test_upload_presign.py 16 例全部经 FakeSigner 注入,
零网络、零 SDK client 构造;覆盖 key 布局逐段断言、三重校验统一 40010、
bucket 别名映射、白名单字符串解析(空串/空格容错)、配置错误分支、
schema 层 size>=1。门禁全绿:ruff check/format、mypy(宽松 48 文件 +
domain strict 7 文件)、lint-imports 4 契约、pytest 185 passed
(基线 169 + 新增 16)。

偏差说明:
1) SDK 实际无 qcloud_cos.method 模块,Method 按官方文档传字符串 "PUT";
   get_presigned_url 参数名为 Expired(非任务文档写的 Expire);配置类
   导出名为 CosConfig(非 CosS3Config)。按文档 §3 "以实际 SDK 签名为准"
   处理,已用假密钥端到端冒烟验证(纯本地签名计算,零网络)。
2) build_presign 增加可选 keyword-only 参数 now(默认 None=当前 UTC),
   供单测注入固定时钟(对齐 DailyBudgetGuard 的 clock 注入惯例),
   不改变文档签名的调用兼容性。
3) 两处防御分支文档未规定,均配了单测:场景 bucket 别名非法报
   CONFIG_ERROR 而非 40010(配置错误不应让客户端背 4xx);COS 密钥为空
   在 SDK 构造前报 CONFIG_ERROR,避免未处理的裸 500。
2026-08-14 16:17:49 +08:00
xsl 64175a1ea1 feat(analytics): POST /events/batch with fail-close validation
ob_event 落库通道:形状校验 + props 字节上限 + client_ts 合理性 +
PII 键剔除,整批原子(FR-SYS-06 服务端部分)。

实现:packages/analytics/schemas.py(EventIn/EventBatchRequest/
EventBatchResponse + 事件名形状正则常量);service.py 拆出纯函数
validate_batch(批量/props 字节/事件名形状/client_ts 窗口/PII 剔除,
任一不过整批 40020),AnalyticsService.record 组装 ObEvent 逐条
add + 一次 flush;routers/events.py 提供 POST /events/batch
(/api/v1 与 /v1 别名,认证必需)。

测试:单测 tests/unit/test_events_validation.py 8 例(合法批、超批量、
props 超字节/不可序列化、事件名形状含尾随换行、client_ts 双边界、
naive 时区按 UTC、PII 键含大小写变体剔除);集成
tests/integration/test_events.py 3 例(两前缀可用+落库断言、
非法批 40020 且库无半条、无 token 40100)。门禁全绿:ruff
check/format、mypy(T2 列表 + domain strict)、lint-imports 4 契约、
单测 177 通过、集成 6 通过。

偏差说明:event_name 正则未写成 Field(pattern=...),而是作为
EVENT_NAME_PATTERN 常量放 schemas.py、由服务层 validate_batch 强制——
若挂在请求体校验上,非法事件名会先被 RequestValidationError 拦截、
返回通用 40001/422 信封,无法满足任务契约 §4「大写 event_name →
40020」的集成行为。正则本体与文档一致。
2026-08-14 16:16:26 +08:00
xsl 2009aa938c fix(billing): make quota seed and column names match the product SSOT
The product plan is the only source for quota numbers, but the 0001 seed
used a second set (agent_chat 20/50, tryon hard 10) and omitted hairstyle,
hd_export and report regen. Migration 0003 replaces the seed and renames
points_reserved → quota_reserved and bonus_value → bonus_granted so the
docs and the schema stop contradicting each other.

Effective limit now clips at hard_cap. CI runs the integration slice
against the Postgres/Redis services it already starts.
2026-08-14 11:28:35 +08:00
xsl c97f8ae3e1 feat(cost): persist reasoning tokens and the peak/off-peak tier
CI / lint / typecheck / import-linter / tests / secrets (push) Canceled after 0s
ob_ai_call is the only input to V2 pricing, and it was dropping the two
numbers needed to interpret its own cost column.

tokens_reasoning was captured from the provider response but never written to
the table, so the cost of thinking mode was invisible once it left the
process. Folding it into tokens_out was not an option either -- it is billed
at the output rate, and mixing the two hides exactly the difference we now
have to watch.

rate_tier records which price list applied. Without it a row's cost_cents
cannot be rechecked: DeepSeek's peak rate is double off-peak from 2026-08-17,
so recomputing an old row at "the current rate" silently produces a different
answer than what was actually billed.

Migration 0002 adds both columns. tokens_reasoning defaults to 0 rather than
NULL so cost aggregates do not collapse to NULL on historical rows;
rate_tier stays nullable because rows written before this change genuinely
have an unknown tier, and NULL says that more honestly than a guessed 'flat'.

The smoke script now asserts both: reasoning tokens must be 0 (a non-zero
value means a provider default re-enabled thinking mode behind our backs) and
rate_tier must be present. Verified against the real API -- tokens_reasoning=0,
rate_tier=off_peak, 3.9s round trip.
2026-08-14 01:02:13 +08:00
xsl f46ecc5eb0 fix(llm): price by call time and stop paying for thinking tokens
Two defects that only surface against the real DeepSeek API, both of which
break the cost model rather than the code:

deepseek-v4-flash enables thinking mode by default. The same single-turn
question took 13.5s with it on and 1.8s with it off, and the thinking tokens
are billed at the output rate. That misses NFR-PERF-02b (15s per turn) and
inflates every conversational call. SceneConfig now carries an explicit
`thinking` flag, all four V1 scenes set it to false, and the provider always
sends the parameter rather than relying on the upstream default. Reasoning
tokens are recorded separately in Usage.tokens_reasoning so the cost of ever
turning it back on is visible.

DeepSeek moves to peak/off-peak pricing on 2026-08-17, with peak (Beijing
09:00-12:00 and 14:00-18:00) at twice the off-peak rate. Reports are generated
mostly during peak hours, so a single blended rate would leave ob_ai_call --
the only input to V2 pricing -- off by nearly a factor of two.
PeakAwareRateTable resolves the rate at call time; cost is computed and stored
then, never recomputed later at whatever the current rate happens to be.

Also here:
- Replace the live API key in the redaction test with a synthetic one. The
  test that proves we redact secrets was itself a secret leak waiting for its
  first commit.
- Correct LLM_COST_RATES_JSON, which held placeholder rates.
- Make ids.SnowflakeGenerator take an injectable clock so rollover and
  clock-rollback are testable without patching a slots class.
- Rename exceptions to the -Error suffix (N818) and clear the remaining ruff,
  mypy and import-linter findings, including two dead lines: a duplicate
  pytestmark assignment and a no-op assert on a None return.

Gates: ruff, mypy (96 files incl. tests), 4/4 import contracts,
detect-secrets, 166 unit + 3 integration tests, migration up/down round-trip.
2026-08-14 00:54:40 +08:00
xsl 93b88db2f5 test: unit suite for the gateway invariants plus an integration slice
Covers routing, failover, the circuit breaker, structured-output retries, the
cache-prefix validator, cost computation including cached-token pricing, the
daily budget breaker, the quota protocol, the task state machine and log
redaction. HTTP is mocked -- no unit test spends money. The integration test
runs the real slice against PostgreSQL and Redis with a faked provider.
2026-08-14 00:26:39 +08:00