feat(shared): cross-cutting plumbing and pure domain layer
Typed pydantic-settings config, structlog JSON logging with request/trace id propagation and redaction of phone numbers, tokens and face keys, the unified response envelope and error-code table from 架构文档 §9.2, async SQLAlchemy session management, ORM models for the eight foundation tables, and the OpenTelemetry/Prometheus wiring. packages/domain stays pure logic -- snowflake ids, the quota three-phase protocol and the tk_task state machine -- so it can be tested without a database and is held to mypy strict.
This commit is contained in:
Executable
+46
@@ -0,0 +1,46 @@
|
|||||||
|
# `packages/` — 业务域与能力包
|
||||||
|
|
||||||
|
架构 §3.2:**单体不等于混乱**。这里的每个子目录都是一个边界,越界会被 CI 拦下。
|
||||||
|
|
||||||
|
## 目录含义
|
||||||
|
|
||||||
|
| 包 | 类型 | 内容 |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `domain/` | 纯逻辑层 | 领域模型与业务规则。**无 IO、无框架、可纯单测** |
|
||||||
|
| `shared/` | 基础设施 | 配置、日志、错误码、统一响应、DB、Redis、可观测 |
|
||||||
|
| `llm/` | AI 能力 | LLM 网关(已实现) |
|
||||||
|
| `retrieval/` `rules/` `memory/` | AI 能力 | 混合检索 / 规则引擎 / 记忆系统(占位) |
|
||||||
|
| `compliance/` | 业务域 | 统一合规出口(占位,CR-05/CR-06) |
|
||||||
|
| `identity/` `billing/` `task/` `agent/` | 业务域 | 已有实现 |
|
||||||
|
| `profile/` `report/` `tryon/` `kb/` `notify/` `wardrobe/` | 业务域 | 占位 |
|
||||||
|
|
||||||
|
每个业务域包的标准结构:
|
||||||
|
|
||||||
|
```
|
||||||
|
packages/<域>/
|
||||||
|
├─ __init__.py
|
||||||
|
├─ README.md 这个域负责什么、不负责什么
|
||||||
|
├─ service.py 对外唯一入口。其他域只能调这里
|
||||||
|
└─ repo.py SQL。只有本域可以 import
|
||||||
|
```
|
||||||
|
|
||||||
|
## 硬性规则(由 `make importlint` 机器校验)
|
||||||
|
|
||||||
|
1. **禁止跨域直接查表**。要用别的域的数据,调它的 `service`,不要 import 它的
|
||||||
|
`repo`,更不要在自己的 SQL 里 JOIN 别人的表。表名的域前缀(`id_` / `bl_` /
|
||||||
|
`tk_` / …)就是为将来拆库预留的边界。
|
||||||
|
2. **`packages/domain/` 必须保持纯净**。不许 import `fastapi`、`sqlalchemy`、
|
||||||
|
`httpx`、`redis`、`celery`,也不许 import `packages/shared`。放不进去说明
|
||||||
|
它不是领域逻辑,是应用逻辑,应该待在 `service.py`。
|
||||||
|
3. **`compliance` 不得依赖任何业务域**。它必须能被任何模块安全调用。
|
||||||
|
4. **`identity` 不得依赖任何其他业务域**。登录链路不能被业务拖累。
|
||||||
|
5. **AI 能力包(`llm` / `retrieval` / `rules`)不得依赖业务域**,也不得 import
|
||||||
|
`apps`。它们要能在 V2 原样拆成独立服务。
|
||||||
|
|
||||||
|
## 加一个新业务域时
|
||||||
|
|
||||||
|
1. 建目录,写 `README.md` 说明职责与不负责的事。
|
||||||
|
2. 在 `pyproject.toml` 的 `[tool.importlinter]` 契约里把它加进相应的
|
||||||
|
`layers` / `forbidden_modules` 列表 —— 新域默认没有约束,忘了加就等于没有边界。
|
||||||
|
3. 领域规则先写进 `packages/domain/<域>/` 并配单测,再写 `service.py`。
|
||||||
|
4. 表加域前缀,走 Alembic,迁移必须向后兼容(NFR-MAINT-06)。
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
"""业务域与能力包。域边界由 import-linter 强制(架构文档 §4.2)。"""
|
||||||
Executable
+39
@@ -0,0 +1,39 @@
|
|||||||
|
# `packages/domain/` — 纯领域逻辑
|
||||||
|
|
||||||
|
架构 §3.2 硬性规则 2:**领域逻辑放在这里,不依赖 FastAPI、不依赖数据库会话,
|
||||||
|
可纯单元测试。**
|
||||||
|
|
||||||
|
## 能放什么
|
||||||
|
|
||||||
|
- 数据类与值对象(配额状态、任务状态、诊断维度…)
|
||||||
|
- 判定与计算(配额是否够、状态能否流转、置信度冲突消解、色差计算…)
|
||||||
|
- 纯函数、不可变数据结构
|
||||||
|
|
||||||
|
## 绝对不能放什么
|
||||||
|
|
||||||
|
- `import fastapi` / `sqlalchemy` / `httpx` / `redis` / `celery`
|
||||||
|
- `import packages.shared`(连日志和配置都不行)
|
||||||
|
- 任何 IO:读文件、发请求、查库、读环境变量
|
||||||
|
- `async def` 里真的 await 了 IO 的东西
|
||||||
|
|
||||||
|
违反会被 import-linter 的 `packages.domain is pure` 契约在 CI 拦下。
|
||||||
|
|
||||||
|
## 为什么这么严
|
||||||
|
|
||||||
|
这是 AI 大量写代码时唯一还能快速验证正确性的地方 —— 没有 mock、没有
|
||||||
|
fixture、没有容器,几毫秒跑完几百个用例。一旦让 IO 渗进来,这个性质就没了,
|
||||||
|
后面所有业务规则的测试都会退化成集成测试。
|
||||||
|
|
||||||
|
## mypy strict
|
||||||
|
|
||||||
|
本目录与 `packages/llm/` 一起,是 mypy **strict** 模式覆盖的范围
|
||||||
|
(架构 §16.2)。所有函数必须有完整类型注解。
|
||||||
|
|
||||||
|
## 现有内容
|
||||||
|
|
||||||
|
| 模块 | 内容 |
|
||||||
|
| --- | --- |
|
||||||
|
| `ids.py` | 雪花 ID 生成(§6.1 原则 2) |
|
||||||
|
| `errors.py` | 纯领域异常,适配层负责翻译成 `AppError` |
|
||||||
|
| `billing/quota.py` | 配额三段式的算术与周期键(§7.4) |
|
||||||
|
| `task/state.py` | `tk_task` 状态机(§6.2) |
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
"""纯领域逻辑:无 IO、无框架、可纯单测(架构文档 §3.2)。"""
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
"""配额与账本的纯逻辑(架构文档 §7.4)。"""
|
||||||
Executable
+140
@@ -0,0 +1,140 @@
|
|||||||
|
"""Quota arithmetic — pure logic, no IO (架构文档 §7.4 V1 配额三段式).
|
||||||
|
|
||||||
|
The three-phase protocol lives here as data transformations so it can be unit
|
||||||
|
tested without a database:
|
||||||
|
|
||||||
|
reserve → 冻结额度并写 bl_ledger(status=reserved)
|
||||||
|
commit → 确认扣除 (status=committed)
|
||||||
|
refund → 释放额度并写反向流水 (tx_type=refund)
|
||||||
|
|
||||||
|
V1 doesn't involve money, but 用户对「次数被白扣」的敏感度不比扣钱低,and V1
|
||||||
|
has no customer-service refund path — the mechanism is the only guarantee.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass, replace
|
||||||
|
from datetime import UTC, datetime, timedelta, timezone
|
||||||
|
from typing import Literal
|
||||||
|
|
||||||
|
from packages.domain.errors import QuotaExceededError
|
||||||
|
|
||||||
|
PeriodType = Literal["day", "month", "lifetime"]
|
||||||
|
LedgerStatus = Literal["reserved", "committed", "refunded"]
|
||||||
|
TxType = Literal["system_grant", "consume", "refund", "purchase", "gift"]
|
||||||
|
|
||||||
|
#: FR-BILL-01: 跨时区按 UTC+8 统一。
|
||||||
|
DEFAULT_TZ_OFFSET_HOURS = 8
|
||||||
|
|
||||||
|
|
||||||
|
def period_key(
|
||||||
|
now: datetime,
|
||||||
|
period_type: PeriodType,
|
||||||
|
tz_offset_hours: int = DEFAULT_TZ_OFFSET_HOURS,
|
||||||
|
) -> str:
|
||||||
|
"""Bucket key that makes quota reset naturally, with no cron job.
|
||||||
|
|
||||||
|
``day`` → ``d:2026-08-13``; ``month`` → ``m:2026-08``; ``lifetime`` → ``life``.
|
||||||
|
"""
|
||||||
|
if period_type == "lifetime":
|
||||||
|
return "life"
|
||||||
|
local = now.astimezone(UTC) + timedelta(hours=tz_offset_hours)
|
||||||
|
if period_type == "day":
|
||||||
|
return f"d:{local:%Y-%m-%d}"
|
||||||
|
if period_type == "month":
|
||||||
|
return f"m:{local:%Y-%m}"
|
||||||
|
msg = f"unknown period_type: {period_type}"
|
||||||
|
raise ValueError(msg)
|
||||||
|
|
||||||
|
|
||||||
|
def period_reset_at(
|
||||||
|
now: datetime,
|
||||||
|
period_type: PeriodType,
|
||||||
|
tz_offset_hours: int = DEFAULT_TZ_OFFSET_HOURS,
|
||||||
|
) -> datetime | None:
|
||||||
|
"""When the current period rolls over (used for the `reset_at` hint)."""
|
||||||
|
if period_type == "lifetime":
|
||||||
|
return None
|
||||||
|
tz = timezone(timedelta(hours=tz_offset_hours))
|
||||||
|
local = now.astimezone(tz)
|
||||||
|
if period_type == "day":
|
||||||
|
nxt = (local + timedelta(days=1)).replace(hour=0, minute=0, second=0, microsecond=0)
|
||||||
|
else:
|
||||||
|
year, month = (local.year + 1, 1) if local.month == 12 else (local.year, local.month + 1)
|
||||||
|
nxt = local.replace(
|
||||||
|
year=year, month=month, day=1, hour=0, minute=0, second=0, microsecond=0
|
||||||
|
)
|
||||||
|
return nxt.astimezone(UTC)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class QuotaState:
|
||||||
|
"""Current period counters for one (user, quota_key)."""
|
||||||
|
|
||||||
|
used: int
|
||||||
|
base_limit: int
|
||||||
|
bonus: int = 0
|
||||||
|
|
||||||
|
@property
|
||||||
|
def effective_limit(self) -> int:
|
||||||
|
return self.base_limit + self.bonus
|
||||||
|
|
||||||
|
@property
|
||||||
|
def remaining(self) -> int:
|
||||||
|
return max(self.effective_limit - self.used, 0)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class QuotaDecision:
|
||||||
|
allowed: bool
|
||||||
|
required: int
|
||||||
|
remaining: int
|
||||||
|
effective_limit: int
|
||||||
|
|
||||||
|
|
||||||
|
def evaluate(state: QuotaState, amount: int) -> QuotaDecision:
|
||||||
|
"""Server-side hard check (FR-BILL-06). Client hints are UX only."""
|
||||||
|
if amount <= 0:
|
||||||
|
msg = "amount must be positive"
|
||||||
|
raise ValueError(msg)
|
||||||
|
return QuotaDecision(
|
||||||
|
allowed=state.used + amount <= state.effective_limit,
|
||||||
|
required=amount,
|
||||||
|
remaining=state.remaining,
|
||||||
|
effective_limit=state.effective_limit,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def reserve(state: QuotaState, amount: int) -> QuotaState:
|
||||||
|
"""Freeze ``amount`` units; raises :class:`QuotaExceededError` when over the limit."""
|
||||||
|
decision = evaluate(state, amount)
|
||||||
|
if not decision.allowed:
|
||||||
|
raise QuotaExceededError(required=amount, remaining=decision.remaining)
|
||||||
|
return replace(state, used=state.used + amount)
|
||||||
|
|
||||||
|
|
||||||
|
def refund(state: QuotaState, amount: int) -> QuotaState:
|
||||||
|
"""Release a previously reserved amount. Never drives ``used`` below zero."""
|
||||||
|
if amount <= 0:
|
||||||
|
msg = "amount must be positive"
|
||||||
|
raise ValueError(msg)
|
||||||
|
return replace(state, used=max(state.used - amount, 0))
|
||||||
|
|
||||||
|
|
||||||
|
def balance_after(state: QuotaState) -> int:
|
||||||
|
"""Value written to ``bl_ledger.balance_after``.
|
||||||
|
|
||||||
|
V1 has no wallet balance, so the ledger records *remaining quota* — that is
|
||||||
|
the number the user actually sees ("今日剩余 N 次"). When V2 introduces
|
||||||
|
``bl_wallet`` this becomes the wallet balance without a schema change.
|
||||||
|
"""
|
||||||
|
return state.remaining
|
||||||
|
|
||||||
|
|
||||||
|
def reserve_idempotency_key(*, user_id: int, quota_key: str, request_key: str) -> str:
|
||||||
|
return f"rsv:{user_id}:{quota_key}:{request_key}"
|
||||||
|
|
||||||
|
|
||||||
|
def refund_idempotency_key(reserve_key: str) -> str:
|
||||||
|
"""FR-BILL-05: 退还必须幂等。Derived, so a retry cannot double-refund."""
|
||||||
|
return f"rfd:{reserve_key}"
|
||||||
Executable
+26
@@ -0,0 +1,26 @@
|
|||||||
|
"""Pure-domain exceptions.
|
||||||
|
|
||||||
|
This layer must not import :mod:`packages.shared` (enforced by import-linter),
|
||||||
|
so it defines its own error types. Adapters translate them into
|
||||||
|
:class:`packages.shared.errors.AppError` at the boundary.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
|
||||||
|
class DomainError(Exception):
|
||||||
|
"""Base class for all pure-domain rule violations."""
|
||||||
|
|
||||||
|
|
||||||
|
class QuotaExceededError(DomainError):
|
||||||
|
def __init__(self, *, required: int, remaining: int) -> None:
|
||||||
|
self.required = required
|
||||||
|
self.remaining = remaining
|
||||||
|
super().__init__(f"quota exceeded: required={required} remaining={remaining}")
|
||||||
|
|
||||||
|
|
||||||
|
class InvalidTaskTransitionError(DomainError):
|
||||||
|
def __init__(self, *, current: str, target: str) -> None:
|
||||||
|
self.current = current
|
||||||
|
self.target = target
|
||||||
|
super().__init__(f"invalid task transition: {current} -> {target}")
|
||||||
Executable
+85
@@ -0,0 +1,85 @@
|
|||||||
|
"""Snowflake IDs (架构文档 §6.1 原则 2).
|
||||||
|
|
||||||
|
Auto-increment keys invite enumeration attacks and collide when the database is
|
||||||
|
later sharded; UUID primary keys have poor index locality. A 64-bit snowflake
|
||||||
|
avoids both.
|
||||||
|
|
||||||
|
Layout: 41 bits ms since epoch | 10 bits node id | 12 bits sequence.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
from collections.abc import Callable
|
||||||
|
|
||||||
|
#: 2025-01-01T00:00:00Z in milliseconds. Do not change once IDs are issued.
|
||||||
|
DEFAULT_EPOCH_MS = 1_735_689_600_000
|
||||||
|
|
||||||
|
_NODE_BITS = 10
|
||||||
|
_SEQUENCE_BITS = 12
|
||||||
|
_MAX_NODE_ID = (1 << _NODE_BITS) - 1
|
||||||
|
_SEQUENCE_MASK = (1 << _SEQUENCE_BITS) - 1
|
||||||
|
|
||||||
|
|
||||||
|
def _wall_clock_ms() -> int:
|
||||||
|
return int(time.time() * 1000)
|
||||||
|
|
||||||
|
|
||||||
|
class SnowflakeGenerator:
|
||||||
|
"""Thread-safe monotonic 64-bit id generator."""
|
||||||
|
|
||||||
|
__slots__ = ("_clock", "_epoch_ms", "_last_ms", "_lock", "_node_id", "_sequence")
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
node_id: int = 1,
|
||||||
|
epoch_ms: int = DEFAULT_EPOCH_MS,
|
||||||
|
clock: Callable[[], int] | None = None,
|
||||||
|
) -> None:
|
||||||
|
if not 0 <= node_id <= _MAX_NODE_ID:
|
||||||
|
msg = f"node_id must be within 0..{_MAX_NODE_ID}"
|
||||||
|
raise ValueError(msg)
|
||||||
|
self._node_id = node_id
|
||||||
|
self._epoch_ms = epoch_ms
|
||||||
|
self._sequence = 0
|
||||||
|
self._last_ms = -1
|
||||||
|
self._lock = threading.Lock()
|
||||||
|
self._clock = clock or _wall_clock_ms
|
||||||
|
|
||||||
|
def _now_ms(self) -> int:
|
||||||
|
return self._clock()
|
||||||
|
|
||||||
|
def next_id(self) -> int:
|
||||||
|
with self._lock:
|
||||||
|
now = self._now_ms()
|
||||||
|
if now < self._last_ms:
|
||||||
|
# Clock moved backwards: wait it out rather than issue duplicates.
|
||||||
|
now = self._last_ms
|
||||||
|
if now == self._last_ms:
|
||||||
|
self._sequence = (self._sequence + 1) & _SEQUENCE_MASK
|
||||||
|
if self._sequence == 0:
|
||||||
|
while now <= self._last_ms:
|
||||||
|
now = self._now_ms()
|
||||||
|
else:
|
||||||
|
self._sequence = 0
|
||||||
|
self._last_ms = now
|
||||||
|
elapsed = now - self._epoch_ms
|
||||||
|
return (
|
||||||
|
(elapsed << (_NODE_BITS + _SEQUENCE_BITS))
|
||||||
|
| (self._node_id << _SEQUENCE_BITS)
|
||||||
|
| self._sequence
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
_default_generator = SnowflakeGenerator()
|
||||||
|
|
||||||
|
|
||||||
|
def next_id() -> int:
|
||||||
|
"""Process-wide default generator."""
|
||||||
|
return _default_generator.next_id()
|
||||||
|
|
||||||
|
|
||||||
|
def timestamp_ms_of(snowflake_id: int, epoch_ms: int = DEFAULT_EPOCH_MS) -> int:
|
||||||
|
"""Extract the creation timestamp from an id (useful for partition pruning)."""
|
||||||
|
return (snowflake_id >> (_NODE_BITS + _SEQUENCE_BITS)) + epoch_ms
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
"""统一任务中心的状态机(架构文档 §6.2)。"""
|
||||||
Executable
+76
@@ -0,0 +1,76 @@
|
|||||||
|
"""tk_task state machine — pure logic (架构文档 §6.2 task 域, FR-TASK-01).
|
||||||
|
|
||||||
|
统一任务中心 is the only long-running-task view in the system; every long task
|
||||||
|
(report / tryon / hairstyle / matting / render) moves through these states.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from enum import StrEnum
|
||||||
|
|
||||||
|
from packages.domain.errors import InvalidTaskTransitionError
|
||||||
|
|
||||||
|
|
||||||
|
class TaskStatus(StrEnum):
|
||||||
|
PENDING = "pending" # created, quota reserved, not yet queued
|
||||||
|
QUEUED = "queued" # handed to Celery
|
||||||
|
RUNNING = "running"
|
||||||
|
SUCCEEDED = "succeeded"
|
||||||
|
FAILED = "failed"
|
||||||
|
CANCELED = "canceled"
|
||||||
|
|
||||||
|
|
||||||
|
class TaskType(StrEnum):
|
||||||
|
REPORT = "report"
|
||||||
|
TRYON = "tryon"
|
||||||
|
HAIRSTYLE = "hairstyle"
|
||||||
|
MATTING = "matting"
|
||||||
|
RENDER = "render"
|
||||||
|
PING = "ping" # scaffold-only: proves the async path end to end
|
||||||
|
|
||||||
|
|
||||||
|
TERMINAL_STATUSES: frozenset[TaskStatus] = frozenset(
|
||||||
|
{TaskStatus.SUCCEEDED, TaskStatus.FAILED, TaskStatus.CANCELED}
|
||||||
|
)
|
||||||
|
|
||||||
|
#: Refunds are owed for every terminal state other than success (FR-TRY-05).
|
||||||
|
REFUNDABLE_STATUSES: frozenset[TaskStatus] = frozenset({TaskStatus.FAILED, TaskStatus.CANCELED})
|
||||||
|
|
||||||
|
ALLOWED_TRANSITIONS: dict[TaskStatus, frozenset[TaskStatus]] = {
|
||||||
|
TaskStatus.PENDING: frozenset({TaskStatus.QUEUED, TaskStatus.FAILED, TaskStatus.CANCELED}),
|
||||||
|
TaskStatus.QUEUED: frozenset({TaskStatus.RUNNING, TaskStatus.FAILED, TaskStatus.CANCELED}),
|
||||||
|
TaskStatus.RUNNING: frozenset(
|
||||||
|
{TaskStatus.SUCCEEDED, TaskStatus.FAILED, TaskStatus.CANCELED, TaskStatus.QUEUED}
|
||||||
|
),
|
||||||
|
TaskStatus.SUCCEEDED: frozenset(),
|
||||||
|
TaskStatus.FAILED: frozenset({TaskStatus.QUEUED}), # retry re-queues
|
||||||
|
TaskStatus.CANCELED: frozenset(),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def can_transition(current: TaskStatus, target: TaskStatus) -> bool:
|
||||||
|
return target in ALLOWED_TRANSITIONS[current]
|
||||||
|
|
||||||
|
|
||||||
|
def assert_transition(current: TaskStatus, target: TaskStatus) -> None:
|
||||||
|
if not can_transition(current, target):
|
||||||
|
raise InvalidTaskTransitionError(current=current.value, target=target.value)
|
||||||
|
|
||||||
|
|
||||||
|
def is_terminal(status: TaskStatus) -> bool:
|
||||||
|
return status in TERMINAL_STATUSES
|
||||||
|
|
||||||
|
|
||||||
|
def needs_refund(status: TaskStatus) -> bool:
|
||||||
|
return status in REFUNDABLE_STATUSES
|
||||||
|
|
||||||
|
|
||||||
|
def progress_for(status: TaskStatus) -> int:
|
||||||
|
return {
|
||||||
|
TaskStatus.PENDING: 0,
|
||||||
|
TaskStatus.QUEUED: 5,
|
||||||
|
TaskStatus.RUNNING: 50,
|
||||||
|
TaskStatus.SUCCEEDED: 100,
|
||||||
|
TaskStatus.FAILED: 100,
|
||||||
|
TaskStatus.CANCELED: 100,
|
||||||
|
}[status]
|
||||||
Executable
+40
@@ -0,0 +1,40 @@
|
|||||||
|
# `packages/shared/` — 横切基础设施
|
||||||
|
|
||||||
|
配置、日志、错误码、统一响应、数据库、Redis、可观测性。架构 §16.1。
|
||||||
|
|
||||||
|
| 模块 | 内容 |
|
||||||
|
| --- | --- |
|
||||||
|
| `config.py` | pydantic-settings 类型化配置,全部来自环境变量 |
|
||||||
|
| `errors.py` | 统一错误码(§9.2:4xxxx 客户端 / 5xxxx 服务端 / 6xxxx 外部依赖) |
|
||||||
|
| `response.py` | 统一响应信封 `{code, message, data, request_id, hint}` |
|
||||||
|
| `handlers.py` | FastAPI 异常处理器,保证任何异常都返回信封且不泄露堆栈 |
|
||||||
|
| `middleware.py` | 请求 ID / 追踪 ID、访问日志、HTTP 指标、`/v1` 路径别名 |
|
||||||
|
| `logging.py` | structlog JSON 日志 + **脱敏处理器** |
|
||||||
|
| `context.py` | 请求上下文(request_id / trace_id / user_id / task_id) |
|
||||||
|
| `db.py` | 异步 SQLAlchemy 引擎与会话 |
|
||||||
|
| `models.py` | ORM 表定义 |
|
||||||
|
| `redis_client.py` | Redis 连接 |
|
||||||
|
| `observability.py` | OTel tracing + Prometheus 指标注册表 |
|
||||||
|
|
||||||
|
## 能放什么
|
||||||
|
|
||||||
|
跨所有域复用、且不含任何业务判断的东西。
|
||||||
|
|
||||||
|
## 不能放什么
|
||||||
|
|
||||||
|
- **任何业务规则**。「配额够不够」属于 `packages/domain/billing`,
|
||||||
|
「任务能不能取消」属于 `packages/domain/task`。放进来就没人能纯单测了。
|
||||||
|
- 对具体业务域的 import。`shared` 在依赖层级的底部,反向依赖会立刻形成环。
|
||||||
|
|
||||||
|
## 日志脱敏不是可选项
|
||||||
|
|
||||||
|
`logging.py` 的脱敏是 structlog **处理器**,作用于每一条日志,没有 opt-in。
|
||||||
|
新增敏感字段时加进 `SENSITIVE_KEYS`,或用 `LOG_EXTRA_REDACT_KEYS` 环境变量。
|
||||||
|
NFR-SEC-03 / 架构 §11.2:手机号、UnionID、图片地址、诊断结论、令牌一律脱敏。
|
||||||
|
|
||||||
|
## 新增错误码
|
||||||
|
|
||||||
|
在 `ErrorCode` 加值,**同时**在 `ERROR_META` 加 message 和 hint。
|
||||||
|
测试 `test_every_error_code_has_metadata` 会强制这一点。
|
||||||
|
FR-BILL-07:配额相关文案里不能出现价格、购买、会员、升级字样,
|
||||||
|
`test_quota_hint_never_mentions_price_or_purchase` 会扫描全部文案。
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
"""横切基础设施:配置、日志、错误、响应、DB、Redis、可观测。"""
|
||||||
Executable
+148
@@ -0,0 +1,148 @@
|
|||||||
|
"""Typed application configuration.
|
||||||
|
|
||||||
|
Every value comes from the environment (架构文档 §11.2: 密钥不进代码/镜像).
|
||||||
|
`.env` is only a developer convenience; dev/staging/prod inject via SSM/KMS.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from decimal import Decimal
|
||||||
|
from functools import lru_cache
|
||||||
|
from typing import Any, Literal
|
||||||
|
|
||||||
|
from pydantic import Field, ValidationInfo, field_validator
|
||||||
|
from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||||
|
|
||||||
|
Environment = Literal["local", "dev", "staging", "prod"]
|
||||||
|
CachePrefixMode = Literal["strict", "warn", "off"]
|
||||||
|
|
||||||
|
_PLACEHOLDER_JWT_SECRET = "change-me-local-only-do-not-use-outside-local" # noqa: S105
|
||||||
|
|
||||||
|
|
||||||
|
class Settings(BaseSettings):
|
||||||
|
"""Root settings object. Read it through :func:`get_settings`."""
|
||||||
|
|
||||||
|
model_config = SettingsConfigDict(
|
||||||
|
env_file=".env",
|
||||||
|
env_file_encoding="utf-8",
|
||||||
|
extra="ignore",
|
||||||
|
case_sensitive=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
# --- Application -------------------------------------------------------
|
||||||
|
app_env: Environment = "local"
|
||||||
|
app_name: str = "meida-backend"
|
||||||
|
api_prefix: str = "/api/v1"
|
||||||
|
debug: bool = False
|
||||||
|
log_level: str = "INFO"
|
||||||
|
log_json: bool = True
|
||||||
|
log_extra_redact_keys: str = ""
|
||||||
|
|
||||||
|
# --- Database ----------------------------------------------------------
|
||||||
|
database_url: str = "postgresql+asyncpg://meida:meida@localhost:55432/meida"
|
||||||
|
db_pool_size: int = 10
|
||||||
|
db_max_overflow: int = 20
|
||||||
|
db_echo: bool = False
|
||||||
|
|
||||||
|
# --- Redis / Celery ----------------------------------------------------
|
||||||
|
redis_url: str = "redis://localhost:56379/0"
|
||||||
|
celery_broker_url: str = "redis://localhost:56379/1"
|
||||||
|
celery_result_backend: str = "redis://localhost:56379/2"
|
||||||
|
|
||||||
|
# --- Auth --------------------------------------------------------------
|
||||||
|
jwt_secret: str = _PLACEHOLDER_JWT_SECRET
|
||||||
|
jwt_algorithm: str = "HS256"
|
||||||
|
jwt_access_ttl_seconds: int = 900
|
||||||
|
jwt_issuer: str = "meida"
|
||||||
|
dev_auth_enabled: bool = True
|
||||||
|
|
||||||
|
# --- LLM routing -------------------------------------------------------
|
||||||
|
llm_primary_provider: str = "deepseek"
|
||||||
|
llm_backup_provider: str = "zhipu"
|
||||||
|
llm_structured_max_retries: int = 2
|
||||||
|
llm_circuit_failure_threshold: int = 3
|
||||||
|
llm_circuit_cooldown_seconds: float = 30.0
|
||||||
|
llm_scene_overrides: dict[str, dict[str, Any]] = Field(
|
||||||
|
default_factory=dict, alias="LLM_SCENE_OVERRIDES_JSON"
|
||||||
|
)
|
||||||
|
|
||||||
|
# --- DeepSeek (primary, OpenAI-compatible) -----------------------------
|
||||||
|
deepseek_api_key: str = ""
|
||||||
|
deepseek_base_url: str = "https://api.deepseek.com/v1"
|
||||||
|
deepseek_model: str = "deepseek-v4-flash"
|
||||||
|
deepseek_timeout_seconds: float = 30.0
|
||||||
|
|
||||||
|
# --- 智谱 GLM (backup slot, not implemented in V1) ---------------------
|
||||||
|
zhipu_api_key: str = ""
|
||||||
|
zhipu_base_url: str = "https://open.bigmodel.cn/api/paas/v4"
|
||||||
|
zhipu_model: str = "glm-4-flash"
|
||||||
|
zhipu_timeout_seconds: float = 30.0
|
||||||
|
|
||||||
|
# --- Prompt caching discipline (架构文档 §7.2) -------------------------
|
||||||
|
llm_cache_prefix_mode: CachePrefixMode = "strict"
|
||||||
|
prompts_dir: str = "prompts"
|
||||||
|
|
||||||
|
# --- Cost attribution (架构文档 §13) -----------------------------------
|
||||||
|
llm_cost_rates: dict[str, dict[str, Decimal]] = Field(
|
||||||
|
default_factory=dict, alias="LLM_COST_RATES_JSON"
|
||||||
|
)
|
||||||
|
llm_default_cost_rate: dict[str, Decimal] = Field(
|
||||||
|
default_factory=lambda: {
|
||||||
|
"input_cents_per_mtok": Decimal("100"),
|
||||||
|
"cached_input_cents_per_mtok": Decimal("20"),
|
||||||
|
"output_cents_per_mtok": Decimal("200"),
|
||||||
|
},
|
||||||
|
alias="LLM_DEFAULT_COST_RATE_JSON",
|
||||||
|
)
|
||||||
|
|
||||||
|
# --- Global daily cost circuit breaker ---------------------------------
|
||||||
|
llm_daily_budget_enabled: bool = True
|
||||||
|
llm_daily_budget_cents: Decimal = Decimal("100000")
|
||||||
|
llm_budget_timezone_offset_hours: int = 8
|
||||||
|
|
||||||
|
# --- Observability -----------------------------------------------------
|
||||||
|
otel_enabled: bool = False
|
||||||
|
otel_service_name: str = "meida-api"
|
||||||
|
otel_exporter_otlp_endpoint: str = ""
|
||||||
|
prometheus_enabled: bool = True
|
||||||
|
|
||||||
|
@field_validator(
|
||||||
|
"llm_scene_overrides", "llm_cost_rates", "llm_default_cost_rate", mode="before"
|
||||||
|
)
|
||||||
|
@classmethod
|
||||||
|
def _parse_json_object(cls, value: Any) -> Any:
|
||||||
|
if isinstance(value, str):
|
||||||
|
stripped = value.strip()
|
||||||
|
if not stripped:
|
||||||
|
return {}
|
||||||
|
return json.loads(stripped)
|
||||||
|
return value
|
||||||
|
|
||||||
|
@field_validator("jwt_secret")
|
||||||
|
@classmethod
|
||||||
|
def _reject_placeholder_secret_outside_local(cls, value: str, info: ValidationInfo) -> str:
|
||||||
|
env = info.data.get("app_env", "local")
|
||||||
|
if env in ("staging", "prod") and (value == _PLACEHOLDER_JWT_SECRET or len(value) < 32):
|
||||||
|
msg = "JWT_SECRET must be set to a >=32 char random value outside local/dev"
|
||||||
|
raise ValueError(msg)
|
||||||
|
return value
|
||||||
|
|
||||||
|
@property
|
||||||
|
def is_local(self) -> bool:
|
||||||
|
return self.app_env in ("local", "dev")
|
||||||
|
|
||||||
|
@property
|
||||||
|
def extra_redact_keys(self) -> tuple[str, ...]:
|
||||||
|
return tuple(k.strip() for k in self.log_extra_redact_keys.split(",") if k.strip())
|
||||||
|
|
||||||
|
@property
|
||||||
|
def sync_database_url(self) -> str:
|
||||||
|
"""psycopg-style URL, used by tooling that cannot speak asyncpg."""
|
||||||
|
return self.database_url.replace("+asyncpg", "")
|
||||||
|
|
||||||
|
|
||||||
|
@lru_cache(maxsize=1)
|
||||||
|
def get_settings() -> Settings:
|
||||||
|
"""Process-wide settings singleton."""
|
||||||
|
return Settings()
|
||||||
Executable
+75
@@ -0,0 +1,75 @@
|
|||||||
|
"""Request-scoped context propagated into logs and traces (架构文档 §15)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import uuid
|
||||||
|
from contextvars import ContextVar, Token
|
||||||
|
from dataclasses import dataclass
|
||||||
|
|
||||||
|
_request_id: ContextVar[str | None] = ContextVar("meida_request_id", default=None)
|
||||||
|
_trace_id: ContextVar[str | None] = ContextVar("meida_trace_id", default=None)
|
||||||
|
_user_id: ContextVar[int | None] = ContextVar("meida_user_id", default=None)
|
||||||
|
_task_id: ContextVar[int | None] = ContextVar("meida_task_id", default=None)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(slots=True)
|
||||||
|
class ContextTokens:
|
||||||
|
request_id: Token[str | None]
|
||||||
|
trace_id: Token[str | None]
|
||||||
|
user_id: Token[int | None]
|
||||||
|
task_id: Token[int | None]
|
||||||
|
|
||||||
|
|
||||||
|
def new_request_id() -> str:
|
||||||
|
return uuid.uuid4().hex
|
||||||
|
|
||||||
|
|
||||||
|
def bind_context(
|
||||||
|
*,
|
||||||
|
request_id: str | None = None,
|
||||||
|
trace_id: str | None = None,
|
||||||
|
user_id: int | None = None,
|
||||||
|
task_id: int | None = None,
|
||||||
|
) -> ContextTokens:
|
||||||
|
return ContextTokens(
|
||||||
|
request_id=_request_id.set(request_id if request_id is not None else _request_id.get()),
|
||||||
|
trace_id=_trace_id.set(trace_id if trace_id is not None else _trace_id.get()),
|
||||||
|
user_id=_user_id.set(user_id if user_id is not None else _user_id.get()),
|
||||||
|
task_id=_task_id.set(task_id if task_id is not None else _task_id.get()),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def reset_context(tokens: ContextTokens) -> None:
|
||||||
|
_request_id.reset(tokens.request_id)
|
||||||
|
_trace_id.reset(tokens.trace_id)
|
||||||
|
_user_id.reset(tokens.user_id)
|
||||||
|
_task_id.reset(tokens.task_id)
|
||||||
|
|
||||||
|
|
||||||
|
def get_request_id() -> str | None:
|
||||||
|
return _request_id.get()
|
||||||
|
|
||||||
|
|
||||||
|
def get_trace_id() -> str | None:
|
||||||
|
return _trace_id.get()
|
||||||
|
|
||||||
|
|
||||||
|
def get_user_id() -> int | None:
|
||||||
|
return _user_id.get()
|
||||||
|
|
||||||
|
|
||||||
|
def get_task_id() -> int | None:
|
||||||
|
return _task_id.get()
|
||||||
|
|
||||||
|
|
||||||
|
def context_snapshot() -> dict[str, str | int]:
|
||||||
|
snapshot: dict[str, str | int] = {}
|
||||||
|
for key, value in (
|
||||||
|
("request_id", _request_id.get()),
|
||||||
|
("trace_id", _trace_id.get()),
|
||||||
|
("user_id", _user_id.get()),
|
||||||
|
("task_id", _task_id.get()),
|
||||||
|
):
|
||||||
|
if value is not None:
|
||||||
|
snapshot[key] = value
|
||||||
|
return snapshot
|
||||||
Executable
+73
@@ -0,0 +1,73 @@
|
|||||||
|
"""Async SQLAlchemy engine and session management (架构文档 §5, §12.2).
|
||||||
|
|
||||||
|
The engine is created lazily so that importing this module never opens a
|
||||||
|
connection — important for unit tests and for Celery workers that fork.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from collections.abc import AsyncIterator
|
||||||
|
from contextlib import asynccontextmanager
|
||||||
|
|
||||||
|
from sqlalchemy.ext.asyncio import (
|
||||||
|
AsyncEngine,
|
||||||
|
AsyncSession,
|
||||||
|
async_sessionmaker,
|
||||||
|
create_async_engine,
|
||||||
|
)
|
||||||
|
|
||||||
|
from packages.shared.config import Settings, get_settings
|
||||||
|
|
||||||
|
_engine: AsyncEngine | None = None
|
||||||
|
_session_factory: async_sessionmaker[AsyncSession] | None = None
|
||||||
|
|
||||||
|
|
||||||
|
def create_engine(settings: Settings | None = None) -> AsyncEngine:
|
||||||
|
cfg = settings or get_settings()
|
||||||
|
return create_async_engine(
|
||||||
|
cfg.database_url,
|
||||||
|
echo=cfg.db_echo,
|
||||||
|
pool_size=cfg.db_pool_size,
|
||||||
|
max_overflow=cfg.db_max_overflow,
|
||||||
|
pool_pre_ping=True,
|
||||||
|
pool_recycle=1800,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def get_engine() -> AsyncEngine:
|
||||||
|
global _engine
|
||||||
|
if _engine is None:
|
||||||
|
_engine = create_engine()
|
||||||
|
return _engine
|
||||||
|
|
||||||
|
|
||||||
|
def get_session_factory() -> async_sessionmaker[AsyncSession]:
|
||||||
|
global _session_factory
|
||||||
|
if _session_factory is None:
|
||||||
|
_session_factory = async_sessionmaker(
|
||||||
|
bind=get_engine(),
|
||||||
|
expire_on_commit=False,
|
||||||
|
autoflush=False,
|
||||||
|
)
|
||||||
|
return _session_factory
|
||||||
|
|
||||||
|
|
||||||
|
@asynccontextmanager
|
||||||
|
async def session_scope() -> AsyncIterator[AsyncSession]:
|
||||||
|
"""Transactional scope: commit on success, rollback on error."""
|
||||||
|
factory = get_session_factory()
|
||||||
|
async with factory() as session:
|
||||||
|
try:
|
||||||
|
yield session
|
||||||
|
await session.commit()
|
||||||
|
except Exception:
|
||||||
|
await session.rollback()
|
||||||
|
raise
|
||||||
|
|
||||||
|
|
||||||
|
async def dispose_engine() -> None:
|
||||||
|
global _engine, _session_factory
|
||||||
|
if _engine is not None:
|
||||||
|
await _engine.dispose()
|
||||||
|
_engine = None
|
||||||
|
_session_factory = None
|
||||||
Executable
+138
@@ -0,0 +1,138 @@
|
|||||||
|
"""Unified error codes (架构文档 §9.2, FR-SYS-02).
|
||||||
|
|
||||||
|
Segments:
|
||||||
|
``4xxxx`` client error, ``5xxxx`` server error, ``6xxxx`` external dependency.
|
||||||
|
|
||||||
|
Splitting 5xxxx from 6xxxx lets monitoring separate "our bug" from "vendor
|
||||||
|
outage" without parsing messages.
|
||||||
|
|
||||||
|
Every code carries a user-readable ``message`` and an optional ``hint`` telling
|
||||||
|
the user what to do next — required by FR-SYS-02 and NFR-UX-02.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from enum import IntEnum
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
|
||||||
|
class ErrorCode(IntEnum):
|
||||||
|
OK = 0
|
||||||
|
|
||||||
|
# --- 4xxxx client -----------------------------------------------------
|
||||||
|
BAD_REQUEST = 40000
|
||||||
|
VALIDATION_FAILED = 40001
|
||||||
|
IDEMPOTENCY_CONFLICT = 40002
|
||||||
|
UNAUTHENTICATED = 40100
|
||||||
|
TOKEN_EXPIRED = 40101
|
||||||
|
TOKEN_INVALID = 40102
|
||||||
|
FORBIDDEN = 40300
|
||||||
|
QUOTA_EXHAUSTED = 40301
|
||||||
|
CONSENT_REQUIRED = 40302
|
||||||
|
MINOR_RESTRICTED = 40303
|
||||||
|
NOT_FOUND = 40400
|
||||||
|
STATE_CONFLICT = 40900
|
||||||
|
RATE_LIMITED = 42900
|
||||||
|
|
||||||
|
# --- 5xxxx server -----------------------------------------------------
|
||||||
|
INTERNAL_ERROR = 50000
|
||||||
|
DB_ERROR = 50001
|
||||||
|
TASK_STATE_INVALID = 50002
|
||||||
|
CONFIG_ERROR = 50003
|
||||||
|
NOT_IMPLEMENTED = 50100
|
||||||
|
|
||||||
|
# --- 6xxxx external dependency ---------------------------------------
|
||||||
|
UPSTREAM_ERROR = 60000
|
||||||
|
LLM_ALL_PROVIDERS_FAILED = 60001
|
||||||
|
LLM_STRUCTURED_OUTPUT_FAILED = 60002
|
||||||
|
LLM_DAILY_BUDGET_EXCEEDED = 60003
|
||||||
|
LLM_PROMPT_CACHE_VIOLATION = 60004
|
||||||
|
MODERATION_UNAVAILABLE = 60005
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class ErrorMeta:
|
||||||
|
http_status: int
|
||||||
|
message: str
|
||||||
|
hint: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
ERROR_META: dict[ErrorCode, ErrorMeta] = {
|
||||||
|
ErrorCode.OK: ErrorMeta(200, "ok"),
|
||||||
|
ErrorCode.BAD_REQUEST: ErrorMeta(400, "请求参数有误", "请检查后重试"),
|
||||||
|
ErrorCode.VALIDATION_FAILED: ErrorMeta(422, "请求参数校验未通过", "请按提示修改后重试"),
|
||||||
|
ErrorCode.IDEMPOTENCY_CONFLICT: ErrorMeta(
|
||||||
|
409, "相同请求正在处理中", "请稍候查看任务中心,不要重复提交"
|
||||||
|
),
|
||||||
|
ErrorCode.UNAUTHENTICATED: ErrorMeta(401, "请先登录", "登录后即可继续"),
|
||||||
|
ErrorCode.TOKEN_EXPIRED: ErrorMeta(401, "登录状态已过期", "请重新登录"),
|
||||||
|
ErrorCode.TOKEN_INVALID: ErrorMeta(401, "登录状态无效", "请重新登录"),
|
||||||
|
ErrorCode.FORBIDDEN: ErrorMeta(403, "没有访问权限"),
|
||||||
|
ErrorCode.QUOTA_EXHAUSTED: ErrorMeta(403, "今日额度已用完", "明天恢复,或完成任务多换几次"),
|
||||||
|
ErrorCode.CONSENT_REQUIRED: ErrorMeta(403, "需要先完成授权", "请阅读并同意后继续"),
|
||||||
|
ErrorCode.MINOR_RESTRICTED: ErrorMeta(403, "该功能未向未成年人开放"),
|
||||||
|
ErrorCode.NOT_FOUND: ErrorMeta(404, "内容不存在或已删除"),
|
||||||
|
ErrorCode.STATE_CONFLICT: ErrorMeta(409, "当前状态无法执行该操作"),
|
||||||
|
ErrorCode.RATE_LIMITED: ErrorMeta(429, "操作太频繁了", "请稍后再试"),
|
||||||
|
ErrorCode.INTERNAL_ERROR: ErrorMeta(500, "服务开小差了", "请稍后重试"),
|
||||||
|
ErrorCode.DB_ERROR: ErrorMeta(500, "服务开小差了", "请稍后重试"),
|
||||||
|
ErrorCode.TASK_STATE_INVALID: ErrorMeta(500, "任务状态异常", "请到任务中心重试"),
|
||||||
|
ErrorCode.CONFIG_ERROR: ErrorMeta(500, "服务配置异常", "请联系客服"),
|
||||||
|
ErrorCode.NOT_IMPLEMENTED: ErrorMeta(501, "该能力尚未开放"),
|
||||||
|
ErrorCode.UPSTREAM_ERROR: ErrorMeta(502, "外部服务暂时不可用", "请稍后重试"),
|
||||||
|
ErrorCode.LLM_ALL_PROVIDERS_FAILED: ErrorMeta(
|
||||||
|
502, "AI 服务暂时不可用", "请稍后重试,额度不会被扣除"
|
||||||
|
),
|
||||||
|
ErrorCode.LLM_STRUCTURED_OUTPUT_FAILED: ErrorMeta(
|
||||||
|
502, "AI 返回结果格式异常", "请稍后重试,额度不会被扣除"
|
||||||
|
),
|
||||||
|
ErrorCode.LLM_DAILY_BUDGET_EXCEEDED: ErrorMeta(
|
||||||
|
503, "AI 能力今日已达服务上限", "请明天再试,核心功能不受影响"
|
||||||
|
),
|
||||||
|
ErrorCode.LLM_PROMPT_CACHE_VIOLATION: ErrorMeta(500, "服务配置异常", "请联系客服"),
|
||||||
|
ErrorCode.MODERATION_UNAVAILABLE: ErrorMeta(
|
||||||
|
503, "内容安全服务不可用", "为保证安全本次请求已中止,请稍后重试"
|
||||||
|
),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
class AppError(Exception):
|
||||||
|
"""Base application error carrying a unified error code.
|
||||||
|
|
||||||
|
Never let a raw traceback reach the client (FR-SYS-02).
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
code: ErrorCode,
|
||||||
|
*,
|
||||||
|
message: str | None = None,
|
||||||
|
hint: str | None = None,
|
||||||
|
data: dict[str, Any] | None = None,
|
||||||
|
) -> None:
|
||||||
|
meta = ERROR_META.get(code, ERROR_META[ErrorCode.INTERNAL_ERROR])
|
||||||
|
self.code = code
|
||||||
|
self.message = message or meta.message
|
||||||
|
self.hint = hint if hint is not None else meta.hint
|
||||||
|
self.http_status = meta.http_status
|
||||||
|
self.data = data or {}
|
||||||
|
super().__init__(f"[{int(code)}] {self.message}")
|
||||||
|
|
||||||
|
|
||||||
|
class QuotaExhaustedError(AppError):
|
||||||
|
def __init__(self, *, required: int, remaining: int, reset_at: str | None = None) -> None:
|
||||||
|
super().__init__(
|
||||||
|
ErrorCode.QUOTA_EXHAUSTED,
|
||||||
|
data={"required": required, "remaining": remaining, "reset_at": reset_at},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class NotFoundError(AppError):
|
||||||
|
def __init__(self, message: str | None = None) -> None:
|
||||||
|
super().__init__(ErrorCode.NOT_FOUND, message=message)
|
||||||
|
|
||||||
|
|
||||||
|
class ForbiddenError(AppError):
|
||||||
|
def __init__(self, message: str | None = None) -> None:
|
||||||
|
super().__init__(ErrorCode.FORBIDDEN, message=message)
|
||||||
Executable
+69
@@ -0,0 +1,69 @@
|
|||||||
|
"""FastAPI exception handlers producing the unified envelope (架构文档 §9.2).
|
||||||
|
|
||||||
|
No handler is allowed to leak a stack trace to the client (FR-SYS-02).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from fastapi import FastAPI, Request
|
||||||
|
from fastapi.exceptions import RequestValidationError
|
||||||
|
from fastapi.responses import JSONResponse
|
||||||
|
from starlette.exceptions import HTTPException as StarletteHTTPException
|
||||||
|
|
||||||
|
from packages.shared.errors import ERROR_META, AppError, ErrorCode
|
||||||
|
from packages.shared.logging import get_logger
|
||||||
|
from packages.shared.response import fail
|
||||||
|
|
||||||
|
logger = get_logger(__name__)
|
||||||
|
|
||||||
|
_HTTP_STATUS_TO_CODE: dict[int, ErrorCode] = {
|
||||||
|
400: ErrorCode.BAD_REQUEST,
|
||||||
|
401: ErrorCode.UNAUTHENTICATED,
|
||||||
|
403: ErrorCode.FORBIDDEN,
|
||||||
|
404: ErrorCode.NOT_FOUND,
|
||||||
|
405: ErrorCode.BAD_REQUEST,
|
||||||
|
409: ErrorCode.STATE_CONFLICT,
|
||||||
|
422: ErrorCode.VALIDATION_FAILED,
|
||||||
|
429: ErrorCode.RATE_LIMITED,
|
||||||
|
501: ErrorCode.NOT_IMPLEMENTED,
|
||||||
|
502: ErrorCode.UPSTREAM_ERROR,
|
||||||
|
503: ErrorCode.UPSTREAM_ERROR,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _json(status: int, body: dict[str, Any]) -> JSONResponse:
|
||||||
|
return JSONResponse(status_code=status, content=body)
|
||||||
|
|
||||||
|
|
||||||
|
def install_exception_handlers(app: FastAPI) -> None:
|
||||||
|
@app.exception_handler(AppError)
|
||||||
|
async def _app_error(_request: Request, exc: AppError) -> JSONResponse:
|
||||||
|
logger.warning("app_error", code=int(exc.code), error_message=exc.message)
|
||||||
|
return _json(
|
||||||
|
exc.http_status,
|
||||||
|
fail(exc.code, message=exc.message, hint=exc.hint, data=exc.data or None),
|
||||||
|
)
|
||||||
|
|
||||||
|
@app.exception_handler(RequestValidationError)
|
||||||
|
async def _validation_error(_request: Request, exc: RequestValidationError) -> JSONResponse:
|
||||||
|
details = [
|
||||||
|
{"field": ".".join(str(p) for p in err.get("loc", ())), "reason": err.get("msg", "")}
|
||||||
|
for err in exc.errors()
|
||||||
|
]
|
||||||
|
return _json(
|
||||||
|
422,
|
||||||
|
fail(ErrorCode.VALIDATION_FAILED, data={"errors": details}),
|
||||||
|
)
|
||||||
|
|
||||||
|
@app.exception_handler(StarletteHTTPException)
|
||||||
|
async def _http_error(_request: Request, exc: StarletteHTTPException) -> JSONResponse:
|
||||||
|
code = _HTTP_STATUS_TO_CODE.get(exc.status_code, ErrorCode.INTERNAL_ERROR)
|
||||||
|
message = exc.detail if isinstance(exc.detail, str) else ERROR_META[code].message
|
||||||
|
return _json(exc.status_code, fail(code, message=message))
|
||||||
|
|
||||||
|
@app.exception_handler(Exception)
|
||||||
|
async def _unhandled(_request: Request, exc: Exception) -> JSONResponse:
|
||||||
|
logger.exception("unhandled_error", error_type=type(exc).__name__)
|
||||||
|
return _json(500, fail(ErrorCode.INTERNAL_ERROR))
|
||||||
Executable
+165
@@ -0,0 +1,165 @@
|
|||||||
|
"""Structured JSON logging with request/trace propagation and redaction.
|
||||||
|
|
||||||
|
NFR-SEC-03 / 架构文档 §11.2: 手机号、UnionID、图片地址、诊断结论、令牌
|
||||||
|
一律脱敏或不记录. Redaction runs as a structlog processor so it applies to
|
||||||
|
every log line regardless of who wrote it — there is no opt-in to forget.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
import re
|
||||||
|
import sys
|
||||||
|
from collections.abc import Iterable, MutableMapping
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import structlog
|
||||||
|
|
||||||
|
from packages.shared.context import context_snapshot
|
||||||
|
|
||||||
|
REDACTED = "***"
|
||||||
|
|
||||||
|
#: Field names whose values must never be logged in clear text.
|
||||||
|
SENSITIVE_KEYS: frozenset[str] = frozenset(
|
||||||
|
{
|
||||||
|
# credentials
|
||||||
|
"authorization",
|
||||||
|
"access_token",
|
||||||
|
"refresh_token",
|
||||||
|
"token",
|
||||||
|
"id_token",
|
||||||
|
"password",
|
||||||
|
"secret",
|
||||||
|
"api_key",
|
||||||
|
"apikey",
|
||||||
|
"jwt",
|
||||||
|
"purchase_token",
|
||||||
|
"refresh_token_hash",
|
||||||
|
# personal identifiers
|
||||||
|
"phone",
|
||||||
|
"phone_number",
|
||||||
|
"mobile",
|
||||||
|
"id_card",
|
||||||
|
"union_id",
|
||||||
|
"unionid",
|
||||||
|
"open_id",
|
||||||
|
"openid",
|
||||||
|
"device_fp",
|
||||||
|
"client_ip",
|
||||||
|
"nickname",
|
||||||
|
# face / image material (架构文档 §11.2, CR-02)
|
||||||
|
"face_image",
|
||||||
|
"face_image_key",
|
||||||
|
"face_key",
|
||||||
|
"avatar_key",
|
||||||
|
"image_key",
|
||||||
|
"image_url",
|
||||||
|
"photo_url",
|
||||||
|
"cos_key",
|
||||||
|
"presigned_url",
|
||||||
|
# diagnosis output is sensitive derived data
|
||||||
|
"diagnosis",
|
||||||
|
"dimensions",
|
||||||
|
"derived_features",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
_PHONE_RE = re.compile(r"(?<!\d)(1[3-9]\d)\d{4}(\d{4})(?!\d)")
|
||||||
|
_SK_RE = re.compile(r"\bsk-[A-Za-z0-9]{16,}\b")
|
||||||
|
_BEARER_RE = re.compile(r"(?i)\bbearer\s+[A-Za-z0-9._\-]{8,}")
|
||||||
|
|
||||||
|
|
||||||
|
def _scrub_text(value: str) -> str:
|
||||||
|
value = _SK_RE.sub("sk-" + REDACTED, value)
|
||||||
|
value = _BEARER_RE.sub("Bearer " + REDACTED, value)
|
||||||
|
return _PHONE_RE.sub(r"\1****\2", value)
|
||||||
|
|
||||||
|
|
||||||
|
def _redact_value(value: Any, sensitive_keys: frozenset[str], depth: int = 0) -> Any:
|
||||||
|
if depth > 6:
|
||||||
|
return value
|
||||||
|
if isinstance(value, str):
|
||||||
|
return _scrub_text(value)
|
||||||
|
if isinstance(value, dict):
|
||||||
|
return {
|
||||||
|
k: (
|
||||||
|
REDACTED
|
||||||
|
if isinstance(k, str) and k.lower() in sensitive_keys
|
||||||
|
else _redact_value(v, sensitive_keys, depth + 1)
|
||||||
|
)
|
||||||
|
for k, v in value.items()
|
||||||
|
}
|
||||||
|
if isinstance(value, (list, tuple)):
|
||||||
|
return type(value)(_redact_value(v, sensitive_keys, depth + 1) for v in value)
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def make_redaction_processor(
|
||||||
|
extra_keys: Iterable[str] = (),
|
||||||
|
) -> Any:
|
||||||
|
sensitive = SENSITIVE_KEYS | {k.lower() for k in extra_keys}
|
||||||
|
|
||||||
|
def processor(
|
||||||
|
_logger: Any, _name: str, event_dict: MutableMapping[str, Any]
|
||||||
|
) -> MutableMapping[str, Any]:
|
||||||
|
for key in list(event_dict):
|
||||||
|
if isinstance(key, str) and key.lower() in sensitive:
|
||||||
|
event_dict[key] = REDACTED
|
||||||
|
else:
|
||||||
|
event_dict[key] = _redact_value(event_dict[key], sensitive)
|
||||||
|
return event_dict
|
||||||
|
|
||||||
|
return processor
|
||||||
|
|
||||||
|
|
||||||
|
def _context_processor(
|
||||||
|
_logger: Any, _name: str, event_dict: MutableMapping[str, Any]
|
||||||
|
) -> MutableMapping[str, Any]:
|
||||||
|
for key, value in context_snapshot().items():
|
||||||
|
event_dict.setdefault(key, value)
|
||||||
|
return event_dict
|
||||||
|
|
||||||
|
|
||||||
|
def configure_logging(
|
||||||
|
*,
|
||||||
|
level: str = "INFO",
|
||||||
|
json_output: bool = True,
|
||||||
|
extra_redact_keys: Iterable[str] = (),
|
||||||
|
) -> None:
|
||||||
|
"""Idempotently configure stdlib logging + structlog."""
|
||||||
|
logging.basicConfig(
|
||||||
|
format="%(message)s",
|
||||||
|
stream=sys.stdout,
|
||||||
|
level=getattr(logging, level.upper(), logging.INFO),
|
||||||
|
force=True,
|
||||||
|
)
|
||||||
|
for noisy in ("uvicorn.access", "httpx", "httpcore"):
|
||||||
|
logging.getLogger(noisy).setLevel(logging.WARNING)
|
||||||
|
|
||||||
|
renderer: Any = (
|
||||||
|
structlog.processors.JSONRenderer(ensure_ascii=False)
|
||||||
|
if json_output
|
||||||
|
else structlog.dev.ConsoleRenderer(colors=False)
|
||||||
|
)
|
||||||
|
|
||||||
|
structlog.configure(
|
||||||
|
processors=[
|
||||||
|
structlog.contextvars.merge_contextvars,
|
||||||
|
structlog.processors.add_log_level,
|
||||||
|
structlog.processors.TimeStamper(fmt="iso", utc=True),
|
||||||
|
_context_processor,
|
||||||
|
make_redaction_processor(extra_redact_keys),
|
||||||
|
structlog.processors.StackInfoRenderer(),
|
||||||
|
structlog.processors.format_exc_info,
|
||||||
|
renderer,
|
||||||
|
],
|
||||||
|
wrapper_class=structlog.make_filtering_bound_logger(
|
||||||
|
getattr(logging, level.upper(), logging.INFO)
|
||||||
|
),
|
||||||
|
logger_factory=structlog.PrintLoggerFactory(file=sys.stdout),
|
||||||
|
cache_logger_on_first_use=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def get_logger(name: str | None = None) -> Any:
|
||||||
|
return structlog.get_logger(name)
|
||||||
Executable
+74
@@ -0,0 +1,74 @@
|
|||||||
|
"""ASGI middleware: request id, access log, metrics, /v1 path alias."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import time
|
||||||
|
from collections.abc import Awaitable, Callable
|
||||||
|
|
||||||
|
from fastapi import FastAPI, Request, Response
|
||||||
|
from starlette.middleware.base import BaseHTTPMiddleware
|
||||||
|
|
||||||
|
from packages.shared.context import bind_context, new_request_id, reset_context
|
||||||
|
from packages.shared.logging import get_logger
|
||||||
|
from packages.shared.observability import HTTP_LATENCY, HTTP_REQUESTS, current_trace_id
|
||||||
|
|
||||||
|
REQUEST_ID_HEADER = "X-Request-Id"
|
||||||
|
|
||||||
|
logger = get_logger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
class RequestContextMiddleware(BaseHTTPMiddleware):
|
||||||
|
"""Bind request/trace ids, emit the access log, and record HTTP metrics."""
|
||||||
|
|
||||||
|
async def dispatch(
|
||||||
|
self, request: Request, call_next: Callable[[Request], Awaitable[Response]]
|
||||||
|
) -> Response:
|
||||||
|
request_id = request.headers.get(REQUEST_ID_HEADER) or new_request_id()
|
||||||
|
tokens = bind_context(request_id=request_id, trace_id=current_trace_id())
|
||||||
|
started = time.perf_counter()
|
||||||
|
status_code = 500
|
||||||
|
try:
|
||||||
|
response = await call_next(request)
|
||||||
|
status_code = response.status_code
|
||||||
|
response.headers[REQUEST_ID_HEADER] = request_id
|
||||||
|
return response
|
||||||
|
finally:
|
||||||
|
elapsed = time.perf_counter() - started
|
||||||
|
route = _route_template(request)
|
||||||
|
HTTP_REQUESTS.labels(request.method, route, str(status_code)).inc()
|
||||||
|
HTTP_LATENCY.labels(request.method, route).observe(elapsed)
|
||||||
|
logger.info(
|
||||||
|
"http_request",
|
||||||
|
method=request.method,
|
||||||
|
route=route,
|
||||||
|
status=status_code,
|
||||||
|
duration_ms=round(elapsed * 1000, 2),
|
||||||
|
)
|
||||||
|
reset_context(tokens)
|
||||||
|
|
||||||
|
|
||||||
|
def _route_template(request: Request) -> str:
|
||||||
|
route = request.scope.get("route")
|
||||||
|
path = getattr(route, "path", None)
|
||||||
|
return str(path) if path else request.url.path
|
||||||
|
|
||||||
|
|
||||||
|
def install_version_alias(app: FastAPI, api_prefix: str) -> None:
|
||||||
|
"""Accept ``/v1/...`` as an alias of ``/api/v1/...``.
|
||||||
|
|
||||||
|
架构文档 §9.1 fixes the base path at ``/api/v1``; several internal specs and
|
||||||
|
smoke scripts refer to ``/v1/...``. Rewriting at the ASGI scope keeps a
|
||||||
|
single set of route definitions and one OpenAPI document.
|
||||||
|
"""
|
||||||
|
legacy_prefix = "/v1"
|
||||||
|
if not api_prefix.endswith(legacy_prefix):
|
||||||
|
return
|
||||||
|
|
||||||
|
@app.middleware("http")
|
||||||
|
async def _rewrite(
|
||||||
|
request: Request, call_next: Callable[[Request], Awaitable[Response]]
|
||||||
|
) -> Response:
|
||||||
|
path = request.scope.get("path", "")
|
||||||
|
if path == legacy_prefix or path.startswith(legacy_prefix + "/"):
|
||||||
|
request.scope["path"] = api_prefix + path[len(legacy_prefix) :]
|
||||||
|
return await call_next(request)
|
||||||
Executable
+249
@@ -0,0 +1,249 @@
|
|||||||
|
"""SQLAlchemy 2.0 ORM models.
|
||||||
|
|
||||||
|
Scope of this scaffold: only the tables needed to prove the vertical slice
|
||||||
|
(架构文档 §6.2 identity / billing / task / 观测与成本). All remaining domains
|
||||||
|
(pf_ / rp_ / kb_ / ag_ / tr_ / cp_) are intentionally absent — see
|
||||||
|
packages/*/README.md for what still has to be built.
|
||||||
|
|
||||||
|
Conventions enforced here (架构文档 §6.1):
|
||||||
|
1. 表名带域前缀 (`id_` / `bl_` / `tk_` / `ob_`),为将来拆库预留边界。
|
||||||
|
2. 主键统一雪花 ID (bigint),不用自增、不用 UUID。
|
||||||
|
3. 涉及配额的表必须有幂等键与流水。
|
||||||
|
4. JSONB 只用于「结构会演进且不需要索引全部字段」的场景。
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import datetime
|
||||||
|
from decimal import Decimal
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from sqlalchemy import (
|
||||||
|
BigInteger,
|
||||||
|
Boolean,
|
||||||
|
DateTime,
|
||||||
|
Index,
|
||||||
|
Integer,
|
||||||
|
Numeric,
|
||||||
|
PrimaryKeyConstraint,
|
||||||
|
SmallInteger,
|
||||||
|
String,
|
||||||
|
Text,
|
||||||
|
UniqueConstraint,
|
||||||
|
func,
|
||||||
|
)
|
||||||
|
from sqlalchemy.dialects.postgresql import JSONB
|
||||||
|
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column
|
||||||
|
|
||||||
|
|
||||||
|
class Base(DeclarativeBase):
|
||||||
|
"""Declarative base; ``Base.metadata`` is Alembic's ``target_metadata``."""
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# identity 域
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
class User(Base):
|
||||||
|
__tablename__ = "id_user"
|
||||||
|
|
||||||
|
user_id: Mapped[int] = mapped_column(BigInteger, primary_key=True, autoincrement=False)
|
||||||
|
status: Mapped[str] = mapped_column(String(16), nullable=False, default="active")
|
||||||
|
nickname: Mapped[str | None] = mapped_column(String(64))
|
||||||
|
avatar_key: Mapped[str | None] = mapped_column(String(255))
|
||||||
|
register_source: Mapped[str] = mapped_column(String(32), nullable=False, default="unknown")
|
||||||
|
created_at: Mapped[datetime] = mapped_column(
|
||||||
|
DateTime(timezone=True), nullable=False, server_default=func.now()
|
||||||
|
)
|
||||||
|
last_active_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||||
|
deleted_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||||
|
|
||||||
|
__table_args__ = (Index("ix_id_user_status", "status"),)
|
||||||
|
|
||||||
|
|
||||||
|
class UserIdentity(Base):
|
||||||
|
"""External identity mapping. UNIQUE(provider, union_id) per 架构文档 §6.2."""
|
||||||
|
|
||||||
|
__tablename__ = "id_user_identity"
|
||||||
|
|
||||||
|
identity_id: Mapped[int] = mapped_column(BigInteger, primary_key=True, autoincrement=False)
|
||||||
|
user_id: Mapped[int] = mapped_column(BigInteger, nullable=False)
|
||||||
|
provider: Mapped[str] = mapped_column(String(32), nullable=False)
|
||||||
|
union_id: Mapped[str] = mapped_column(String(128), nullable=False)
|
||||||
|
open_id: Mapped[str | None] = mapped_column(String(128))
|
||||||
|
created_at: Mapped[datetime] = mapped_column(
|
||||||
|
DateTime(timezone=True), nullable=False, server_default=func.now()
|
||||||
|
)
|
||||||
|
|
||||||
|
__table_args__ = (
|
||||||
|
UniqueConstraint("provider", "union_id", name="uq_id_user_identity_provider_union"),
|
||||||
|
Index("ix_id_user_identity_user", "user_id"),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class Consent(Base):
|
||||||
|
"""CR-01 留证:人脸的单独同意必须独立成行,不得混入总协议。"""
|
||||||
|
|
||||||
|
__tablename__ = "id_consent"
|
||||||
|
|
||||||
|
consent_id: Mapped[int] = mapped_column(BigInteger, primary_key=True, autoincrement=False)
|
||||||
|
user_id: Mapped[int] = mapped_column(BigInteger, nullable=False)
|
||||||
|
consent_type: Mapped[str] = mapped_column(String(32), nullable=False)
|
||||||
|
doc_version: Mapped[str] = mapped_column(String(32), nullable=False)
|
||||||
|
granted_at: Mapped[datetime] = mapped_column(
|
||||||
|
DateTime(timezone=True), nullable=False, server_default=func.now()
|
||||||
|
)
|
||||||
|
revoked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||||
|
client_ip: Mapped[str | None] = mapped_column(String(64))
|
||||||
|
ua: Mapped[str | None] = mapped_column(String(512))
|
||||||
|
|
||||||
|
__table_args__ = (Index("ix_id_consent_user_type", "user_id", "consent_type"),)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# billing 域(V1:配额与计量,无支付)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
class QuotaRule(Base):
|
||||||
|
"""FR-BILL-04 配额规则配置化:改配置不发版。"""
|
||||||
|
|
||||||
|
__tablename__ = "bl_quota_rule"
|
||||||
|
|
||||||
|
quota_key: Mapped[str] = mapped_column(String(64), primary_key=True)
|
||||||
|
period_type: Mapped[str] = mapped_column(String(16), nullable=False) # day/month/lifetime
|
||||||
|
base_limit: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||||
|
task_bonus_max: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
||||||
|
hard_cap: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||||
|
description: Mapped[str | None] = mapped_column(String(255))
|
||||||
|
updated_at: Mapped[datetime] = mapped_column(
|
||||||
|
DateTime(timezone=True), nullable=False, server_default=func.now()
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class Quota(Base):
|
||||||
|
"""周期配额计量:按 ``period_key`` 自然重置,无需定时任务清零。"""
|
||||||
|
|
||||||
|
__tablename__ = "bl_quota"
|
||||||
|
|
||||||
|
user_id: Mapped[int] = mapped_column(BigInteger, nullable=False)
|
||||||
|
quota_key: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||||
|
period_key: Mapped[str] = mapped_column(String(32), nullable=False)
|
||||||
|
used: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
||||||
|
limit_value: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||||
|
bonus_value: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
||||||
|
updated_at: Mapped[datetime] = mapped_column(
|
||||||
|
DateTime(timezone=True), nullable=False, server_default=func.now()
|
||||||
|
)
|
||||||
|
|
||||||
|
__table_args__ = (
|
||||||
|
PrimaryKeyConstraint("user_id", "quota_key", "period_key", name="pk_bl_quota"),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class Ledger(Base):
|
||||||
|
"""流水是真相,余额是缓存 (架构文档 §6.1 原则 4 / §6.2 bl_ledger).
|
||||||
|
|
||||||
|
``tx_type`` 在 V1 只用 system_grant / consume / refund;V2 增加
|
||||||
|
purchase / gift —— 这是 V1→V2 免数据迁移的关键,不要拆表。
|
||||||
|
``status`` 承载 §7.4 的预扣三段式:reserved → committed / refunded。
|
||||||
|
"""
|
||||||
|
|
||||||
|
__tablename__ = "bl_ledger"
|
||||||
|
|
||||||
|
tx_id: Mapped[int] = mapped_column(BigInteger, primary_key=True, autoincrement=False)
|
||||||
|
user_id: Mapped[int] = mapped_column(BigInteger, nullable=False)
|
||||||
|
delta: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||||
|
tx_type: Mapped[str] = mapped_column(String(24), nullable=False)
|
||||||
|
status: Mapped[str] = mapped_column(String(16), nullable=False, default="committed")
|
||||||
|
reason: Mapped[str | None] = mapped_column(String(128))
|
||||||
|
ref_type: Mapped[str | None] = mapped_column(String(32))
|
||||||
|
ref_id: Mapped[str | None] = mapped_column(String(64))
|
||||||
|
quota_key: Mapped[str | None] = mapped_column(String(64))
|
||||||
|
period_key: Mapped[str | None] = mapped_column(String(32))
|
||||||
|
balance_after: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
||||||
|
idempotency_key: Mapped[str] = mapped_column(String(128), nullable=False)
|
||||||
|
created_at: Mapped[datetime] = mapped_column(
|
||||||
|
DateTime(timezone=True), nullable=False, server_default=func.now()
|
||||||
|
)
|
||||||
|
|
||||||
|
__table_args__ = (
|
||||||
|
UniqueConstraint("idempotency_key", name="uq_bl_ledger_idempotency_key"),
|
||||||
|
Index("ix_bl_ledger_user_created", "user_id", "created_at"),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# task 域(统一任务中心)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
class Task(Base):
|
||||||
|
"""全系统唯一的长耗时任务视图,前端任务中心直接读它 (架构文档 §6.2)。"""
|
||||||
|
|
||||||
|
__tablename__ = "tk_task"
|
||||||
|
|
||||||
|
task_id: Mapped[int] = mapped_column(BigInteger, primary_key=True, autoincrement=False)
|
||||||
|
user_id: Mapped[int] = mapped_column(BigInteger, nullable=False)
|
||||||
|
task_type: Mapped[str] = mapped_column(String(32), nullable=False)
|
||||||
|
status: Mapped[str] = mapped_column(String(16), nullable=False)
|
||||||
|
progress: Mapped[int] = mapped_column(SmallInteger, nullable=False, default=0)
|
||||||
|
stage_text: Mapped[str | None] = mapped_column(String(128))
|
||||||
|
idempotency_key: Mapped[str] = mapped_column(String(128), nullable=False)
|
||||||
|
payload: Mapped[dict[str, Any]] = mapped_column(JSONB, nullable=False, default=dict)
|
||||||
|
result: Mapped[dict[str, Any] | None] = mapped_column(JSONB)
|
||||||
|
error_code: Mapped[int | None] = mapped_column(Integer)
|
||||||
|
error_message: Mapped[str | None] = mapped_column(String(512))
|
||||||
|
retry_count: Mapped[int] = mapped_column(SmallInteger, nullable=False, default=0)
|
||||||
|
points_reserved: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
||||||
|
reserve_tx_id: Mapped[int | None] = mapped_column(BigInteger)
|
||||||
|
submitted_at: Mapped[datetime] = mapped_column(
|
||||||
|
DateTime(timezone=True), nullable=False, server_default=func.now()
|
||||||
|
)
|
||||||
|
started_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||||
|
finished_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||||
|
eta_seconds: Mapped[int | None] = mapped_column(Integer)
|
||||||
|
|
||||||
|
__table_args__ = (
|
||||||
|
UniqueConstraint("idempotency_key", name="uq_tk_task_idempotency_key"),
|
||||||
|
Index("ix_tk_task_user_status", "user_id", "status"),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# 观测与成本
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
class AiCall(Base):
|
||||||
|
"""成本归因的唯一数据源 (架构文档 §13, FR-BILL-08).
|
||||||
|
|
||||||
|
**不允许「先上线后补埋点」** —— V1 不收费,成本是唯一财务变量,且这张表
|
||||||
|
是 V2 定价的唯一数据来源。每一次 provider 调用(含失败与结构化重试)都写
|
||||||
|
一行。生产环境按月分区。
|
||||||
|
"""
|
||||||
|
|
||||||
|
__tablename__ = "ob_ai_call"
|
||||||
|
|
||||||
|
call_id: Mapped[int] = mapped_column(BigInteger, primary_key=True, autoincrement=False)
|
||||||
|
user_id: Mapped[int | None] = mapped_column(BigInteger)
|
||||||
|
scene: Mapped[str] = mapped_column(String(48), nullable=False)
|
||||||
|
provider: Mapped[str] = mapped_column(String(32), nullable=False)
|
||||||
|
model: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||||
|
# Prompt 版本:架构文档 §8.1「可复现」要求记录,§6.2 的列清单漏了,这里补上。
|
||||||
|
prompt_id: Mapped[str | None] = mapped_column(String(64))
|
||||||
|
prompt_version: Mapped[str | None] = mapped_column(String(16))
|
||||||
|
tokens_in: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
||||||
|
tokens_out: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
||||||
|
tokens_cached: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
||||||
|
latency_ms: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
||||||
|
cost_cents: Mapped[Decimal] = mapped_column(
|
||||||
|
Numeric(14, 6), nullable=False, default=Decimal("0")
|
||||||
|
)
|
||||||
|
success: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
|
||||||
|
error_code: Mapped[str | None] = mapped_column(String(64))
|
||||||
|
error_message: Mapped[str | None] = mapped_column(Text)
|
||||||
|
ref_type: Mapped[str | None] = mapped_column(String(32))
|
||||||
|
ref_id: Mapped[str | None] = mapped_column(String(64))
|
||||||
|
created_at: Mapped[datetime] = mapped_column(
|
||||||
|
DateTime(timezone=True), nullable=False, server_default=func.now()
|
||||||
|
)
|
||||||
|
|
||||||
|
__table_args__ = (
|
||||||
|
Index("ix_ob_ai_call_created", "created_at"),
|
||||||
|
Index("ix_ob_ai_call_scene_created", "scene", "created_at"),
|
||||||
|
Index("ix_ob_ai_call_user_created", "user_id", "created_at"),
|
||||||
|
)
|
||||||
Executable
+116
@@ -0,0 +1,116 @@
|
|||||||
|
"""OpenTelemetry tracing + Prometheus metrics (架构文档 §15, TD-24).
|
||||||
|
|
||||||
|
The three dashboards required by §15 (业务漏斗 / AI 成本 / 任务健康) are built
|
||||||
|
from the metrics declared here plus the ``ob_ai_call`` table.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import TYPE_CHECKING, Any
|
||||||
|
|
||||||
|
from prometheus_client import CollectorRegistry, Counter, Histogram
|
||||||
|
|
||||||
|
if TYPE_CHECKING: # pragma: no cover
|
||||||
|
from fastapi import FastAPI
|
||||||
|
|
||||||
|
from packages.shared.config import Settings
|
||||||
|
|
||||||
|
REGISTRY = CollectorRegistry(auto_describe=True)
|
||||||
|
|
||||||
|
HTTP_REQUESTS = Counter(
|
||||||
|
"meida_http_requests_total",
|
||||||
|
"HTTP requests handled by the API",
|
||||||
|
labelnames=("method", "route", "status"),
|
||||||
|
registry=REGISTRY,
|
||||||
|
)
|
||||||
|
HTTP_LATENCY = Histogram(
|
||||||
|
"meida_http_request_duration_seconds",
|
||||||
|
"HTTP request latency",
|
||||||
|
labelnames=("method", "route"),
|
||||||
|
buckets=(0.01, 0.05, 0.1, 0.2, 0.3, 0.5, 1.0, 2.0, 5.0, 8.0, 15.0),
|
||||||
|
registry=REGISTRY,
|
||||||
|
)
|
||||||
|
|
||||||
|
# --- AI 成本看板 ---------------------------------------------------------
|
||||||
|
LLM_CALLS = Counter(
|
||||||
|
"meida_llm_calls_total",
|
||||||
|
"LLM provider calls (one per HTTP attempt, including retries and failures)",
|
||||||
|
labelnames=("scene", "provider", "model", "success"),
|
||||||
|
registry=REGISTRY,
|
||||||
|
)
|
||||||
|
LLM_TOKENS = Counter(
|
||||||
|
"meida_llm_tokens_total",
|
||||||
|
"LLM tokens by kind; `cached` drives the prompt-cache hit-rate panel (NFR-COST-03)",
|
||||||
|
labelnames=("scene", "provider", "model", "kind"),
|
||||||
|
registry=REGISTRY,
|
||||||
|
)
|
||||||
|
LLM_COST_CENTS = Counter(
|
||||||
|
"meida_llm_cost_cents_total",
|
||||||
|
"Accumulated LLM cost in cents, attributable to scene/provider/model",
|
||||||
|
labelnames=("scene", "provider", "model"),
|
||||||
|
registry=REGISTRY,
|
||||||
|
)
|
||||||
|
LLM_LATENCY = Histogram(
|
||||||
|
"meida_llm_latency_seconds",
|
||||||
|
"LLM provider call latency",
|
||||||
|
labelnames=("scene", "provider", "model"),
|
||||||
|
buckets=(0.2, 0.5, 1.0, 2.0, 4.0, 8.0, 15.0, 30.0, 60.0),
|
||||||
|
registry=REGISTRY,
|
||||||
|
)
|
||||||
|
|
||||||
|
# --- 业务漏斗 / 任务健康 -------------------------------------------------
|
||||||
|
QUOTA_EVENTS = Counter(
|
||||||
|
"meida_quota_events_total",
|
||||||
|
"Quota three-phase events (reserve/commit/refund) and their outcome",
|
||||||
|
labelnames=("quota_key", "phase", "result"),
|
||||||
|
registry=REGISTRY,
|
||||||
|
)
|
||||||
|
TASK_TRANSITIONS = Counter(
|
||||||
|
"meida_task_transitions_total",
|
||||||
|
"tk_task state machine transitions",
|
||||||
|
labelnames=("task_type", "status"),
|
||||||
|
registry=REGISTRY,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def setup_tracing(app: FastAPI, settings: Settings) -> None:
|
||||||
|
"""Install the OTel tracer provider and instrument FastAPI.
|
||||||
|
|
||||||
|
Disabled by default locally; enable with ``OTEL_ENABLED=true`` and an
|
||||||
|
OTLP endpoint.
|
||||||
|
"""
|
||||||
|
if not settings.otel_enabled:
|
||||||
|
return
|
||||||
|
|
||||||
|
from opentelemetry import trace
|
||||||
|
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
|
||||||
|
from opentelemetry.instrumentation.fastapi import FastAPIInstrumentor
|
||||||
|
from opentelemetry.sdk.resources import Resource
|
||||||
|
from opentelemetry.sdk.trace import TracerProvider
|
||||||
|
from opentelemetry.sdk.trace.export import BatchSpanProcessor
|
||||||
|
|
||||||
|
resource = Resource.create(
|
||||||
|
{"service.name": settings.otel_service_name, "deployment.environment": settings.app_env}
|
||||||
|
)
|
||||||
|
provider = TracerProvider(resource=resource)
|
||||||
|
if settings.otel_exporter_otlp_endpoint:
|
||||||
|
provider.add_span_processor(
|
||||||
|
BatchSpanProcessor(
|
||||||
|
OTLPSpanExporter(endpoint=f"{settings.otel_exporter_otlp_endpoint}/v1/traces")
|
||||||
|
)
|
||||||
|
)
|
||||||
|
trace.set_tracer_provider(provider)
|
||||||
|
FastAPIInstrumentor.instrument_app(app, tracer_provider=provider)
|
||||||
|
|
||||||
|
|
||||||
|
def current_trace_id() -> str | None:
|
||||||
|
"""Return the active W3C trace id, if any."""
|
||||||
|
try:
|
||||||
|
from opentelemetry import trace
|
||||||
|
except ImportError: # pragma: no cover
|
||||||
|
return None
|
||||||
|
span = trace.get_current_span()
|
||||||
|
ctx: Any = span.get_span_context()
|
||||||
|
if not getattr(ctx, "is_valid", False):
|
||||||
|
return None
|
||||||
|
return format(ctx.trace_id, "032x")
|
||||||
Executable
+28
@@ -0,0 +1,28 @@
|
|||||||
|
"""Shared Redis connection (cache / quota counters / cost budget counter)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from redis.asyncio import Redis, from_url
|
||||||
|
|
||||||
|
from packages.shared.config import Settings, get_settings
|
||||||
|
|
||||||
|
_client: Redis | None = None
|
||||||
|
|
||||||
|
|
||||||
|
def create_redis(settings: Settings | None = None) -> Redis:
|
||||||
|
cfg = settings or get_settings()
|
||||||
|
return from_url(cfg.redis_url, encoding="utf-8", decode_responses=True)
|
||||||
|
|
||||||
|
|
||||||
|
def get_redis() -> Redis:
|
||||||
|
global _client
|
||||||
|
if _client is None:
|
||||||
|
_client = create_redis()
|
||||||
|
return _client
|
||||||
|
|
||||||
|
|
||||||
|
async def close_redis() -> None:
|
||||||
|
global _client
|
||||||
|
if _client is not None:
|
||||||
|
await _client.aclose()
|
||||||
|
_client = None
|
||||||
Executable
+49
@@ -0,0 +1,49 @@
|
|||||||
|
"""Unified response envelope (架构文档 §9.2).
|
||||||
|
|
||||||
|
{"code": 0, "message": "ok", "data": {...}, "request_id": "..."}
|
||||||
|
{"code": 40301, "message": "今日额度已用完", "data": {...},
|
||||||
|
"request_id": "...", "hint": "明天恢复,或完成任务多换 3 次"}
|
||||||
|
|
||||||
|
Every endpoint returns this shape. Handlers in :mod:`packages.shared.handlers`
|
||||||
|
guarantee it even for unhandled exceptions.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from pydantic import BaseModel, Field
|
||||||
|
|
||||||
|
from packages.shared.context import get_request_id
|
||||||
|
from packages.shared.errors import ERROR_META, ErrorCode
|
||||||
|
|
||||||
|
|
||||||
|
class Envelope(BaseModel):
|
||||||
|
code: int = Field(default=int(ErrorCode.OK), description="0 表示成功;4/5/6 开头为错误段")
|
||||||
|
message: str = Field(default="ok", description="用户可读文案")
|
||||||
|
data: Any = Field(default=None)
|
||||||
|
request_id: str | None = Field(default=None)
|
||||||
|
hint: str | None = Field(default=None, description="告诉用户下一步怎么办")
|
||||||
|
|
||||||
|
|
||||||
|
def ok(data: Any = None) -> dict[str, Any]:
|
||||||
|
return Envelope(
|
||||||
|
code=int(ErrorCode.OK), message="ok", data=data, request_id=get_request_id()
|
||||||
|
).model_dump(exclude_none=True)
|
||||||
|
|
||||||
|
|
||||||
|
def fail(
|
||||||
|
code: ErrorCode,
|
||||||
|
*,
|
||||||
|
message: str | None = None,
|
||||||
|
hint: str | None = None,
|
||||||
|
data: Any = None,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
meta = ERROR_META.get(code, ERROR_META[ErrorCode.INTERNAL_ERROR])
|
||||||
|
return Envelope(
|
||||||
|
code=int(code),
|
||||||
|
message=message or meta.message,
|
||||||
|
data=data,
|
||||||
|
request_id=get_request_id(),
|
||||||
|
hint=hint if hint is not None else meta.hint,
|
||||||
|
).model_dump(exclude_none=True)
|
||||||
Executable
+76
@@ -0,0 +1,76 @@
|
|||||||
|
"""JWT access tokens (架构文档 §9.1: Bearer JWT, 15 分钟).
|
||||||
|
|
||||||
|
The revocable refresh-token half (`id_session`) is FR-ACC-05 and is deliberately
|
||||||
|
NOT implemented in this scaffold — see packages/identity/README.md.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import time
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import jwt
|
||||||
|
|
||||||
|
from packages.shared.errors import AppError, ErrorCode
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class AccessTokenClaims:
|
||||||
|
user_id: int
|
||||||
|
issued_at: int
|
||||||
|
expires_at: int
|
||||||
|
issuer: str
|
||||||
|
|
||||||
|
|
||||||
|
def create_access_token(
|
||||||
|
*,
|
||||||
|
user_id: int,
|
||||||
|
secret: str,
|
||||||
|
algorithm: str = "HS256",
|
||||||
|
ttl_seconds: int = 900,
|
||||||
|
issuer: str = "meida",
|
||||||
|
now: int | None = None,
|
||||||
|
) -> str:
|
||||||
|
issued_at = int(now if now is not None else time.time())
|
||||||
|
payload: dict[str, Any] = {
|
||||||
|
"sub": str(user_id),
|
||||||
|
"iat": issued_at,
|
||||||
|
"exp": issued_at + ttl_seconds,
|
||||||
|
"iss": issuer,
|
||||||
|
"typ": "access",
|
||||||
|
}
|
||||||
|
return jwt.encode(payload, secret, algorithm=algorithm)
|
||||||
|
|
||||||
|
|
||||||
|
def decode_access_token(
|
||||||
|
token: str,
|
||||||
|
*,
|
||||||
|
secret: str,
|
||||||
|
algorithm: str = "HS256",
|
||||||
|
issuer: str = "meida",
|
||||||
|
) -> AccessTokenClaims:
|
||||||
|
try:
|
||||||
|
payload = jwt.decode(
|
||||||
|
token,
|
||||||
|
secret,
|
||||||
|
algorithms=[algorithm],
|
||||||
|
issuer=issuer,
|
||||||
|
options={"require": ["exp", "iat", "sub"]},
|
||||||
|
)
|
||||||
|
except jwt.ExpiredSignatureError as exc:
|
||||||
|
raise AppError(ErrorCode.TOKEN_EXPIRED) from exc
|
||||||
|
except jwt.InvalidTokenError as exc:
|
||||||
|
raise AppError(ErrorCode.TOKEN_INVALID) from exc
|
||||||
|
|
||||||
|
try:
|
||||||
|
user_id = int(payload["sub"])
|
||||||
|
except (KeyError, TypeError, ValueError) as exc:
|
||||||
|
raise AppError(ErrorCode.TOKEN_INVALID) from exc
|
||||||
|
|
||||||
|
return AccessTokenClaims(
|
||||||
|
user_id=user_id,
|
||||||
|
issued_at=int(payload["iat"]),
|
||||||
|
expires_at=int(payload["exp"]),
|
||||||
|
issuer=str(payload.get("iss", issuer)),
|
||||||
|
)
|
||||||
Reference in New Issue
Block a user