build: local stack, container images and the end-to-end smoke script

docker-compose brings up PostgreSQL 16 with pgvector and Redis 7 on
non-standard host ports so it never collides with an existing local database.
scripts/smoke.sh is the proof: compose, migrate, boot the API, issue a token,
call the real DeepSeek API and print the resulting ob_ai_call and bl_ledger
rows. services/vision and services/matting are placeholders noting that the
local RTX 5090 makes BiRefNet and RTMPose developable without cloud GPU.
This commit is contained in:
xsl
2026-08-14 00:26:39 +08:00
parent 93b88db2f5
commit 8a4720472b
14 changed files with 481 additions and 0 deletions
+55
View File
@@ -0,0 +1,55 @@
# syntax=docker/dockerfile:1
# API image (架构文档 §14.1 CVM-app / api). Multi-stage so the runtime layer
# carries no build toolchain and no dev dependencies.
FROM python:3.12-slim AS builder
COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
ENV UV_COMPILE_BYTECODE=1 \
UV_LINK_MODE=copy \
UV_PYTHON_DOWNLOADS=never
WORKDIR /app
COPY pyproject.toml uv.lock* README.md ./
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --frozen --no-install-project --no-dev || uv sync --no-install-project --no-dev
COPY apps ./apps
COPY packages ./packages
COPY services ./services
COPY prompts ./prompts
COPY migrations ./migrations
COPY alembic.ini ./
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --no-dev --frozen || uv sync --no-dev
FROM python:3.12-slim AS runtime
ENV PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1 \
PATH="/app/.venv/bin:$PATH" \
TZ=Asia/Shanghai
RUN groupadd --system meida && useradd --system --gid meida --home /app meida
WORKDIR /app
COPY --from=builder --chown=meida:meida /app /app
USER meida
EXPOSE 8000
HEALTHCHECK --interval=15s --timeout=5s --start-period=20s --retries=3 \
CMD python -c "import urllib.request;urllib.request.urlopen('http://localhost:8000/healthz')"
# Gunicorn supervises uvicorn workers (架构文档 §5: FastAPI + Uvicorn/Gunicorn).
CMD ["gunicorn", "apps.api.main:app", \
"--worker-class", "uvicorn.workers.UvicornWorker", \
"--workers", "4", \
"--bind", "0.0.0.0:8000", \
"--timeout", "120", \
"--graceful-timeout", "30", \
"--access-logfile", "-"]
+50
View File
@@ -0,0 +1,50 @@
# syntax=docker/dockerfile:1
# Celery worker image. Same build as the API; different entrypoint and queue set.
# 架构文档 §14.1 runs worker-cpu / worker-io on CVM-app and worker-gpu on CVM-gpu,
# so the queue list is injected rather than baked in.
FROM python:3.12-slim AS builder
COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
ENV UV_COMPILE_BYTECODE=1 \
UV_LINK_MODE=copy \
UV_PYTHON_DOWNLOADS=never
WORKDIR /app
COPY pyproject.toml uv.lock* README.md ./
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --frozen --no-install-project --no-dev || uv sync --no-install-project --no-dev
COPY apps ./apps
COPY packages ./packages
COPY services ./services
COPY prompts ./prompts
COPY migrations ./migrations
COPY alembic.ini ./
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --no-dev --frozen || uv sync --no-dev
FROM python:3.12-slim AS runtime
ENV PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1 \
PATH="/app/.venv/bin:$PATH" \
TZ=Asia/Shanghai \
CELERY_QUEUES=cpu,io \
CELERY_CONCURRENCY=4
RUN groupadd --system meida && useradd --system --gid meida --home /app meida
WORKDIR /app
COPY --from=builder --chown=meida:meida /app /app
USER meida
HEALTHCHECK --interval=30s --timeout=10s --start-period=30s --retries=3 \
CMD celery -A apps.worker.celery_app.celery_app inspect ping -d "celery@$(hostname)" || exit 1
CMD ["sh", "-c", "celery -A apps.worker.celery_app.celery_app worker -Q ${CELERY_QUEUES} -c ${CELERY_CONCURRENCY} -l info"]
+41
View File
@@ -0,0 +1,41 @@
# `deploy/`
| 文件 | 用途 |
| --- | --- |
| `docker-compose.dev.yml` | 本地开发栈:PostgreSQL 16 + pgvector、Redis 7(均带 healthcheck) |
| `Dockerfile.api` | API 镜像(gunicorn + uvicorn worker) |
| `Dockerfile.worker` | Celery worker 镜像(队列由 `CELERY_QUEUES` 注入) |
```bash
make up # 只起 postgres + redis
make down
docker compose -f deploy/docker-compose.dev.yml --profile app up -d # 连 api/worker 一起
```
本地端口刻意用 **55432 / 56379**,避免和开发机上已有的 PG/Redis 撞车。
## V1 生产形态(架构 §14.1)
单可用区 + Docker Compose,不上 K8s(TD-03、团队规模):
```
CLB → CVM-app: nginx / api / admin / worker-cpu / worker-io / beat / vision-svc
CVM-gpu(按需启停): tryon-svc / worker-gpu / matting
云数据库 PostgreSQL(pgvector) · 云数据库 Redis · COS×4 + CDN · CLS
```
**GPU 实例按需启停是 V1 最有效的成本手段**:队列有积压时拉起,空闲一段时间后
释放(NFR-COST-04)。
## 规矩
- **镜像里不许有密钥**(架构 §11.2)。配置一律从环境变量注入,生产走 SSM/KMS。
`env_file: ../.env` 只在本地 compose 用。
- 数据库迁移是发布流程里的**独立步骤**,先迁移后发布(§14.3)。
- 镜像打标签化版本,不要用 `latest` 发布。
- 容器以非 root 用户运行,两个 Dockerfile 都已如此。
## 还没有的东西
nginx 配置、TLS 终止、CLB/CDN 配置、COS 四桶的生命周期策略(临时桶 ≤ 2 小时,
CR-02)、Grafana 三张看板、告警规则、备份与恢复演练脚本(FR-SYS-09)。
+89
View File
@@ -0,0 +1,89 @@
# Local development stack (T-P1-03: 新成员一条命令起环境).
# Host ports are deliberately non-standard so this never collides with a
# PostgreSQL/Redis already running on the developer's machine.
name: meida-dev
services:
postgres:
# pgvector is required from day one (架构文档 §5, TD-08) — proving the
# extension exists here is what stops "works on my laptop" later.
image: pgvector/pgvector:pg16
container_name: meida-postgres
environment:
POSTGRES_USER: meida
POSTGRES_PASSWORD: meida
POSTGRES_DB: meida
TZ: Asia/Shanghai
ports:
- "55432:5432"
volumes:
- pgdata:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U meida -d meida"]
interval: 5s
timeout: 5s
retries: 20
start_period: 10s
redis:
image: redis:7-alpine
container_name: meida-redis
command: ["redis-server", "--appendonly", "yes"]
ports:
- "56379:6379"
volumes:
- redisdata:/data
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 5s
timeout: 3s
retries: 20
# --- optional: run the app itself in containers (`--profile app`) ---------
api:
profiles: ["app"]
build:
context: ..
dockerfile: deploy/Dockerfile.api
container_name: meida-api
env_file: ../.env
environment:
DATABASE_URL: postgresql+asyncpg://meida:meida@postgres:5432/meida
REDIS_URL: redis://redis:6379/0
CELERY_BROKER_URL: redis://redis:6379/1
CELERY_RESULT_BACKEND: redis://redis:6379/2
ports:
- "8000:8000"
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_healthy
healthcheck:
test: ["CMD", "python", "-c", "import urllib.request;urllib.request.urlopen('http://localhost:8000/healthz')"]
interval: 10s
timeout: 5s
retries: 10
worker-cpu:
profiles: ["app"]
build:
context: ..
dockerfile: deploy/Dockerfile.worker
container_name: meida-worker-cpu
env_file: ../.env
environment:
DATABASE_URL: postgresql+asyncpg://meida:meida@postgres:5432/meida
REDIS_URL: redis://redis:6379/0
CELERY_BROKER_URL: redis://redis:6379/1
CELERY_RESULT_BACKEND: redis://redis:6379/2
CELERY_QUEUES: cpu,io
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_healthy
volumes:
pgdata:
redisdata:
+22
View File
@@ -0,0 +1,22 @@
# `scripts/`
| 脚本 | 用途 |
| --- | --- |
| `smoke.sh` | 端到端验证:compose up → migrate → 起 API → 发 dev token → 调 **真实 DeepSeek** → 打印 `ob_ai_call` 与 `bl_ledger` |
```bash
make smoke # 或 bash scripts/smoke.sh
```
## 注意
- **`smoke.sh` 会真的调用 DeepSeek,产生真实费用。** 这是仓库里唯一会花钱的
入口;`make test` 全程离线。
- 需要 `.env` 里有可用的 `DEEPSEEK_API_KEY`。
- 它在 8010 端口起一个临时 API 进程,结束时自动清理;日志留在 `.smoke/api.log`。
## 写新脚本的规矩
- `set -Eeuo pipefail`,并且失败要给出**可操作的**错误信息,不要只是退出码。
- 不要把密钥写进脚本或 echo 出来。从 `.env` / 环境变量读。
- 任何会改数据或花钱的脚本,在文件头第一行注释里写清楚。
+126
View File
@@ -0,0 +1,126 @@
#!/usr/bin/env bash
# End-to-end proof that the scaffold works against the REAL DeepSeek API.
#
# compose up → migrate → start API → dev token → POST /v1/agent/chat
# → print the ob_ai_call row and the bl_ledger rows
#
# This is the only script that spends real money. Everything in `make test` is
# hermetic.
set -Eeuo pipefail
cd "$(dirname "$0")/.."
COMPOSE="docker compose -f deploy/docker-compose.dev.yml"
API_PORT="${API_PORT:-8010}"
BASE="http://127.0.0.1:${API_PORT}"
LOG_DIR=".smoke"
API_LOG="${LOG_DIR}/api.log"
API_PID=""
bold() { printf '\n\033[1m== %s\033[0m\n' "$1"; }
fail() { printf '\033[31mFAIL: %s\033[0m\n' "$1" >&2; exit 1; }
cleanup() {
if [[ -n "${API_PID}" ]] && kill -0 "${API_PID}" 2>/dev/null; then
kill "${API_PID}" 2>/dev/null || true
wait "${API_PID}" 2>/dev/null || true
fi
}
trap cleanup EXIT
mkdir -p "${LOG_DIR}"
[[ -f .env ]] || fail ".env not found. Copy .env.example to .env and set DEEPSEEK_API_KEY."
set -a; source .env; set +a
[[ -n "${DEEPSEEK_API_KEY:-}" ]] || fail "DEEPSEEK_API_KEY is empty in .env"
bold "1/7 Starting PostgreSQL (pgvector) and Redis"
${COMPOSE} up -d postgres redis
for _ in $(seq 1 60); do
if ${COMPOSE} exec -T postgres pg_isready -U meida -d meida >/dev/null 2>&1; then break; fi
sleep 1
done
${COMPOSE} exec -T postgres pg_isready -U meida -d meida >/dev/null || fail "PostgreSQL never became ready"
${COMPOSE} exec -T redis redis-cli ping >/dev/null || fail "Redis never became ready"
echo "postgres + redis ready"
bold "2/7 Applying migrations"
uv run alembic upgrade head
${COMPOSE} exec -T postgres psql -U meida -d meida -c "\dt"
${COMPOSE} exec -T postgres psql -U meida -d meida -tAc \
"SELECT extname FROM pg_extension WHERE extname='vector'" | grep -q vector \
|| fail "pgvector extension missing"
echo "pgvector extension present"
bold "3/7 Starting the API"
uv run uvicorn apps.api.main:app --host 127.0.0.1 --port "${API_PORT}" >"${API_LOG}" 2>&1 &
API_PID=$!
for _ in $(seq 1 60); do
if curl -fsS "${BASE}/healthz" >/dev/null 2>&1; then break; fi
sleep 0.5
done
curl -fsS "${BASE}/healthz" >/dev/null || { cat "${API_LOG}"; fail "API did not start"; }
curl -fsS "${BASE}/readyz" | tee /dev/null >/dev/null || { cat "${API_LOG}"; fail "API not ready"; }
echo "api up on ${BASE} (logs: ${API_LOG})"
bold "4/7 Issuing a dev token"
DEVICE_KEY="smoke-$(date +%s)"
TOKEN_JSON=$(curl -fsS -X POST "${BASE}/v1/auth/dev-token" \
-H 'Content-Type: application/json' \
-d "{\"device_key\":\"${DEVICE_KEY}\"}")
TOKEN=$(printf '%s' "${TOKEN_JSON}" | python3 -c 'import json,sys;print(json.load(sys.stdin)["data"]["access_token"])')
USER_ID=$(printf '%s' "${TOKEN_JSON}" | python3 -c 'import json,sys;print(json.load(sys.stdin)["data"]["user_id"])')
[[ -n "${TOKEN}" ]] || fail "no access token issued"
echo "user_id=${USER_ID}"
bold "5/7 Calling POST /v1/agent/chat against the REAL DeepSeek API"
CHAT_JSON=$(curl -fsS -X POST "${BASE}/v1/agent/chat" \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer ${TOKEN}" \
-d '{"message":"三十岁男生,第一次去互联网公司面试,穿什么比较稳妥?"}')
printf '%s' "${CHAT_JSON}" >"${LOG_DIR}/chat.json"
python3 -m json.tool --no-ensure-ascii "${LOG_DIR}/chat.json"
python3 - "${LOG_DIR}/chat.json" <<'PYEOF'
import json, sys
with open(sys.argv[1], encoding="utf-8") as fh:
body = json.load(fh)
assert body["code"] == 0, body
data = body["data"]
assert data["reply"].strip(), "empty model reply"
print("\n--- MODEL REPLY -------------------------------------------------")
print(data["reply"])
print("-----------------------------------------------------------------")
print("provider={provider} model={model} prompt={prompt_id}@{prompt_version}".format(**data))
print("usage={}".format(data["usage"]))
print("quota_remaining={}".format(data["quota_remaining"]))
PYEOF
bold "6/7 ob_ai_call row (成本归因的唯一数据源, 架构文档 §13)"
${COMPOSE} exec -T postgres psql -U meida -d meida -x -c \
"SELECT call_id, user_id, scene, provider, model, prompt_id, prompt_version,
tokens_in, tokens_out, tokens_cached, latency_ms, cost_cents,
success, error_code, ref_type, created_at
FROM ob_ai_call
WHERE user_id = ${USER_ID}
ORDER BY created_at DESC"
ROWS=$(${COMPOSE} exec -T postgres psql -U meida -d meida -tAc \
"SELECT count(*) FROM ob_ai_call WHERE user_id = ${USER_ID} AND success")
[[ "${ROWS// /}" -ge 1 ]] || fail "no successful ob_ai_call row was written"
bold "7/7 bl_ledger rows (预扣三段式, 架构文档 §7.4)"
${COMPOSE} exec -T postgres psql -U meida -d meida -c \
"SELECT tx_id, delta, tx_type, status, quota_key, period_key, balance_after,
idempotency_key, created_at
FROM bl_ledger
WHERE user_id = ${USER_ID}
ORDER BY created_at"
COMMITTED=$(${COMPOSE} exec -T postgres psql -U meida -d meida -tAc \
"SELECT count(*) FROM bl_ledger
WHERE user_id = ${USER_ID} AND tx_type='consume' AND status='committed'")
[[ "${COMMITTED// /}" -ge 1 ]] || fail "quota was not committed after a successful call"
${COMPOSE} exec -T postgres psql -U meida -d meida -c \
"SELECT quota_key, period_key, used, limit_value FROM bl_quota WHERE user_id = ${USER_ID}"
printf '\n\033[32mSMOKE PASSED\033[0m — real DeepSeek reply received, ob_ai_call and bl_ledger populated.\n'
+33
View File
@@ -0,0 +1,33 @@
# `services/` — 独立部署的 AI 能力服务
架构 §3.2:业务域做单体,**AI 能力做独立服务**。理由是资源画像完全不同 ——
GPU 依赖、模型文件体积、推理并发模型、扩缩容节奏。混在一个进程里会导致
「换装排队把 API 拖慢」。
| 目录 | 服务 | 资源 | 状态 |
| --- | --- | --- | --- |
| `vision/` | 人像分割、人体关键点、色值采样、体型测算、人脸属性 | CPU + ONNX | 占位 |
| `matting/` | BiRefNet 批量抠图 + 白底标准化 + 规格校验 | GPU(按需) | 占位 |
| `tryon/` | 换装三工作流、发型更换与发色 | GPU | 占位(现有服务改造纳管) |
## 本地有 RTX 5090
开发机带一块 **RTX 5090(32GB VRAM)**。`vision` 的 RTMPose ONNX 推理
(TD-05/TD-06)与 `matting` 的 BiRefNet 抠图(TD-07)**完全可以在本地开发、
调参和跑批,不需要租云 GPU**。素材生产期的批量抠图也可以先在本地跑通再决定
要不要上云。
## 服务边界规矩
- 对上以**内部 API** 暴露,不共享进程内存。即使 V1 某个能力先做成进程内包,
接口也要按远程调用设计(明确超时、可序列化的入参出参)。
- 服务间调用必须带**服务间令牌**,且只允许内网访问(架构 §4.4)。
- 每次推理必须返回**资源消耗**(如 `gpu_seconds`)与**模型/工作流版本号**,
否则成本无法归因、效果变化无法回溯。
- 接受 `idempotency_key`,重复提交返回同一结果。
- 明确对外暴露**最大并发数**,让上游队列按容量限流。
## 不能放什么
- 业务判断。这里只做推理,「该不该做这次推理」是业务域的事。
- 直接访问业务库。需要数据由调用方传入。
+1
View File
@@ -0,0 +1 @@
"""独立部署的 AI 能力服务(架构文档 §4.3)。"""
+17
View File
@@ -0,0 +1,17 @@
# `services/matting/` — 抠图批处理(占位)
架构 §4.3。离线批处理,GPU 按需。
- **BiRefNet**(TD-07 / ADR-08)。**禁用 RMBG-2.0**(许可证原因)。
- 输出白底标准化图 + 规格校验,写 `kb_asset.tryon_ready`
## 本地有 RTX 5090(32GB VRAM)
**BiRefNet 抠图可以完全在本地开发、调参和跑批,不需要租云 GPU。**
素材生产期(TD-26:300 件 `tryon_ready` 是关键路径)先在本地测出每周产能,
再决定要不要上云扩产能。
## 验收
FR-KB-08:10 件样本走通全流程;`tryon_ready` 标志准确;
抠图人工质检通过率 ≥ 90%。
+1
View File
@@ -0,0 +1 @@
"""抠图批处理(占位):BiRefNet 抠图 + 白底标准化 + 规格校验。"""
+14
View File
@@ -0,0 +1,14 @@
# `services/tryon/` — 换装服务(占位)
现有自有服务,需按架构 §4.4 改造后纳管。**不改造它会成为整个系统的不稳定源。**
| 改造项 | 要求 |
| --- | --- |
| 鉴权 | 改为内部服务,仅内网/网关调用,带服务间令牌 |
| 调用模式 | 同步长连接 → **提交任务 + 回调通知**(或任务查询接口) |
| 幂等 | 接受 `idempotency_key`,重复提交返回同一结果 |
| 并发控制 | 明确最大并发数并对外暴露,让上游队列按容量限流 |
| 成本埋点 | 每次推理返回 `gpu_seconds` 与模型版本,供成本归因 |
| 结果存储 | 阿里云 OSS → 腾讯云 COS;历史数据迁移或双读 |
| 输入校验 | 前置校验图片规格、人脸存在性、单品白底图规格,尽早失败 |
| 版本管理 | 模型/工作流版本号随结果返回,便于回溯效果变化 |
+1
View File
@@ -0,0 +1 @@
"""换装服务(占位):现有 GPU 服务改造纳管。"""
+30
View File
@@ -0,0 +1,30 @@
# `services/vision/` — 视觉服务(占位)
架构 §8.2。独立服务,CPU + ONNX。
```
POST /internal/vision/analyze
├─ segment_portrait 腾讯云 SegmentPortraitPic → 前景掩膜
├─ body_keypoints RTMPose ONNX(自建,腾讯云已下线,ADR-06)
├─ face_landmarks 腾讯云 AnalyzeFace(90点) / AnalyzeDenseLandmarks(888点)
├─ face_attributes 腾讯云 DetectFaceAttributes
├─ color_sampling 自研:按关键点定位采样区 → Lab,稳健统计(去高光/阴影)
└─ body_measure 自研:掩膜宽度剖面法(TD-06、ADR-07)
返回:结构化特征 + 每项置信度 + 质量问题标记
```
## 本地有 RTX 5090
**RTMPose ONNX 推理可以完全在本地开发和调参,不需要租云 GPU。**
准确率验证(体型 ≥ 80%、季型 ≥ 75%、脸型 ≥ 80%,各 30~50 样本,FR-RPT-05)
也可以在本地跑完。
## 质量门禁必须前置
架构 §8.2:图片质量不达标(多人、遮挡、侧脸过大、光线过暗)应在分析**前**就
返回明确的重拍指引,而不是硬算出一个低置信度结果。
**用户宁愿重拍一次,也不愿收到一份错的报告。**
## 人脸原图
不落盘。处理完立即删除(TD-20 / CR-02)。只返回派生特征。
+1
View File
@@ -0,0 +1 @@
"""视觉推理服务(占位):分割/关键点/色值采样/体型测算/人脸属性。"""