feat(gateway): 全局串行化(并发=1) + 记录入参/出参/耗时日志

并发模型从「每worker并发1 + 多worker并行」改为全局串行:同一时间
只处理1个请求,其余排队;多 worker 仅作热备(主 worker 坏了才用备机)。
接口4(face/features) 走豆包、不占 GPU,不纳入串行。

- pool: asyncio.Semaphore(max_global_concurrency=1) + acquire/release_global_slot
  (依赖单进程 uvicorn 部署,已在注释中标注)
- forward: proxy_request 最外层 acquire 全局槽、try/finally 全路径释放;
  入参 multipart 解析挂 request.state;原重试/故障转移逻辑抽到 _dispatch_with_retries
- reqlog(新): 标量入参保留;图片(file/base64)存盘转URL,绝不内嵌base64;
  出参递归摘要截断(landmarks/长串/大数组)
- logging_middleware: RequestLogEntry 加 request_params/response_data,
  jsonl 全量记录;_load_from_logfile 同步映射防重启丢字段;get_stats recent 暴露
- /gateway-health 暴露 global_max/global_busy/global_waiting
- config: dispatch 新增 max_global_concurrency / max_queue_wait_seconds
- tests: test_reqlog + test_gateway_serialization(9 用例)

顺带提交此前未提交的网关统计(daily stats 接口)与耗时看板(api_timing_dashboard.html)。

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Ubuntu
2026-07-23 22:56:48 +08:00
co-authored by Claude
parent 98b9108837
commit bb9f55e93c
10 changed files with 1037 additions and 7 deletions
+46 -4
View File
@@ -19,10 +19,13 @@ from fastapi.responses import JSONResponse
from gateway.pool import (
NoWorkerAvailable,
acquire_global_slot,
acquire_worker,
mark_worker_unhealthy,
release_global_slot,
release_worker,
)
from gateway.reqlog import extract_form_params
logger = logging.getLogger("gateway.forward")
@@ -143,17 +146,56 @@ async def proxy_request(request: Request, path: str) -> JSONResponse:
"""
from gateway.config import get_config
cfg = get_config()
# --- 1. 读取客户端请求体(原始字节,不做解析) ---
body = await request.body()
# --- 入参日志解析(非致命,绝不影响转发) ---
try:
request.state.log_request_params = extract_form_params(
request.headers.get("content-type", ""),
body,
cfg["static_dir"],
cfg["public_base_url"],
)
except Exception as ex: # noqa: BLE001
request.state.log_request_params = {"_parse_error": str(ex)}
# --- 全局串行槽(GPU 请求在此排队;超时拒绝) ---
if not await acquire_global_slot(cfg):
request.state.worker_url = ""
return JSONResponse(
status_code=503,
content={
"code": 1007,
"message": "排队超时,请稍后重试",
"request_id": f"gw-{uuid.uuid4().hex[:8]}",
"data": None,
},
)
try:
return await _dispatch_with_retries(request, path, cfg, body)
finally:
release_global_slot()
async def _dispatch_with_retries(
request: Request, path: str, cfg: dict, body: bytes
) -> JSONResponse:
"""在已持有全局串行槽的前提下:选 worker → 转发 → 重试 → 改写 base64 → 返回。
故障转移:worker 失败 → mark_worker_unhealthy → release_worker → 循环再 acquire
(此时主 worker 已下线,自动选备机)。全程在同一个全局槽持有期内,故
「平时只打主 worker,坏了才用备机」。
"""
dispatch_cfg = cfg["dispatch"]
token = cfg["shared_password"]
request_timeout = dispatch_cfg["request_timeout_seconds"]
max_retries = dispatch_cfg.get("max_retries", 1)
retry_on_failure = dispatch_cfg.get("retry_on_failure", True)
public_base_url = cfg["public_base_url"]
static_dir = cfg["static_dir"]
# --- 1. 读取客户端请求体(原始字节,不做解析) ---
body = await request.body()
# --- 2. 获取 worker(最多重试 max_retries+1 次) ---
attempts = max_retries + 1
last_error_response = None