feat(gateway): 全局串行化(并发=1) + 记录入参/出参/耗时日志
并发模型从「每worker并发1 + 多worker并行」改为全局串行:同一时间 只处理1个请求,其余排队;多 worker 仅作热备(主 worker 坏了才用备机)。 接口4(face/features) 走豆包、不占 GPU,不纳入串行。 - pool: asyncio.Semaphore(max_global_concurrency=1) + acquire/release_global_slot (依赖单进程 uvicorn 部署,已在注释中标注) - forward: proxy_request 最外层 acquire 全局槽、try/finally 全路径释放; 入参 multipart 解析挂 request.state;原重试/故障转移逻辑抽到 _dispatch_with_retries - reqlog(新): 标量入参保留;图片(file/base64)存盘转URL,绝不内嵌base64; 出参递归摘要截断(landmarks/长串/大数组) - logging_middleware: RequestLogEntry 加 request_params/response_data, jsonl 全量记录;_load_from_logfile 同步映射防重启丢字段;get_stats recent 暴露 - /gateway-health 暴露 global_max/global_busy/global_waiting - config: dispatch 新增 max_global_concurrency / max_queue_wait_seconds - tests: test_reqlog + test_gateway_serialization(9 用例) 顺带提交此前未提交的网关统计(daily stats 接口)与耗时看板(api_timing_dashboard.html)。 Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
+46
-4
@@ -19,10 +19,13 @@ from fastapi.responses import JSONResponse
|
||||
|
||||
from gateway.pool import (
|
||||
NoWorkerAvailable,
|
||||
acquire_global_slot,
|
||||
acquire_worker,
|
||||
mark_worker_unhealthy,
|
||||
release_global_slot,
|
||||
release_worker,
|
||||
)
|
||||
from gateway.reqlog import extract_form_params
|
||||
|
||||
logger = logging.getLogger("gateway.forward")
|
||||
|
||||
@@ -143,17 +146,56 @@ async def proxy_request(request: Request, path: str) -> JSONResponse:
|
||||
"""
|
||||
from gateway.config import get_config
|
||||
cfg = get_config()
|
||||
|
||||
# --- 1. 读取客户端请求体(原始字节,不做解析) ---
|
||||
body = await request.body()
|
||||
|
||||
# --- 入参日志解析(非致命,绝不影响转发) ---
|
||||
try:
|
||||
request.state.log_request_params = extract_form_params(
|
||||
request.headers.get("content-type", ""),
|
||||
body,
|
||||
cfg["static_dir"],
|
||||
cfg["public_base_url"],
|
||||
)
|
||||
except Exception as ex: # noqa: BLE001
|
||||
request.state.log_request_params = {"_parse_error": str(ex)}
|
||||
|
||||
# --- 全局串行槽(GPU 请求在此排队;超时拒绝) ---
|
||||
if not await acquire_global_slot(cfg):
|
||||
request.state.worker_url = ""
|
||||
return JSONResponse(
|
||||
status_code=503,
|
||||
content={
|
||||
"code": 1007,
|
||||
"message": "排队超时,请稍后重试",
|
||||
"request_id": f"gw-{uuid.uuid4().hex[:8]}",
|
||||
"data": None,
|
||||
},
|
||||
)
|
||||
|
||||
try:
|
||||
return await _dispatch_with_retries(request, path, cfg, body)
|
||||
finally:
|
||||
release_global_slot()
|
||||
|
||||
|
||||
async def _dispatch_with_retries(
|
||||
request: Request, path: str, cfg: dict, body: bytes
|
||||
) -> JSONResponse:
|
||||
"""在已持有全局串行槽的前提下:选 worker → 转发 → 重试 → 改写 base64 → 返回。
|
||||
|
||||
故障转移:worker 失败 → mark_worker_unhealthy → release_worker → 循环再 acquire
|
||||
(此时主 worker 已下线,自动选备机)。全程在同一个全局槽持有期内,故
|
||||
「平时只打主 worker,坏了才用备机」。
|
||||
"""
|
||||
dispatch_cfg = cfg["dispatch"]
|
||||
token = cfg["shared_password"]
|
||||
request_timeout = dispatch_cfg["request_timeout_seconds"]
|
||||
max_retries = dispatch_cfg.get("max_retries", 1)
|
||||
retry_on_failure = dispatch_cfg.get("retry_on_failure", True)
|
||||
public_base_url = cfg["public_base_url"]
|
||||
static_dir = cfg["static_dir"]
|
||||
|
||||
# --- 1. 读取客户端请求体(原始字节,不做解析) ---
|
||||
body = await request.body()
|
||||
|
||||
# --- 2. 获取 worker(最多重试 max_retries+1 次) ---
|
||||
attempts = max_retries + 1
|
||||
last_error_response = None
|
||||
|
||||
Reference in New Issue
Block a user