feat(接口2): 移植head3d发际线管线 + 接口2实现方案文档

- 从head3d复制发际线检测管线到 hairline/ 包:MediaPipe Tasks + SegFormer分割
  + 17锚点射线检测 + 502点mesh(face_ext.obj)+UV
- 复制模型:face_landmarker.task(3.7MB)、SegFormer config/preprocessor
  (model.safetensors 340MB 单独下载中)
- 新增 docs/接口2-C端生发-技术实现方案.md:第一步=发际线曲线叠加预览图,
  新增gender必填参数,按性别贴图数量输出(female5/male4),hairline_type英文key,
  服务端cv2逐三角形warp渲染器(head3d只有浏览器端Three.js渲染)
- 接口文档.md 接口2章节同步:gender参数、输出语义、错误码说明
- hairline_texture/ 9张发际线贴图入库
This commit is contained in:
xsl
2026-06-14 16:59:39 +08:00
parent 5e513c7006
commit db32fa12e5
19 changed files with 6913 additions and 10 deletions
@@ -0,0 +1,246 @@
# 接口 2:C 端生发 — 技术实现方案(第一步:发际线遮罩渲染)
> 在 **高性能 worker(GPU 机)** 上实现,与接口 1 同机。对外接口经网关代理(见 [`系统架构-网关与高性能后端.md`](系统架构-网关与高性能后端.md))。
> 发际线检测算法移植自 **head3d** 项目(已实现 502 点 mesh + UV 贴图方案)。
---
## 0. 本期范围(第一步)
接口 2 输入用户正面照 + **性别**,按性别对应的发际线类型贴图,**逐张把发际线曲线渲染到照片上**,输出多张「叠加了建议发际线的预览图」,按固定顺序返回。
- **本期只做「渲染遮罩/预览图」**,不做真正的文生图生发(那是后续步骤)。当前 `image_url` 返回的是「原照片 + 发际线曲线叠加图」。
- 排序 `order` 本期不计算,按贴图顺序 `1..N`
---
## 1. 与现接口文档的差异(接口 2 需同步更新 `接口文档.md`)
| 项 | 现状 | 本期改为 |
|----|------|----------|
| 输入参数 | `beauty_enabled` | **新增必填 `gender``male`/`female`**`beauty_enabled` 保留但本期不生效 |
| 输出 `results[]` 数量 | Mock 2 个 | = 该性别的贴图数量(**female 5 张 / male 4 张** |
| `results[].image_url` | 生发后图片 | **本期 = 发际线曲线叠加在原照片上的预览图** |
| `results[].hairline_type` | 中文(花瓣形…) | **英文 key**`flower`/`wave`/`heart`/`ellipse`/`straight`/`m`/`inverse_arc` |
| `results[].order` | 排序 | 本期固定 `1..N`(不排序) |
| 错误码 1004(性别判断异常) | 待确认 | `gender` 改为必填入参 → **不再自动判别性别**1004 仅在 `gender` 非法值时使用(或弃用) |
> ⚠️ 这是接口 2 的**有意契约变更**(加入参 + 改输出语义),需在 `接口文档.md` 接口 2 章节同步。其余 4 个接口契约不变。
### gender → 贴图集合
`hairline_texture/` 目录下贴图(512×512 RGBA,白色发际线曲线在顶部 UV 条带):
| gender | 贴图文件 | hairline_type (key) |
|--------|----------|---------------------|
| female | `girl_ellipse.png` | `ellipse` |
| female | `girl_flower.png` | `flower` |
| female | `girl_heart.png` | `heart` |
| female | `girl_straight.png` | `straight` |
| female | `girl_wave.png` | `wave` |
| male | `man_ellipse.png` | `ellipse` |
| male | `man_m.png` | `m` |
| male | `man_straight.png` | `straight` |
| male | `man_ inverse_arc.png` | `inverse_arc` |
> 注意 `man_ inverse_arc.png` 文件名里有个空格,代码里按 `gender + '_' + key` 生成文件名时需保留/清洗一致。建议**启动时扫描目录**建立 `{gender: [(key, path)]}` 映射,而不是硬编码文件名,并把文件名规范化(去空格)。
---
## 2. 已从 head3d 复制到本项目的文件
全部放在 `hairline/` 包下(已复制,agent 直接用):
```
hairline/
├── __init__.py
├── constants.py # 17 锚点、UV 偏移、分割类别、矢状-arc 常量、HF 模型 id
├── obj_io.py # OBJ 读写
├── face_landmarks.py # MediaPipe Tasks FaceLandmarker 封装(用 face_landmarker.task
├── face_parsing.py # SegFormer 人脸分割封装
├── hairline_2d.py # 射线检测发际线 2D + 平滑 + 回退
├── lift_3d.py # 2D→3D 矢状-arc 提升 + 中间行 + assemble 502 点
├── extract_hairline.py # 主管线(image → 502 点),可复用 run()
├── _index_map_data.py # 468→OBJ indexMapbuild_extended_obj 用,本期渲染不需要)
├── _mediapipe_subprocess.py# WSL 下子进程跑 MediaPipe 的兜底(可选)
├── models/
│ ├── face_landmarker.task # MediaPipe 模型(3.7MB,已复制)
│ └── face-parsing/ # SegFormer 权重(离线,已下载,见 OFFLINE_ASSETS.md
│ ├── config.json
│ ├── preprocessor_config.json
│ └── model.safetensors
├── mesh/
│ ├── face_ext.obj # 502 点扩展 mesh + UV + 三角面(渲染器读这个)
│ └── face.obj # 原始 468 点 mesh(参考/重生成用)
└── reference/
├── texture0.png # head3d 原 5 弧线贴图(核对 UV 用)
└── uv_template.png # UV 布局参考
```
发际线类型贴图在仓库根目录 `hairline_texture/`(用户提供,9 张)。
### 2.1 移植后需要修改的集成点
1. **`face_landmarks.py``DEFAULT_MODEL_PATH`**:原逻辑是 `dirname(dirname(__file__))/models/...`head3d 里模块在 `python/` 子目录)。现在模块在 `hairline/` 根,该路径会指向 `hair/models/`,而模型在 `hairline/models/`。**改为** `os.path.join(os.path.dirname(__file__), "models", "face_landmarker.task")`
2. **`face_parsing.py` 离线加载**`C.HF_FACE_PARSER_MODEL` 当前是 HF 在线 id `"jonathandinu/face-parsing"`。内网/离线改为本地目录:把 `constants.py``HF_FACE_PARSER_MODEL` 指向 `hairline/models/face-parsing` 的绝对路径(`from_pretrained` 支持本地目录);或设 `HF_HUB_OFFLINE=1`
3. **相对导入**:模块用 `from . import constants`,已加 `hairline/__init__.py`,作为包导入即可(`from hairline.extract_hairline import run`)。
4. **GPU**`FaceParser(device="cuda")`worker 有 GPU。
---
## 3. 算法管线(整体)
```
输入: 用户正面照 + gender
[A] head3d 管线(复用 hairline.extract_hairline 的步骤)
- MediaPipe 468 点(face_landmarker.task
- SegFormer 人脸分割 → parse_map
- 17 锚点射线检测发际线 → 17 个 2D 点 → 平滑
- 矢状-arc 提升 → 502 点(归一化 x,y,z)
│ 失败处理:无人脸→1001
[B] 投影到图像像素
- 502 点的 (x,y) × (W,H) → 502 个 2D 图像坐标
- 读 face_ext.objUV(502) + 扩展三角面(涉及顶点 ≥468 的 64 个三角形)
[C] 逐张贴图渲染(新写的服务端渲染器,本方案核心)
for 每个该性别的发际线贴图 t:
- 对每个扩展三角形:src=UV→贴图像素, dst=投影 2D 坐标 → cv2 仿射 warp
- 累积成一张 RGBA 曲线层(贴图 alpha 控制曲线/透明)
- 把曲线层 alpha 合成到原照片上 → 预览图
[D] 输出
results[] = N 个 {image(预览图), hairline_type(key), order=1..N}
worker 侧每张图以 base64 返回(见 §6)
```
---
## 4. 渲染器(新代码,本期重点)★
head3d 把贴图渲染到照片是**浏览器 Three.js** 做的(`/preview` ortho overlay),**没有服务端实现**。本期新写一个 **OpenCV 逐三角形仿射 warp** 渲染器,无需 OpenGL 离屏上下文,确定性好、部署简单。
### 4.1 原理
face_ext.obj 的 502 顶点里:
- `[0..467]` MediaPipe 点,其中 17 个 `MP_TOP_ANCHORS` 是发际线 ribbon 的**下边沿**
- `[468..484]` 中间行、`[485..501]` 发际线行,是 ribbon 的中、上两行。
这 34 个新点 + 17 个锚点之间连成 64 个三角形(ribbon),它们的 UV 落在贴图**顶部条带**V_raw≈0.67..0.94,正是发际线曲线所在)。所以只要把**这 64 个三角形**按 UV→图像坐标 warp,就能把贴图里的发际线曲线贴到照片的额头/发际线区域。
### 4.2 步骤
```python
# 伪代码
def render_hairline_overlay(photo_bgr, points502_norm, ext_faces, uv502, texture_rgba):
H, W = photo_bgr.shape[:2]
# 502 点投影到图像像素
img_xy = points502_norm[:, :2] * [W, H] # (502, 2)
TW, TH = texture_rgba.shape[1], texture_rgba.shape[0] # 512, 512
overlay = np.zeros((H, W, 4), np.float32) # 累积曲线层 RGBA
for (i, j, k) in ext_faces: # 仅扩展三角形(顶点含 ≥468
dst = img_xy[[i, j, k]].astype(np.float32) # 图像坐标
# UV → 贴图像素。注意 flipY:贴图 y = (1 - v_raw) * TH
src = np.array([[uv502[v][0]*TW, (1-uv502[v][1])*TH] for v in (i,j,k)], np.float32)
M = cv2.getAffineTransform(src, dst)
warped = cv2.warpAffine(texture_rgba, M, (W, H), flags=cv2.INTER_LINEAR,
borderMode=cv2.BORDER_CONSTANT, borderValue=(0,0,0,0))
# 三角形掩码,避免覆盖整张 warp 结果
tri_mask = np.zeros((H, W), np.uint8)
cv2.fillConvexPoly(tri_mask, dst.astype(np.int32), 255)
sel = tri_mask > 0
overlay[sel] = warped[sel] # 逐三角形写入(相邻共享边,覆盖等价)
# alpha 合成到原照片
a = overlay[:, :, 3:4] / 255.0
out = photo_bgr.astype(np.float32)
out = out * (1 - a) + overlay[:, :, :3][..., ::-1] * a # RGBA→BGR 注意通道序
return out.astype(np.uint8)
```
### 4.3 注意点
- **通道序**:贴图是 RGBA,照片 OpenCV 是 BGR,合成时注意 R/B 调换。
- **flipY**face_ext.obj 的 UV 是 V_rawV=1 对应贴图顶部),转贴图像素 y 要 `(1 - v)`,与 head3d Three.js `texture.flipY=true` 一致。
- **只 warp 扩展三角形**:从 face_ext.obj 筛出顶点索引含 ≥468 的面(约 64 个)。不要 warp 整脸。
- **抗锯齿/接缝**:逐三角形 `fillConvexPoly` 掩码可能在共享边留 1px 缝。可对 `tri_mask` 略膨胀,或最后对 overlay alpha 做轻微羽化。先跑通看效果再优化。
- **裁剪到额头**:曲线层只在 ribbon 区域有内容(贴图其余透明),天然不会画到脸下半部。
---
## 5. 依赖
worker 已有(接口 1):`opencv-python``numpy``Pillow`、torch(CUDA)。接口 2 **新增**
```
mediapipe>=0.10 # Tasks Vision FaceLandmarker(注意与接口1的 solutions API 可共存)
transformers>=4.40 # SegFormer 人脸分割
# torch/torchvision 已由接口1引入(worker CUDA 版)
```
> ⚠️ **两套人脸分割模型**:接口 1 用 BiSeNet`79999_iter.pth`),接口 2 用 head3d 的 SegFormer`jonathandinu/face-parsing`)。两者并存,显存/内存够(worker 32G+GPU)。后续可评估是否统一为一个分割模型,本期先各用各的,**不强行合并**。
>
> ⚠️ **MediaPipe API 差异**:接口 1 用 `mp.solutions.face_mesh`468 点 + 虹膜 refine),接口 2 用 `mp.tasks.vision.FaceLandmarker`(读 `.task` 文件)。同一个 mediapipe 包都支持,但版本需兼容两者(建议先用一个版本把两接口都跑通)。
---
## 6. worker 集成(接口 2 handler
`app.py` 替换 `/api/v1/hair/grow` 的 Mock
```
1. 解析图片(三选一)+ 读 gender(必填,male/female;非法→1004 或 1008 参数错误)
2. 校验(大小/解码/分辨率,同接口1)
3. 跑 hairline.extract_hairline 的步骤拿 502 点(无人脸→1001)
4. 按 gender 取贴图集合(启动时扫描 hairline_texture/ 建映射)
5. for 每张贴图: render_hairline_overlay → PNG
6. results[] = [{image_base64, hairline_type, order}], 逐张 base64
7. return ok({"results": results})
```
- **拆分架构**worker 返回 `results[].image_base64`,**不落盘不拼 URL**。网关把每个 `image_base64` 落盘改写成 `image_url`(架构文档 §9 的映射表需支持**数组里的图片字段** `results[].image`)。
- 模型单例:`FaceLandmarker``FaceParser` 在模块加载时初始化一次,避免每请求重建。face_ext.obj 的 UV/faces 也只解析一次缓存。
---
## 7. 离线资产(内网部署)
接口 2 新增需要随项目带入内网的模型(已下载,登记到 `OFFLINE_ASSETS.md`):
- `hairline/models/face_landmarker.task`MediaPipe~3.7MB
- `hairline/models/face-parsing/`SegFormerconfig + preprocessor + model.safetensors
> SegFormer 加载方式改本地路径后,内网无需联网(见 §2.1)。
---
## 8. 开发步骤与验证(agent 执行)
| 阶段 | 内容 | 验证 |
|------|------|------|
| **M0 跑通管线** | 修好集成点(§2.1),用一张人像跑 `hairline.extract_hairline.run()` 得 502 点 JSON | 502 点、valid_hairline 有 true |
| **M1 解析 mesh** | 读 face_ext.obj 拿 UV + 扩展三角面(顶点≥468 的面),缓存 | 打印扩展面数(~64)、502 个 UV |
| **M2 渲染器** | 实现 `render_hairline_overlay`,对 1 张贴图渲染 | 输出预览图,**目视**:发际线曲线贴在额头正确位置、跟随脸 |
| **M3 全量 + 性别** | 扫描 `hairline_texture/` 建 gender→贴图映射,按性别渲染 N 张 | female 出 5 张、male 出 4 张,hairline_type 对 |
| **M4 接 app.py** | handler + gender 必填 + base64 返回 | curl 验证 results 数量/字段;无人脸→1001;缺 gender→报错 |
| **M5 网关映射** | 网关支持 `results[].image_base64``image_url`(网关任务书侧) | 端到端经网关返回 image_url,公网可访问 |
**M2 是关键里程碑**:渲染器对齐效果好不好,决定整个接口可用性,先用几张测试人像目视确认贴合。
---
## 9. 风险与待办
1. **新贴图 UV 是否与 texture0 完全一致**:本方案假设 9 张贴图沿用 head3d 的顶部条带 UV 布局(已肉眼确认曲线在顶部)。M2 渲染若位置偏移,核对贴图内容所在的 V 区间与 `UV_MIDDLE_DV/UV_HAIRLINE_DV`
2. **接缝/锯齿**:逐三角形 warp 的共享边接缝,M2 跑通后按 §4.3 优化。
3. **歪头/非正面**head3d 矢状-arc 假设近正脸,大角度发际线贴合差。可复用接口 1 的 solvePnP 做前置姿态校验(可选)。
4. **秃头/高发际线/刘海**:SegFormer 找不到头发时射线回退几何外推,曲线可能偏高;valid_hairline 标记可用于提示。
5. **排序**:本期 order=1..N。后续排序需定义依据(脸型/额型匹配度)。
6. **真正的生发(文生图)**:本期只出遮罩/预览。下一步把预览图/曲线作为 ControlNet/inpaint 输入接文生图模型,再替换 `image_url` 为真实生发图。
---
> **文档版本**: v1.0 **创建日期**: 2026-06-14 算法来源: head3d502 点 mesh + UV)| 运行位置: worker(GPU)
> **本期产出**: 发际线曲线叠加预览图(非最终生发图)
+13 -10
View File
@@ -177,7 +177,9 @@
## 接口 2C 端生发接口
**说明**:输入用户正面照,输出生发后的图片,以及推荐的发际线(可能多张),并按合适度排序
**说明**:输入用户正面照 + 性别,按性别对应的发际线类型,逐张把建议发际线渲染到照片上,输出多张方案
> **当前阶段(第一步)**`image_url` 返回的是「**原照片 + 发际线曲线叠加的预览图**」,尚未做真正的文生图生发;后续会替换为生发后图片。实现见 [`接口2-C端生发-技术实现方案.md`](接口2-C端生发-技术实现方案.md)。
**请求**`POST /api/v1/hair/grow`
@@ -187,19 +189,20 @@
| 参数 | 类型 | 必填 | 说明 |
|------|------|------|------|
| beauty_enabled | bool | 否 | 生发图是否带美颜效果,默认 false。提供该开关 |
| gender | string | **是** | 性别:`male` / `female`。决定使用的发际线贴图集合 |
| beauty_enabled | bool | 否 | 生发图是否带美颜效果,默认 false(当前阶段不生效) |
### 输出(data
`results`:发际线方案数组(可能多张),每个元素:
`results`:发际线方案数组,**数量 = 该性别的发际线类型数**(`female` 5 个 / `male` 4 个)。每个元素:
| 字段 | 类型 | 说明 |
|------|------|------|
| image_url | string | 生发后图片 URL |
| hairline_type | string | 图片对应的发际线形(如:花瓣形、波浪形 |
| order | int | 排序序号(1 = 最优,2 次之 … |
| image_url | string | 方案预览图 URL(当前 = 发际线叠加图) |
| hairline_type | string | 发际线类型 key`ellipse`/`flower`/`heart`/`straight`/`wave`female),`ellipse`/`m`/`straight`/`inverse_arc`male |
| order | int | 排序序号(当前阶段固定 `1..N`,按贴图顺序,暂不计算合适度 |
### 响应示例(当前 Mock 返回值)
### 响应示例
```json
{
@@ -208,14 +211,14 @@
"request_id": "mock-request-id",
"data": {
"results": [
{ "image_url": "https://hair.xiangsilian.com/static/sample.jpg", "hairline_type": "花瓣形", "order": 1 },
{ "image_url": "https://hair.xiangsilian.com/static/sample.jpg", "hairline_type": "波浪形", "order": 2 }
{ "image_url": "https://hair.xiangsilian.com/static/annotations/uuid1.png", "hairline_type": "ellipse", "order": 1 },
{ "image_url": "https://hair.xiangsilian.com/static/annotations/uuid2.png", "hairline_type": "flower", "order": 2 }
]
}
}
```
> 识别失败时返回通用错误码(1001 / 1002 / 1003 等)。
> 识别失败时返回通用错误码(1001 / 1002 / 1003 等)。`gender` 缺失或非法值返回参数错误(1008);本接口已改为必填入参,不再自动判别性别(1004 不再使用)。
---
+7
View File
@@ -0,0 +1,7 @@
"""接口 2C 端生发)发际线渲染管线。
从 head3d 项目移植:MediaPipe 468 点 + SegFormer 人脸分割 + 17 锚点射线检测发际线
→ 502 点 3D meshface_ext.obj+ UV → 把发际线类型贴图渲染到照片对应位置。
模块来源见 docs/接口2-C端生发-技术实现方案.md。
"""
+55
View File
@@ -0,0 +1,55 @@
"""Auto-extracted indexMap[468] from the SDK's hardcode_data.h.
Each entry: OBJ-vertex-index i -> MediaPipe canonical landmark index.
Do not edit by hand; regenerate from the C++ source.
"""
INDEX_MAP_468 = [
127, 34, 139, 11, 0, 37, 232, 231, 120, 72,
39, 128, 121, 47, 104, 69, 67, 175, 171, 148,
118, 50, 101, 73, 40, 9, 151, 108, 48, 115,
131, 194, 204, 211, 74, 185, 80, 42, 183, 92,
186, 230, 229, 202, 212, 214, 83, 18, 17, 76,
61, 146, 160, 29, 30, 56, 157, 173, 106, 135,
192, 203, 165, 98, 21, 71, 68, 51, 45, 4,
144, 24, 23, 77, 91, 205, 187, 201, 200, 182,
90, 181, 85, 84, 206, 36, 140, 193, 189, 244,
159, 158, 28, 247, 246, 161, 236, 3, 196, 54,
168, 8, 117, 228, 31, 55, 97, 99, 126, 100,
166, 79, 218, 155, 154, 26, 209, 49, 136, 150,
217, 223, 52, 53, 134, 170, 43, 119, 226, 130,
63, 238, 20, 242, 46, 70, 156, 78, 62, 96,
143, 227, 123, 111, 44, 125, 19, 216, 153, 22,
167, 208, 142, 57, 60, 35, 113, 27, 210, 225,
137, 116, 41, 38, 129, 64, 240, 102, 207, 184,
169, 149, 176, 105, 66, 122, 6, 147, 65, 107,
89, 180, 93, 15, 86, 14, 87, 145, 88, 179,
95, 138, 172, 215, 58, 219, 81, 195, 199, 82,
163, 110, 234, 109, 235, 191, 222, 141, 221, 197,
25, 7, 33, 220, 237, 245, 162, 188, 174, 2,
241, 164, 12, 13, 198, 133, 112, 243, 239, 190,
32, 178, 132, 177, 1, 213, 59, 94, 75, 224,
233, 114, 124, 356, 389, 368, 302, 267, 452, 350,
349, 303, 269, 357, 343, 277, 453, 333, 332, 297,
152, 377, 347, 348, 330, 304, 270, 336, 337, 278,
279, 360, 418, 262, 431, 408, 409, 310, 415, 407,
410, 450, 422, 430, 434, 313, 314, 306, 307, 375,
387, 388, 260, 286, 414, 398, 335, 406, 364, 367,
416, 423, 358, 327, 251, 284, 298, 281, 5, 373,
374, 253, 320, 321, 425, 427, 411, 421, 405, 404,
315, 16, 426, 266, 400, 369, 322, 391, 417, 465,
464, 386, 257, 258, 466, 456, 399, 419, 285, 346,
340, 261, 413, 441, 460, 328, 355, 371, 329, 392,
439, 438, 382, 341, 256, 429, 420, 394, 379, 437,
443, 444, 283, 275, 440, 363, 338, 273, 451, 446,
342, 467, 293, 334, 282, 458, 461, 462, 276, 353,
383, 308, 324, 325, 300, 372, 345, 447, 352, 274,
248, 436, 381, 252, 393, 428, 287, 250, 384, 265,
259, 424, 292, 366, 271, 294, 455, 272, 432, 395,
299, 351, 280, 319, 295, 296, 403, 323, 454, 316,
380, 318, 402, 365, 435, 397, 344, 311, 291, 396,
268, 445, 254, 339, 449, 264, 10, 442, 370, 263,
255, 359, 412, 301, 378, 326, 457, 362, 459, 463,
354, 401, 361, 309, 376, 433, 289, 305, 448, 290,
288, 249, 103, 385, 331, 317, 312, 390,
]
+67
View File
@@ -0,0 +1,67 @@
"""Standalone MediaPipe FaceLandmarker runner used by web_service.py.
Running MediaPipe Tasks in the same process as the Flask dev server can
segfault under WSL (D3D12 EGL backend). Spawning a fresh subprocess per
request keeps the web server alive and lets us inject WSL-friendly env
vars before any mediapipe import.
Usage:
python -m python._mediapipe_subprocess <image_path> <output_npy>
"""
from __future__ import annotations
import os
import sys
os.environ.setdefault("LIBGL_ALWAYS_SOFTWARE", "1")
os.environ.setdefault("MESA_LOADER_DRIVER_OVERRIDE", "llvmpipe")
os.environ.setdefault("GALLIUM_DRIVER", "llvmpipe")
os.environ.setdefault("MEDIAPIPE_DISABLE_GPU", "1")
os.environ.setdefault("EGL_PLATFORM", "surfaceless")
import numpy as np # noqa: E402
THIS_DIR = os.path.dirname(os.path.abspath(__file__))
PROJECT_DIR = os.path.dirname(THIS_DIR)
if PROJECT_DIR not in sys.path:
sys.path.insert(0, PROJECT_DIR)
from python.face_landmarks import FaceLandmarker # noqa: E402
def main() -> int:
if len(sys.argv) != 3:
print(
"usage: python -m python._mediapipe_subprocess <image_path> <output_npy>",
file=sys.stderr,
)
return 2
image_path, output_path = sys.argv[1], sys.argv[2]
import cv2
bgr = cv2.imread(image_path)
if bgr is None:
print(f"could not read image: {image_path}", file=sys.stderr)
return 3
rgb = cv2.cvtColor(bgr, cv2.COLOR_BGR2RGB)
rgb = np.ascontiguousarray(rgb, dtype=np.uint8)
landmarker = FaceLandmarker(static_image_mode=True)
try:
landmarks = landmarker.detect(rgb)
finally:
landmarker.close()
if landmarks is None:
print("no face detected", file=sys.stderr)
return 4
np.save(output_path, landmarks.astype(np.float32))
return 0
if __name__ == "__main__":
sys.exit(main())
+165
View File
@@ -0,0 +1,165 @@
"""Shared constants for the hairline-extension pipeline.
These define the topology of the extended mesh and must stay in sync with
the C++ side (see sdk/ExtensionConstants.h). If you change anything here,
regenerate face_ext.obj and update the C++ header.
"""
from __future__ import annotations
# Number of MediaPipe FaceMesh landmarks (no iris refinement).
N_MP = 468
# Anchors along the upper boundary of the MediaPipe face mesh,
# ordered left-to-right when viewing the face frontally.
# Each anchor will get a paired hairline sample directly "above" it.
#
# Verify visually with scripts/show_anchors.py before locking these in.
MP_TOP_ANCHORS: list[int] = [
127, 234, 162, 21, 54, 103, 67, 109, 10,
338, 297, 332, 284, 251, 389, 356, 454,
]
N_ANCHORS = len(MP_TOP_ANCHORS) # 17
# Vertex ID layout in the extended array of length 468 + 2*N_ANCHORS.
# [0 .. 468) : MediaPipe canonical landmarks
# [468 .. 468+N) : middle row (between MP boundary and hairline)
# [468+N .. 468+2N) : hairline row
N_EXT = 2 * N_ANCHORS # 34
N_TOTAL = N_MP + N_EXT # 502
MIDDLE_START = N_MP # 468
HAIRLINE_START = N_MP + N_ANCHORS # 485
# Saggital head-curvature radius (in MediaPipe normalized-Y units),
# expressed as a fraction of face height. The head's mid-line cross
# section is treated locally as a circular arc; for a hairline / middle
# vertex located dy=(y_hair - y_anchor) above an MP top anchor (dy < 0
# since hairline y < anchor y) we compute its Z by
#
# z_hair = z_anchor + dy² / (2 R), R = HEAD_ARC_RADIUS_FRAC × face_h
#
# This always pushes the added vertex BACKWARD (toward +z in MP / face.obj
# convention, i.e. toward the back of the head), matching the actual
# anatomy. See README for the derivation.
#
# Smaller fraction = more pronounced backward bulge. 0.30 produces a
# moderate offset (~0.024 in normalized z for a 0.08-y hairline lift on
# a typical face).
HEAD_ARC_RADIUS_FRAC = 0.30
# Extra lift applied to the detected hairline along the face-up direction,
# expressed as a fraction of the MP face height. The 2D hairline detector
# stops at the hair-skin boundary (start of the visible hair); the mesh
# ribbon's top row should sit at the crown of the head instead, so the
# texture-overlay band can cover the whole forehead → crown region.
#
# 0.06 ≈ moves the hairline row up by 6% of face height, which on the
# reference photos lands the top row just above the visible hairline and
# below the crown — empirically tuned with the /preview slider and locked
# in as the project default. Bump to 0.10..0.15 for taller foreheads /
# higher crowns; drop to 0.03 to keep the ribbon hugging the hair-skin
# boundary.
HAIRLINE_CROWN_LIFT_FRAC = 0.06
# Pure-geometry hairline offset for the /preview 502-point pipeline.
#
# When placing the hairline row WITHOUT hair detection (works for bald /
# with-hair / hat — all head types), each anchor is offset upward along
# face-up by GEOMETRIC_HAIRLINE_OFFSET_FRAC × face_h in normalised
# image space. The sagittal-arc model then derives Z.
#
# 0.25 × face_h ≈ 0.12 normalised on a typical face (face_h ≈ 0.48),
# giving dz ≈ 0.05 — matches the real hairline distance observed on
# reference photos where hair detection succeeds.
GEOMETRIC_HAIRLINE_OFFSET_FRAC = 0.25
# UV layout for the 34 forehead-extension vertices.
#
# The texture (imgs/texture0.png, 512×512) is laid out with the original
# MediaPipe face skin in V_raw ≈ 0.00..0.77 (image y ≈ 117..511) and a
# horizontal stack of 5 hairline-design arcs at the TOP of the image
# (image y ≈ 30..170, i.e. V_raw ≈ 0.67..0.94). Those 5 arcs are the
# content that the extension strip is supposed to display: hairline row
# samples the topmost arc (blue), middle row samples the bottom arc
# (orange), and the 3 arcs in between fall out automatically because the
# ribbon triangle interpolates V linearly between the two rows.
#
# V conventions
# -------------
# uv_template.py / Three.js (with texture.flipY=true) treat V_raw=1 as
# the TOP of the image (image y=0) and V_raw=0 as the BOTTOM.
#
# U conventions — IMPORTANT
# -------------------------
# The 17 MP_TOP_ANCHORS in face.obj have NON-uniform u (≈ 0.00 at the
# temples, ≈ 0.50 at the forehead center, ≈ 1.00 at the other temple).
# Each ribbon triangle (anchor[i] → middle[i] → anchor[i+1] etc.) is a
# vertical column in UV space ONLY when the middle/hairline vertex
# inherits its U from the corresponding anchor. If we used a uniform
# 0.05..0.95 U for the strip the columns would slant relative to the
# anchor U values, warping the texture's 5 horizontal arcs into
# zig-zags. So `extension_uv_for` takes `anchor_u` and copies it.
UV_MIDDLE_DV = 0.110 # middle 行 V_raw 相对该列 anchor V_raw 上移这么多
UV_HAIRLINE_DV = 0.220 # hairline 行 V_raw 相对该列 anchor V_raw 上移这么多
# 为什么是"相对 anchor 平行偏移"而不是固定常数:
#
# face.obj 中 17 个 MP_TOP_ANCHORS 的 V_raw 是**非均匀弧形** (额头中央 MP 10
# = 0.7724, 太阳穴 MP 127 = 0.4668, 横跨 0.30 V 单位)。 如果 middle/hairline
# 用固定常数 V_raw (例如 0.82 / 0.998), 那每个 ribbon quad 的 V 跨度
# (= middle.V anchor[i].V) 在 17 列之间差异巨大 (中央列 0.05, 两端列 0.35,
# 差了 7 倍)。 贴图最底部的弧线 (V_raw ≈ 0.76) 正好落在 V 跨度大的列上 →
# 被拉伸成粗大色块, 而顶部弧线 (V_raw ≈ 0.93) 落在 V 跨度小的列上 → 被压
# 缩成细线。 这就是"最下面那条线特别粗、上面 4 根都细"的根因。
#
# 把 middle/hairline 的 V 也设成"anchor V + 固定 Δ", 每列的 V 跨度变成恒定
# 的 Δm / (Δh Δm), ribbon 在贴图上是上下都跟随 anchor 弧度的弯月形带,
# 贴图 5 条弧线在 mesh 上粗细均匀。
#
# 硬约束:
# 1. Δm > 0 且 Δm < Δh (顺序保持 anchor < middle < hairline, 防 V 反向)
# 2. anchor.V_max + Δh ≤ 1.0 (即 Δh ≤ 1 0.7724 = 0.2276)
# 否则中央列 hairline V 溢出, 采到贴图边缘的抗锯齿像素。
# 当前 Δh = 0.220 留 ≈ 0.008 V 单位 buffer; Δm = Δh / 2 让上下两段等宽。
def extension_uv_for(row: int, anchor_u: float, anchor_v: float) -> tuple[float, float]:
"""Return (u, v_raw) UV for an extension vertex.
row: 0 = middle, 1 = hairline.
anchor_u: U of the corresponding MP anchor (copy verbatim → ribbon column
is vertical in UV space).
anchor_v: V_raw of the corresponding MP anchor (we add a constant Δ to
it → ribbon row stays parallel to anchor row in UV space, so
each quad has the same V span and texture arcs render at the
same thickness across all 17 columns).
"""
dv = UV_MIDDLE_DV if row == 0 else UV_HAIRLINE_DV
return (anchor_u, anchor_v + dv)
# Face-parsing class indices for the jonathandinu/face-parsing
# SegFormer model (matches CelebAMask-HQ labels):
PARSE_BG = 0
PARSE_SKIN = 1
PARSE_NOSE = 2
PARSE_EYE_G = 3
PARSE_L_EYE = 4
PARSE_R_EYE = 5
PARSE_L_BROW = 6
PARSE_R_BROW = 7
PARSE_L_EAR = 8
PARSE_R_EAR = 9
PARSE_MOUTH = 10
PARSE_U_LIP = 11
PARSE_L_LIP = 12
PARSE_HAIR = 13
PARSE_HAT = 14
PARSE_EAR_R = 15
PARSE_NECK_L = 16
PARSE_NECK = 17
PARSE_CLOTH = 18
HF_FACE_PARSER_MODEL = "jonathandinu/face-parsing"
+115
View File
@@ -0,0 +1,115 @@
"""Main CLI: image -> JSON of 502 3D points consumable by the SDK.
Pipeline:
1. MediaPipe FaceMesh -> 468 normalized landmarks
2. Face parsing (HF SegFormer) -> per-pixel class map
3. Hairline curve detection + per-anchor ray casting -> 17 hairline 2D points
4. Lift 2D hairline points to 3D using anchor Z + curvature offset
5. Interpolate 17 middle-row 3D points
6. Concatenate into (502, 3) and emit JSON
Output JSON schema:
{
"image": {"width": W, "height": H, "path": "..."},
"n_total": 502,
"n_mp": 468,
"n_extension": 34,
"layout": ["mp[0..468)", "middle[468..485)", "hairline[485..502)"],
"points": [[x_norm, y_norm, z_relative], ...] # length 502
"valid_hairline": [true/false, ...] # length 17
}
Usage:
py -3 python/extract_hairline.py path/to/image.jpg
py -3 python/extract_hairline.py path/to/image.jpg --out data/out.json
"""
from __future__ import annotations
import argparse
import json
import os
import sys
import time
import numpy as np
if __package__ is None or __package__ == "":
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from python import constants as C
from python.face_landmarks import FaceLandmarker
from python.face_parsing import FaceParser
from python.hairline_2d import sample_hairline, smooth_hairline
from python.lift_3d import lift_hairline_to_3d, build_middle_row, assemble_full
else:
from . import constants as C
from .face_landmarks import FaceLandmarker
from .face_parsing import FaceParser
from .hairline_2d import sample_hairline, smooth_hairline
from .lift_3d import lift_hairline_to_3d, build_middle_row, assemble_full
def run(image_path: str, out_path: str | None = None, device: str | None = None) -> dict:
import cv2
bgr = cv2.imread(image_path)
if bgr is None:
raise FileNotFoundError(f"could not read image: {image_path}")
rgb = cv2.cvtColor(bgr, cv2.COLOR_BGR2RGB)
H, W = rgb.shape[:2]
t0 = time.perf_counter()
lmk = FaceLandmarker(static_image_mode=True)
landmarks = lmk.detect(rgb)
lmk.close()
if landmarks is None:
raise RuntimeError("no face detected")
t1 = time.perf_counter()
print(f" [time] mediapipe: {(t1 - t0)*1000:.0f} ms")
parser = FaceParser(device=device)
parse_map = parser.parse(rgb)
t2 = time.perf_counter()
print(f" [time] parsing: {(t2 - t1)*1000:.0f} ms")
hairline_2d, valid = sample_hairline(landmarks, parse_map)
hairline_2d = smooth_hairline(hairline_2d, valid)
hairline_3d = lift_hairline_to_3d(landmarks, hairline_2d)
middle_3d = build_middle_row(landmarks, hairline_3d)
points_full = assemble_full(landmarks, middle_3d, hairline_3d)
t3 = time.perf_counter()
print(f" [time] hairline: {(t3 - t2)*1000:.0f} ms")
record = {
"image": {"width": int(W), "height": int(H), "path": image_path},
"n_total": C.N_TOTAL,
"n_mp": C.N_MP,
"n_extension": C.N_EXT,
"layout": [
f"mp[0..{C.N_MP})",
f"middle[{C.MIDDLE_START}..{C.HAIRLINE_START})",
f"hairline[{C.HAIRLINE_START}..{C.N_TOTAL})",
],
"points": points_full.tolist(),
"valid_hairline": valid.tolist(),
}
if out_path is None:
base = os.path.splitext(os.path.basename(image_path))[0]
out_path = os.path.join("data", base + ".json")
os.makedirs(os.path.dirname(out_path) or ".", exist_ok=True)
with open(out_path, "w", encoding="utf-8") as f:
json.dump(record, f, ensure_ascii=False, indent=2)
print(f" wrote {out_path}")
return record
def main() -> None:
ap = argparse.ArgumentParser(description="Extract MediaPipe + hairline points from one image.")
ap.add_argument("image", help="path to input image (jpg/png)")
ap.add_argument("--out", default=None, help="output JSON path")
ap.add_argument("--device", default=None, help="torch device (cpu/cuda); auto if omitted")
args = ap.parse_args()
run(args.image, args.out, args.device)
if __name__ == "__main__":
main()
+103
View File
@@ -0,0 +1,103 @@
"""MediaPipe FaceMesh wrappers returning 468 3D landmarks in [0,1] x/y space.
`FaceLandmarker` uses the modern mediapipe.tasks.vision API and requires
models/face_landmarker.task.
`SolutionsFaceLandmarker` uses the older mediapipe.solutions.face_mesh API.
It is useful as a CPU fallback in WSL environments where the Tasks API may
segfault while initializing EGL/OpenGL.
"""
from __future__ import annotations
import os
import numpy as np
DEFAULT_MODEL_PATH = os.path.join(
os.path.dirname(os.path.dirname(os.path.abspath(__file__))),
"models", "face_landmarker.task",
)
class FaceLandmarker:
def __init__(self, static_image_mode: bool = True, model_path: str | None = None):
import mediapipe as mp
from mediapipe.tasks import python as mp_python
from mediapipe.tasks.python import vision
path = model_path or DEFAULT_MODEL_PATH
if not os.path.isfile(path):
raise FileNotFoundError(
f"face_landmarker.task not found at {path}. "
"Download it from "
"https://storage.googleapis.com/mediapipe-models/face_landmarker/"
"face_landmarker/float16/1/face_landmarker.task"
)
running_mode = vision.RunningMode.IMAGE if static_image_mode else vision.RunningMode.VIDEO
options = vision.FaceLandmarkerOptions(
base_options=mp_python.BaseOptions(model_asset_path=path),
running_mode=running_mode,
num_faces=1,
output_face_blendshapes=False,
output_facial_transformation_matrixes=False,
)
self._detector = vision.FaceLandmarker.create_from_options(options)
self._mp = mp
def detect(self, image_rgb: np.ndarray) -> np.ndarray | None:
"""Returns (468, 3) float32 of normalized x, y and relative z, or None.
The task model produces 478 landmarks (468 face + 10 iris); we return
only the first 468 to match the canonical FaceMesh topology used by
the SDK's OBJ file.
"""
mp_image = self._mp.Image(image_format=self._mp.ImageFormat.SRGB, data=image_rgb)
result = self._detector.detect(mp_image)
if not result.face_landmarks:
return None
lm = result.face_landmarks[0][:468]
arr = np.array([[p.x, p.y, p.z] for p in lm], dtype=np.float32)
return arr
def close(self):
try:
self._detector.close()
except Exception:
pass
class SolutionsFaceLandmarker:
"""CPU-oriented fallback using mediapipe.solutions.face_mesh."""
def __init__(self, static_image_mode: bool = True):
import mediapipe as mp
try:
face_mesh_module = mp.solutions.face_mesh
except AttributeError as exc:
raise RuntimeError(
"当前 mediapipe 包不包含 mediapipe.solutions.face_mesh。"
"请使用 web_service.py 的默认 parsing backend,或安装包含 solutions API 的 mediapipe 版本。"
) from exc
self._face_mesh = face_mesh_module.FaceMesh(
static_image_mode=static_image_mode,
max_num_faces=1,
refine_landmarks=False,
min_detection_confidence=0.5,
)
def detect(self, image_rgb: np.ndarray) -> np.ndarray | None:
"""Returns (468, 3) float32 of normalized x, y and relative z, or None."""
image_rgb.flags.writeable = False
result = self._face_mesh.process(image_rgb)
image_rgb.flags.writeable = True
if not result.multi_face_landmarks:
return None
lm = result.multi_face_landmarks[0].landmark[:468]
return np.array([[p.x, p.y, p.z] for p in lm], dtype=np.float32)
def close(self):
try:
self._face_mesh.close()
except Exception:
pass
+42
View File
@@ -0,0 +1,42 @@
"""Face parsing via HuggingFace SegFormer (jonathandinu/face-parsing).
First call downloads ~150 MB of weights into the HF cache.
"""
from __future__ import annotations
import numpy as np
from typing import TYPE_CHECKING
from . import constants as C
if TYPE_CHECKING: # avoid hard import at module load
pass
class FaceParser:
"""Wraps a SegFormer face-parsing model and returns a (H, W) int label map."""
def __init__(self, device: str | None = None):
import torch
from transformers import SegformerImageProcessor, SegformerForSemanticSegmentation
self.device = device or ("cuda" if torch.cuda.is_available() else "cpu")
self.processor = SegformerImageProcessor.from_pretrained(C.HF_FACE_PARSER_MODEL)
self.model = SegformerForSemanticSegmentation.from_pretrained(C.HF_FACE_PARSER_MODEL)
self.model.to(self.device).eval()
self._torch = torch
def parse(self, image_rgb: np.ndarray) -> np.ndarray:
"""image_rgb: (H, W, 3) uint8. Returns (H, W) int64 of class indices."""
from PIL import Image
H, W = image_rgb.shape[:2]
pil = Image.fromarray(image_rgb)
inputs = self.processor(images=pil, return_tensors="pt").to(self.device)
with self._torch.no_grad():
logits = self.model(**inputs).logits # (1, C, h, w)
# Upsample to original resolution
up = self._torch.nn.functional.interpolate(
logits, size=(H, W), mode="bilinear", align_corners=False
)
labels = up.argmax(dim=1).squeeze(0).to("cpu").numpy().astype(np.int32)
return labels
File diff suppressed because it is too large Load Diff
+133
View File
@@ -0,0 +1,133 @@
"""Lift 2D hairline samples to 3D in MediaPipe's normalized space.
Saggital-arc Z model
--------------------
MediaPipe / face.obj use ``-z = front of face`` (nose tip is the most
negative z), ``+z = back of head``. The frontal head surface curves
backward as you move up from the forehead to the crown, so any vertex
**above** an MP top anchor (smaller y in image coords) must have a
**larger** z than that anchor.
For each new hairline / middle vertex (x_h, y_h) attached to MP anchor a
located at (x_a, y_a, z_a), we model the local sagittal cross-section
of the head as a circular arc of radius ``R = HEAD_ARC_RADIUS_FRAC × face_h``
and derive
z_new = z_a + (y_h - y_a)² / (2 R)
The squared dy term guarantees ``z_new ≥ z_a`` (the surface always
bulges backward as you walk up the head, never forward). x is taken
directly from the 2D hairline detection (we don't project x onto the
arc — the parsing tells us exactly where the hairline sits in x).
"""
from __future__ import annotations
import numpy as np
from . import constants as C
from .hairline_2d import face_up_vector
def _face_height(landmarks_norm: np.ndarray) -> float:
"""Range of MP y for the visible face (used for the arc radius)."""
y = landmarks_norm[:, 1]
return float(y.max() - y.min())
def _arc_dz(dy: float, R: float) -> float:
"""Backward (positive) z offset along a circular arc of radius R for
a vertical displacement dy. Always non-negative."""
return (dy * dy) / (2.0 * max(R, 1e-6))
def lift_hairline_to_3d(
landmarks_norm: np.ndarray,
hairline_norm_xy: np.ndarray,
crown_lift_frac: float | None = None,
) -> np.ndarray:
"""Combine 2D hairline samples with a sagittal-arc Z, after lifting
the (x, y) along the face-up direction by ``crown_lift_frac × face_h``
so the ribbon's top row sits at the crown of the head rather than at
the detected hair-skin boundary.
landmarks_norm: (468, 3) MediaPipe normalized landmarks.
hairline_norm_xy: (N_ANCHORS, 2) 2D hairline samples in [0,1] image space.
crown_lift_frac: how far above the detected hairline (in fractions of
face height) the mesh row should sit. ``None`` uses
``C.HAIRLINE_CROWN_LIFT_FRAC``.
Returns: (N_ANCHORS, 3) — (x_norm_lifted, y_norm_lifted, z).
"""
face_h = _face_height(landmarks_norm)
R = C.HEAD_ARC_RADIUS_FRAC * face_h
if crown_lift_frac is None:
crown_lift_frac = C.HAIRLINE_CROWN_LIFT_FRAC
up = face_up_vector(landmarks_norm)
lift = up * (crown_lift_frac * face_h) # 2D offset, face-up direction
out = np.zeros((C.N_ANCHORS, 3), dtype=np.float32)
for i, mp_idx in enumerate(C.MP_TOP_ANCHORS):
z_a = float(landmarks_norm[mp_idx, 2])
y_a = float(landmarks_norm[mp_idx, 1])
x_h = float(hairline_norm_xy[i, 0]) + float(lift[0])
y_h = float(hairline_norm_xy[i, 1]) + float(lift[1])
dy = y_h - y_a # < 0 (上移更多 → dz 更大)
out[i, 0] = x_h
out[i, 1] = y_h
out[i, 2] = z_a + _arc_dz(dy, R)
return out
def build_middle_row(
landmarks_norm: np.ndarray,
hairline_3d: np.ndarray,
bias: float = 0.5,
) -> np.ndarray:
"""Place the middle row at the arc-length midpoint between each MP
anchor and its hairline point.
The old approach used bias=0.5 linear interpolation in XY and then
independently recomputed Z via the arc formula. Because dz ∝ dy²,
the middle row only received 25 % of the hairline Z offset, creating
a visible dent where the extension met the face mesh.
New approach: for each anchor→hairline pair, parameterise the
sagittal circular arc by the angle θ and place the middle vertex at
θ_mid = bias × θ_hair. This distributes both the Y displacement
**and** the Z displacement smoothly along the arc, so the surface
transitions from the face mesh through middle to hairline without
any concavity.
"""
R = C.HEAD_ARC_RADIUS_FRAC * _face_height(landmarks_norm)
out = np.zeros((C.N_ANCHORS, 3), dtype=np.float32)
for i, mp_idx in enumerate(C.MP_TOP_ANCHORS):
a = landmarks_norm[mp_idx]
h = hairline_3d[i]
# X: simple linear interpolation (no arc model in the coronal plane)
out[i, 0] = (1 - bias) * float(a[0]) + bias * float(h[0])
# Y, Z: interpolate along the circular arc in the sagittal plane.
# The arc angle for the hairline point:
# θ_h = |dy_h| / R (small-angle: arc-length ≈ R θ)
# Middle sits at θ_m = bias × θ_h, giving:
# dy_m = R sin(θ_m) ≈ R θ_m for small θ
# dz_m = R (1 cos(θ_m))
# For the parabolic regime (θ < 0.8 rad ≈ 46°) these simplify to
# the same formula but we use the full trig for correctness.
dy_h = float(h[1]) - float(a[1]) # negative (upward)
sign = -1.0 if dy_h < 0 else 1.0
theta_h = abs(dy_h) / max(R, 1e-9)
theta_m = bias * theta_h
dy_m = sign * R * np.sin(theta_m) # same sign as dy_h
dz_m = R * (1.0 - np.cos(theta_m)) # always ≥ 0
out[i, 1] = float(a[1]) + dy_m
out[i, 2] = float(a[2]) + dz_m
return out
def assemble_full(landmarks_norm: np.ndarray,
middle_3d: np.ndarray,
hairline_3d: np.ndarray) -> np.ndarray:
"""Concatenate to a (N_TOTAL, 3) array in the SDK's expected order:
[0..468) MediaPipe / [468..485) middle / [485..502) hairline."""
assert landmarks_norm.shape == (C.N_MP, 3)
assert middle_3d.shape == (C.N_ANCHORS, 3)
assert hairline_3d.shape == (C.N_ANCHORS, 3)
return np.concatenate([landmarks_norm, middle_3d, hairline_3d], axis=0).astype(np.float32)
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+111
View File
@@ -0,0 +1,111 @@
{
"_name_or_path": "jonathandinu/face-parsing",
"architectures": [
"SegformerForSemanticSegmentation"
],
"attention_probs_dropout_prob": 0.0,
"classifier_dropout_prob": 0.1,
"decoder_hidden_size": 768,
"depths": [
3,
6,
40,
3
],
"downsampling_rates": [
1,
4,
8,
16
],
"drop_path_rate": 0.1,
"hidden_act": "gelu",
"hidden_dropout_prob": 0.0,
"hidden_sizes": [
64,
128,
320,
512
],
"id2label": {
"0": "background",
"1": "skin",
"2": "nose",
"3": "eye_g",
"4": "l_eye",
"5": "r_eye",
"6": "l_brow",
"7": "r_brow",
"8": "l_ear",
"9": "r_ear",
"10": "mouth",
"11": "u_lip",
"12": "l_lip",
"13": "hair",
"14": "hat",
"15": "ear_r",
"16": "neck_l",
"17": "neck",
"18": "cloth"
},
"image_size": 224,
"initializer_range": 0.02,
"label2id": {
"background": 0,
"skin": 1,
"nose": 2,
"eye_g": 3,
"l_eye": 4,
"r_eye": 5,
"l_brow": 6,
"r_brow": 7,
"l_ear": 8,
"r_ear": 9,
"mouth": 10,
"u_lip": 11,
"l_lip": 12,
"hair": 13,
"hat": 14,
"ear_r": 15,
"neck_l": 16,
"neck": 17,
"cloth": 18
},
"layer_norm_eps": 1e-06,
"mlp_ratios": [
4,
4,
4,
4
],
"model_type": "segformer",
"num_attention_heads": [
1,
2,
5,
8
],
"num_channels": 3,
"num_encoder_blocks": 4,
"patch_sizes": [
7,
3,
3,
3
],
"reshape_last_stage": true,
"semantic_loss_ignore_index": 255,
"sr_ratios": [
8,
4,
2,
1
],
"strides": [
4,
2,
2,
2
],
"transformers_version": "4.37.0.dev0"
}
@@ -0,0 +1,23 @@
{
"do_normalize": true,
"do_reduce_labels": false,
"do_rescale": true,
"do_resize": true,
"image_mean": [
0.485,
0.456,
0.406
],
"image_processor_type": "SegformerFeatureExtractor",
"image_std": [
0.229,
0.224,
0.225
],
"resample": 2,
"rescale_factor": 0.00392156862745098,
"size": {
"height": 512,
"width": 512
}
}
Binary file not shown.
+97
View File
@@ -0,0 +1,97 @@
"""Minimal Wavefront OBJ reader/writer.
Tailored to the project's face.obj:
- Latin-1 / GB-encoded comments allowed (we tolerate undecodable bytes).
- Vertices written with 4 decimal places (matches existing file).
- Preserves comment/mtllib/group lines if requested.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Sequence
@dataclass
class ObjMesh:
"""1-indexed in OBJ files; stored 0-indexed internally."""
positions: list[tuple[float, float, float]] = field(default_factory=list)
texcoords: list[tuple[float, float]] = field(default_factory=list)
normals: list[tuple[float, float, float]] = field(default_factory=list)
# Each face is a list of 3 (pos_idx, uv_idx, normal_idx) tuples, 0-indexed.
# -1 means "absent".
faces: list[list[tuple[int, int, int]]] = field(default_factory=list)
header_lines: list[str] = field(default_factory=list) # comments / mtllib
def n_v(self) -> int: return len(self.positions)
def n_vt(self) -> int: return len(self.texcoords)
def n_vn(self) -> int: return len(self.normals)
def n_f(self) -> int: return len(self.faces)
def read_obj(path: str) -> ObjMesh:
"""Read an OBJ file, ignoring undecodable bytes in comments."""
with open(path, "rb") as f:
raw = f.read()
text = raw.decode("latin-1", errors="replace")
mesh = ObjMesh()
for line in text.splitlines():
line = line.strip()
if not line:
continue
if line.startswith("#") or line.startswith("mtllib") or line.startswith("o ") or line.startswith("g ") or line.startswith("s "):
mesh.header_lines.append(line)
continue
parts = line.split()
kind = parts[0]
if kind == "v":
mesh.positions.append((float(parts[1]), float(parts[2]), float(parts[3])))
elif kind == "vt":
mesh.texcoords.append((float(parts[1]), float(parts[2])))
elif kind == "vn":
mesh.normals.append((float(parts[1]), float(parts[2]), float(parts[3])))
elif kind == "f":
verts = []
for spec in parts[1:]:
seg = spec.split("/")
pi = int(seg[0]) - 1 if seg[0] else -1
ti = int(seg[1]) - 1 if len(seg) > 1 and seg[1] else -1
ni = int(seg[2]) - 1 if len(seg) > 2 and seg[2] else -1
verts.append((pi, ti, ni))
# Triangulate fan if quad/n-gon (shouldn't happen for our mesh)
for k in range(1, len(verts) - 1):
mesh.faces.append([verts[0], verts[k], verts[k + 1]])
# ignore everything else
return mesh
def write_obj(path: str, mesh: ObjMesh, header: Sequence[str] | None = None) -> None:
"""Write an OBJ file with the project's conventions (1-indexed, 4dp)."""
out: list[str] = []
if header is not None:
out.extend(header)
else:
out.append("# head3d extended face mesh")
out.append(f"# vertices={mesh.n_v()} texcoords={mesh.n_vt()} normals={mesh.n_vn()} faces={mesh.n_f()}")
out.append("")
for x, y, z in mesh.positions:
out.append(f"v {x:.4f} {y:.4f} {z:.4f}")
out.append("")
for u, v in mesh.texcoords:
out.append(f"vt {u:.4f} {v:.4f} 0.0000")
out.append("")
for nx, ny, nz in mesh.normals:
out.append(f"vn {nx:.4f} {ny:.4f} {nz:.4f}")
out.append("")
for face in mesh.faces:
toks = []
for pi, ti, ni in face:
a = str(pi + 1)
b = str(ti + 1) if ti >= 0 else ""
c = str(ni + 1) if ni >= 0 else ""
toks.append(f"{a}/{b}/{c}")
out.append("f " + " ".join(toks))
with open(path, "w", encoding="utf-8", newline="\r\n") as f:
f.write("\n".join(out))
f.write("\n")
Binary file not shown.

After

Width:  |  Height:  |  Size: 28 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 272 KiB