xsl/meida_backend
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
7bcbb8d40bc3b5dca4275b2847073a262558d413
meida_backend/tests/golden/cases/rules_cases.json
T
xsl 67d9754823 test(golden): data-driven golden set framework wired into CI 
tests/golden：JSON 用例 + 阈值配置 + runner（规则 pass_rate=1.0、检索 0.8
起步）+ pytest 壳；规则/检索两子集种子，Agent/报告子集占位（T-P2-11）。

实现：
- runner.py 为普通模块（不依赖 pytest）：加载/执行/指标（pass_rate、
  first_hit_in_top3）/逐用例差异明细；test_golden.py 只是薄壳（marker=golden，
  无 MEIDA_RUN_INTEGRATION 时整体 skip）。用例与 expect 的未知键一律判
  fail（封闭词汇表，防断言静默失效）。
- 规则子集 13 例（rules_cases.json，走 packages.rules DSL/evaluate）：
  含 3 个硬禁忌场景（无条件合规硬拦、正式场合硬拦、复合优先级链
  100硬>95禁忌>90体型>50趋势）+ 禁忌压制/维度隔离/多规则叠加/温度边界/
  缺字段静默/include 子集语义等。
- 检索子集 13 例（retrieval_cases.json，DeterministicEmbedder+hybrid_search）：
  含 7 个 must_miss 红线场景（禁忌体型/draft/archived/场景外/类型外/组合红线）。
  种子实体以 golden: 前缀命名，跑前先删（幂等重跑）；跑后也删——CI 防线
  5c 与集成测试共用长活库，留下假实体会破坏其它用例对结果列表的精确断言。
  确定性策略：query=目标 chunk 原文（余弦距离 0 必排第 1）+ 每例独立
  marker token（gqNN）隔离关键词路；检索参数（1024 维/top_k 50/默认 RRF
  权重）钉死在 runner，不读 env。
- thresholds.json：规则 pass_rate_min=1.0（硬结论子集一条不许退）；
  检索 pass_rate_min=0.8 + first_hit_in_top3_min=0.8 起步。
- README.md：四子集 330 例规划、两格式 spec、加用例流程、阈值口径与
  “为什么规则必须 1.0”。

测试：
- MEIDA_RUN_INTEGRATION=1 pytest -m golden --no-cov：2 passed（规则
  13/13=1.0，检索 13/13=1.0、first_hit_in_top3=13/13）；连续重跑幂等，
  跑后库内 golden:% 残留为 0；不设 env 时 2 skipped；基线单测 473 passed、
  集成 38 passed 不受影响；ruff check/format、lint-imports 全绿。
- 阻断语义自证（已改回，提交里只有正确版本）：临时把规则用例
  user-taboo-overrides-trend-recommendation 的 must_avoid 改成不存在的
  取值并把 hard_blocks_nonempty 翻转为 true → 规则子集 12/13=0.9231
  <1.0 fail，明细输出“must_avoid 未命中：期望 top[length=九分裤脚]，实际
  avoid=[top[length=[短款, 露脐]]]”与“hard_blocks_nonempty 期望 True，
  实际 False（硬拦截规则：无）”；再临时破坏 3 条检索用例（2 条 must_hit
  换成不存在的实体名、1 条 must_miss 加入本应命中的实体）→ 检索子集
  10/13=0.7692<0.8 fail，明细含“must_miss 红线违规：golden:r12-good
  出现在第 1 位——硬过滤零漏被破坏”。验证完三条全部改回，复跑全绿。
  （注：检索阈值 0.8 起步，单条退化 0.92 尚不拦截、≥3 条即拦截，符合
  种子期口径；规则阈值 1.0 单条即拦截。）

偏差说明：无
2026-08-14 21:19:57 +08:00

515 lines
16 KiB
JSON
Raw Blame History

[
{
"name": "unconditional-hard-compliance-blocks-exotic-leather",
"rules": [
{
"rule_key": "G-RULES-01-HARD-EXOTIC",
"category": "compliance",
"priority": 100,
"confidence": 1.0,
"is_hard": true,
"conditions": {},
"effects": {
"avoid": [
{"slot": "accessory", "attrs": {"material": ["真兽皮"]}},
{"slot": "shoes", "attrs": {"material": ["真皮"]}}
],
"explanation": "合规红线：任何场景都不推荐真兽皮材质。"
}
}
],
"context": {},
"expect": {
"must_avoid": [
{"slot": "accessory", "attrs_include": {"material": ["真兽皮"]}},
{"slot": "shoes", "attrs_include": {"material": ["真皮"]}}
],
"forbidden_recommend": [
{"slot": "accessory", "attrs_include": {"material": ["真兽皮"]}}
],
"hard_blocks_nonempty": true
}
},
{
"name": "hard-compliance-fires-on-formal-occasion",
"rules": [
{
"rule_key": "G-RULES-02-HARD-FORMAL",
"category": "compliance",
"priority": 100,
"confidence": 1.0,
"is_hard": true,
"conditions": {"occasion": {"include": ["红毯", "晚宴"]}},
"effects": {
"avoid": [
{"slot": "shoes", "attrs": {"material": ["真皮"]}},
{"slot": "accessory", "attrs": {"material": ["真兽皮"]}}
],
"explanation": "正式场合合规红线：不推荐真皮与真兽皮。"
}
}
],
"context": {"occasion": "晚宴"},
"expect": {
"must_avoid": [
{"slot": "shoes", "attrs_include": {"material": ["真皮"]}}
],
"forbidden_recommend": [
{"slot": "shoes", "attrs_include": {"material": ["真皮"]}}
],
"hard_blocks_nonempty": true
}
},
{
"name": "hard-compliance-silent-on-daily-occasion",
"rules": [
{
"rule_key": "G-RULES-03-HARD-FORMAL",
"category": "compliance",
"priority": 100,
"confidence": 1.0,
"is_hard": true,
"conditions": {"occasion": {"include": ["红毯", "晚宴"]}},
"effects": {
"avoid": [{"slot": "shoes", "attrs": {"material": ["真皮"]}}],
"explanation": "正式场合合规红线：不推荐真皮。"
}
}
],
"context": {"occasion": "通勤"},
"expect": {
"forbidden_recommend": [
{"slot": "shoes", "attrs_include": {"material": ["真皮"]}}
],
"hard_blocks_nonempty": false
}
},
{
"name": "user-taboo-overrides-trend-recommendation",
"rules": [
{
"rule_key": "G-RULES-04-TREND-CROP",
"category": "trend",
"priority": 50,
"confidence": 0.7,
"conditions": {"style": "甜酷"},
"effects": {
"recommend": [{"slot": "top", "attrs": {"length": ["短款"]}}],
"explanation": "短款上衣是当季甜酷风的核心单品。"
}
},
{
"rule_key": "G-RULES-05-TABOO-MIDRIFF",
"category": "user_taboo",
"priority": 95,
"confidence": 1.0,
"conditions": {"user_taboos": {"include": ["露脐"]}},
"effects": {
"avoid": [{"slot": "top", "attrs": {"length": ["短款", "露脐"]}}],
"explanation": "用户不接受露脐，已剔除相关单品。"
}
}
],
"context": {"style": "甜酷", "user_taboos": ["露脐"]},
"expect": {
"must_avoid": [
{"slot": "top", "attrs_include": {"length": ["短款", "露脐"]}}
],
"forbidden_recommend": [
{"slot": "top", "attrs_include": {"length": ["短款"]}}
],
"hard_blocks_nonempty": false
}
},
{
"name": "user-taboo-limited-to-conflicting-dimension",
"rules": [
{
"rule_key": "G-RULES-06-BODY-H-BOTTOM",
"category": "body_shape",
"priority": 90,
"confidence": 0.9,
"conditions": {"body_shape": "H型"},
"effects": {
"recommend": [{"slot": "bottom", "attrs": {"silhouette": ["直筒", "阔腿"]}}],
"avoid": [{"slot": "bottom", "attrs": {"silhouette": ["紧身"]}}],
"explanation": "H型宜直线轮廓放大腰线比例。"
}
},
{
"rule_key": "G-RULES-07-TABOO-MIDRIFF",
"category": "user_taboo",
"priority": 95,
"confidence": 1.0,
"conditions": {"user_taboos": {"include": ["露脐"]}},
"effects": {
"avoid": [{"slot": "top", "attrs": {"length": ["短款", "露脐"]}}],
"explanation": "用户不接受露脐，已剔除相关单品。"
}
}
],
"context": {"body_shape": "H型", "user_taboos": ["露脐"]},
"expect": {
"must_recommend": [
{"slot": "bottom", "attrs_include": {"silhouette": ["阔腿"]}}
],
"must_avoid": [
{"slot": "bottom", "attrs_include": {"silhouette": ["紧身"]}}
],
"forbidden_recommend": [
{"slot": "top", "attrs_include": {"length": ["短款"]}}
],
"hard_blocks_nonempty": false
}
},
{
"name": "body-temperature-occasion-stack",
"rules": [
{
"rule_key": "G-RULES-08-BODY-A-SHOULDER",
"category": "body_shape",
"priority": 90,
"confidence": 0.95,
"conditions": {
"body_shape": "A型",
"features": {"include": ["溜肩"]},
"goals": {"include": ["修饰溜肩"]}
},
"effects": {
"recommend": [
{"slot": "top", "attrs": {"shoulder": ["垫肩"], "sleeve": ["泡泡袖"]}}
],
"avoid": [{"slot": "top", "attrs": {"silhouette": ["修身"]}}],
"explanation": "溜肩需要用垫肩与泡泡袖在肩部建立结构感。"
}
},
{
"rule_key": "G-RULES-09-TEMP-18-26",
"category": "temperature",
"priority": 75,
"confidence": 0.85,
"conditions": {"temperature": {"min": 18, "max": 26}},
"effects": {
"recommend": [{"slot": "outer", "attrs": {"thickness": ["薄款"]}}],
"avoid": [{"slot": "outer", "attrs": {"thickness": ["厚款"]}}],
"explanation": "18-26℃ 单层薄外套最舒适。"
}
},
{
"rule_key": "G-RULES-10-OCC-INTERVIEW",
"category": "occasion",
"priority": 80,
"confidence": 0.9,
"conditions": {"occasion": "面试"},
"effects": {
"recommend": [
{"slot": "top", "attrs": {"collar": ["衬衫领"]}},
{"slot": "bottom", "attrs": {"silhouette": ["直筒"]}}
],
"avoid": [{"slot": "top", "attrs": {"neckline": ["深V"]}}],
"explanation": "面试场合需要干练可信的轮廓。"
}
}
],
"context": {
"body_shape": "A型",
"features": ["溜肩"],
"goals": ["修饰溜肩"],
"temperature": 22.0,
"occasion": "面试"
},
"expect": {
"must_recommend": [
{"slot": "top", "attrs_include": {"shoulder": ["垫肩"]}},
{"slot": "top", "attrs_include": {"sleeve": ["泡泡袖"]}},
{"slot": "top", "attrs_include": {"collar": ["衬衫领"]}},
{"slot": "bottom", "attrs_include": {"silhouette": ["直筒"]}},
{"slot": "outer", "attrs_include": {"thickness": ["薄款"]}}
],
"must_avoid": [
{"slot": "top", "attrs_include": {"silhouette": ["修身"]}},
{"slot": "top", "attrs_include": {"neckline": ["深V"]}},
{"slot": "outer", "attrs_include": {"thickness": ["厚款"]}}
],
"hard_blocks_nonempty": false
}
},
{
"name": "missing-profile-fields-keep-rules-silent",
"rules": [
{
"rule_key": "G-RULES-11-BODY-A-SHOULDER",
"category": "body_shape",
"priority": 90,
"confidence": 0.95,
"conditions": {
"body_shape": "A型",
"features": {"include": ["溜肩"]},
"goals": {"include": ["修饰溜肩"]}
},
"effects": {
"recommend": [
{"slot": "top", "attrs": {"shoulder": ["垫肩"], "sleeve": ["泡泡袖"]}}
],
"avoid": [{"slot": "top", "attrs": {"silhouette": ["修身"]}}],
"explanation": "溜肩需要用垫肩与泡泡袖在肩部建立结构感。"
}
},
{
"rule_key": "G-RULES-12-TEMP-18-26",
"category": "temperature",
"priority": 75,
"confidence": 0.85,
"conditions": {"temperature": {"min": 18, "max": 26}},
"effects": {
"recommend": [{"slot": "outer", "attrs": {"thickness": ["薄款"]}}],
"avoid": [{"slot": "outer", "attrs": {"thickness": ["厚款"]}}],
"explanation": "18-26℃ 单层薄外套最舒适。"
}
}
],
"context": {"features": ["溜肩"], "goals": ["修饰溜肩"]},
"expect": {
"forbidden_recommend": [
{"slot": "top", "attrs_include": {}},
{"slot": "outer", "attrs_include": {}}
],
"hard_blocks_nonempty": false
}
},
{
"name": "temperature-lower-bound-inclusive",
"rules": [
{
"rule_key": "G-RULES-13-TEMP-18-26",
"category": "temperature",
"priority": 75,
"confidence": 0.85,
"conditions": {"temperature": {"min": 18, "max": 26}},
"effects": {
"recommend": [{"slot": "outer", "attrs": {"thickness": ["薄款"]}}],
"avoid": [{"slot": "outer", "attrs": {"thickness": ["厚款"]}}],
"explanation": "18-26℃ 单层薄外套最舒适。"
}
}
],
"context": {"temperature": 18.0},
"expect": {
"must_recommend": [
{"slot": "outer", "attrs_include": {"thickness": ["薄款"]}}
],
"must_avoid": [
{"slot": "outer", "attrs_include": {"thickness": ["厚款"]}}
],
"hard_blocks_nonempty": false
}
},
{
"name": "temperature-below-range-stays-silent",
"rules": [
{
"rule_key": "G-RULES-14-TEMP-18-26",
"category": "temperature",
"priority": 75,
"confidence": 0.85,
"conditions": {"temperature": {"min": 18, "max": 26}},
"effects": {
"recommend": [{"slot": "outer", "attrs": {"thickness": ["薄款"]}}],
"avoid": [{"slot": "outer", "attrs": {"thickness": ["厚款"]}}],
"explanation": "18-26℃ 单层薄外套最舒适。"
}
}
],
"context": {"temperature": 17.5},
"expect": {
"forbidden_recommend": [
{"slot": "outer", "attrs_include": {}}
],
"hard_blocks_nonempty": false
}
},
{
"name": "interview-occasion-collar-and-neckline",
"rules": [
{
"rule_key": "G-RULES-15-OCC-INTERVIEW",
"category": "occasion",
"priority": 80,
"confidence": 0.9,
"conditions": {"occasion": "面试"},
"effects": {
"recommend": [
{"slot": "top", "attrs": {"collar": ["衬衫领"]}},
{"slot": "bottom", "attrs": {"silhouette": ["直筒"]}}
],
"avoid": [{"slot": "top", "attrs": {"neckline": ["深V"]}}],
"explanation": "面试场合需要干练可信的轮廓。"
}
}
],
"context": {"occasion": "面试"},
"expect": {
"must_recommend": [
{"slot": "top", "attrs_include": {"collar": ["衬衫领"]}},
{"slot": "bottom", "attrs_include": {"silhouette": ["直筒"]}}
],
"must_avoid": [
{"slot": "top", "attrs_include": {"neckline": ["深V"]}}
],
"hard_blocks_nonempty": false
}
},
{
"name": "composite-priority-chain-hard-taboo-body",
"rules": [
{
"rule_key": "G-RULES-16-HARD-FORMAL",
"category": "compliance",
"priority": 100,
"confidence": 1.0,
"is_hard": true,
"conditions": {"occasion": {"include": ["红毯", "晚宴"]}},
"effects": {
"avoid": [
{"slot": "accessory", "attrs": {"material": ["真兽皮"]}},
{"slot": "shoes", "attrs": {"material": ["真皮"]}}
],
"explanation": "正式场合合规红线：不推荐真兽皮材质。"
}
},
{
"rule_key": "G-RULES-17-TABOO-MIDRIFF",
"category": "user_taboo",
"priority": 95,
"confidence": 1.0,
"conditions": {"user_taboos": {"include": ["露脐"]}},
"effects": {
"avoid": [{"slot": "top", "attrs": {"length": ["短款", "露脐"]}}],
"explanation": "用户不接受露脐，已剔除相关单品。"
}
},
{
"rule_key": "G-RULES-18-BODY-A-SHOULDER",
"category": "body_shape",
"priority": 90,
"confidence": 0.95,
"conditions": {
"body_shape": "A型",
"features": {"include": ["溜肩"]},
"goals": {"include": ["修饰溜肩"]}
},
"effects": {
"recommend": [
{"slot": "top", "attrs": {"shoulder": ["垫肩"], "sleeve": ["泡泡袖"]}}
],
"avoid": [{"slot": "top", "attrs": {"silhouette": ["修身"]}}],
"explanation": "溜肩需要用垫肩与泡泡袖在肩部建立结构感。"
}
},
{
"rule_key": "G-RULES-19-TREND-CROP",
"category": "trend",
"priority": 50,
"confidence": 0.7,
"conditions": {"style": "甜酷"},
"effects": {
"recommend": [{"slot": "top", "attrs": {"length": ["短款"]}}],
"explanation": "短款上衣是当季甜酷风的核心单品。"
}
}
],
"context": {
"body_shape": "A型",
"features": ["溜肩"],
"goals": ["修饰溜肩"],
"style": "甜酷",
"user_taboos": ["露脐"],
"occasion": "红毯"
},
"expect": {
"must_recommend": [
{"slot": "top", "attrs_include": {"shoulder": ["垫肩"]}},
{"slot": "top", "attrs_include": {"sleeve": ["泡泡袖"]}}
],
"must_avoid": [
{"slot": "top", "attrs_include": {"length": ["露脐"]}},
{"slot": "shoes", "attrs_include": {"material": ["真皮"]}}
],
"forbidden_recommend": [
{"slot": "top", "attrs_include": {"length": ["短款"]}},
{"slot": "shoes", "attrs_include": {"material": ["真皮"]}},
{"slot": "accessory", "attrs_include": {"material": ["真兽皮"]}}
],
"hard_blocks_nonempty": true
}
},
{
"name": "features-superset-still-matches-include",
"rules": [
{
"rule_key": "G-RULES-20-BODY-A-SHOULDER",
"category": "body_shape",
"priority": 90,
"confidence": 0.95,
"conditions": {
"body_shape": "A型",
"features": {"include": ["溜肩"]},
"goals": {"include": ["修饰溜肩"]}
},
"effects": {
"recommend": [
{"slot": "top", "attrs": {"shoulder": ["垫肩"], "sleeve": ["泡泡袖"]}}
],
"avoid": [{"slot": "top", "attrs": {"silhouette": ["修身"]}}],
"explanation": "溜肩需要用垫肩与泡泡袖在肩部建立结构感。"
}
}
],
"context": {
"body_shape": "A型",
"features": ["溜肩", "宽胯"],
"goals": ["修饰溜肩"]
},
"expect": {
"must_recommend": [
{"slot": "top", "attrs_include": {"shoulder": ["垫肩"]}}
],
"hard_blocks_nonempty": false
}
},
{
"name": "goals-partial-miss-blocks-rule",
"rules": [
{
"rule_key": "G-RULES-21-BODY-A-SHOULDER",
"category": "body_shape",
"priority": 90,
"confidence": 0.95,
"conditions": {
"body_shape": "A型",
"features": {"include": ["溜肩"]},
"goals": {"include": ["修饰溜肩"]}
},
"effects": {
"recommend": [
{"slot": "top", "attrs": {"shoulder": ["垫肩"], "sleeve": ["泡泡袖"]}}
],
"avoid": [{"slot": "top", "attrs": {"silhouette": ["修身"]}}],
"explanation": "溜肩需要用垫肩与泡泡袖在肩部建立结构感。"
}
}
],
"context": {
"body_shape": "A型",
"features": ["溜肩"],
"goals": ["显瘦"]
},
"expect": {
"forbidden_recommend": [
{"slot": "top", "attrs_include": {}}
],
"hard_blocks_nonempty": false
}
}
]
Reference in New Issue View Git Blame Copy Permalink