feat(data-quality): 规则配置可编辑入口 + 经营口径排除接线(ASK-20260928-05 尾项)
ci/woodpecker/push/main Pipeline failed
ci/woodpecker/push/main Pipeline failed
依据:任总 2026-09-29 08:49「P1和P2批」放行 P1-cma-dq-rule-config-approval-20260923 ①③
(派单件 0-P1-project-20260929-qrc-write-entry-and-demo-exclusion-fullstack)
三处尾项(本单):
1) 前端可编辑入口(补拍板件 ① 未被上轮覆盖的部分)
- frontend/src/views/DataQuality.vue:规则配置 Tab 由「只读」改为
ceo/it 可编辑、finance/business 只读(角色口径与 router/index.ts:45 及
系统设置页一致);新增「操作/编辑」列(平台级项仅 it 可点、演示账套禁用);
编辑走 MyDialog(业务弹窗铁律),必填变更理由(>=10 字),取值区间/枚举
由后端返回字段呈现,越界/非法一律如实回显后端 detail,前端不自造校验通过。
2) scope.excluded_entities 接线 + 出厂默认 [3,4,5,6](拍板件 ③-2「排除」)
- app/dq_rule_config.py:默认由 [] 改为 [3,4,5,6];新增单一事实源
EXCLUDED_ENTITIES_KEY / EXCLUDED_ENTITIES_FIELD、normalize_entity_ids()、
excluded_entities_from_resolution()、excluded_entity_ids()、
is_excluded_from_business_caliber()。
- app/api/data_quality.py:新增 _excluded_clause()(仅全账套口径 eid=0 生效,
COALESCE(col,1) NOT IN (...) 参数化,NULL 归属不吞行)与 _business_excluded();
七条规则的明细查询与 _detail_count 全部叠加排除子句;_plan_id_set /
_pending_alert_plan_links 支持 excluded(播报口径同源)。
- 响应新增 business_caliber{caliber,exclusion_applied,excluded_entities,included}
与 scope_excluded_entities;单账套查询保持展示、仅标注不计入经营口径。
- PUT /rule-config:平台级项改写全账套模板行(entity_id=0),审计 detail 与响应
增 target_entity_id;GET 增 scope_note 与 write_contract{roles,platform_roles,
min_reason_len,entity_config_entities,bot_channel_write,endpoint}。
3) 修复两处实现缺陷(本单测试新增用例的 RED 驱动)
- excluded_entities_from_resolution() 取键与 resolve()["scope"] 的归组形态不一致
(配了 scope. 全名去取已归组后的短名字典)⇒ 恒返回空集,接线静默失效;
改为短名优先、全名兼容(EXCLUDED_ENTITIES_FIELD)。
- upsert_param() 全局项 before 取值多一层 rule_key 查找 ⇒ before 恒 None;
改为全局项直接以 param_key 为键。
4) 测试:tests/test_dq_rule_config_20260928.py 3 处既有断言按新口径更新 +
新增 E 类 8 例(排除默认/单一事实源、全账套与单账套两种口径面、A/B 聚合差异
只来自被排除账套、明细计数同源、平台级项改名单即时生效、平台级写落模板行)。
不改:判定表达式本体 / 级别归属 / 评分函数 / 规则集(DAMA §4 锁死 10 类)、
unit_check.amount_threshold(保持 10000)、virtual_pollution 关键词表与来源白名单。
边界:本提交只含 4 文件(backend/app/dq_rule_config.py、
backend/app/api/data_quality.py、backend/tests/test_dq_rule_config_20260928.py、
frontend/src/views/DataQuality.vue);未裹挟他人在制件
(audit_log.py / migrate_subjects_entity_id_20260923.py / package.json)。
证据:pending-tasks/_evidence/ask5-tail-20260929/
(pytest 全量 1 failed/1516 passed + 失败项归因 A/B、vue-tsc rc=0、
ab-exclusion-pre-write.json 只读 A/B 原始输出、gate dry-run 等)
This commit is contained in:
+189
-42
@@ -557,6 +557,44 @@ def _entity_clause(eid: int, col: str) -> Tuple[str, dict]:
|
||||
return "", {}
|
||||
|
||||
|
||||
def _excluded_clause(eid: int, col: str, excluded) -> Tuple[str, dict]:
|
||||
"""经营口径排除子句(F6 / ASK-05 ③-2 接线,2026-09-29)。
|
||||
|
||||
语义(DAMA 清单 §6.4 + 拍板件 ③-2「排除」):
|
||||
* **仅全账套口径(`eid=0` = 经营口径评分 / 明细聚合)生效** —— 被排除账套(演示/测试)
|
||||
的行不再计入经营口径的评分与播报口径;
|
||||
* 单账套查询(`eid!=0`)**保持原判据**:演示/测试账套仍可查看自己的检查结果
|
||||
(「参与检查展示」),只由响应字段 `excluded_from_business_caliber` 标注不计入。
|
||||
* 名单单一来源 = `dq_rule_config` 的 `scope.excluded_entities`(改配置值即调整,零代码回退);
|
||||
* 列可为空的历史行按「未排除」处理(`COALESCE(..., 1)` → 归默认账套 1),
|
||||
避免 SQL 三值逻辑把 NULL 归属行整批吞掉。
|
||||
"""
|
||||
ids = dqcfg.normalize_entity_ids(excluded)
|
||||
if eid or not ids:
|
||||
return "", {}
|
||||
params = {}
|
||||
keys = []
|
||||
for i, v in enumerate(ids):
|
||||
key = "nxe%d" % i
|
||||
keys.append(":" + key)
|
||||
params[key] = int(v)
|
||||
return " AND COALESCE(%s, 1) NOT IN (%s)" % (col, ", ".join(keys)), params
|
||||
|
||||
|
||||
def _business_excluded(db: Session, eid: int) -> list:
|
||||
"""本口径下生效的经营口径排除名单(**仅全账套 eid=0 生效**,其余口径返回空)。
|
||||
|
||||
单一来源 = `dq_rule_config`(`scope.excluded_entities`);本模块不得另立一份账套名单。
|
||||
"""
|
||||
if eid:
|
||||
return []
|
||||
try:
|
||||
return dqcfg.excluded_entity_ids(db, 0)
|
||||
except Exception: # noqa: BLE001
|
||||
logger.warning("DQ_SCOPE_EXCLUDED_RESOLVE_FAILED eid=0 → 按不排除处理", exc_info=True)
|
||||
return []
|
||||
|
||||
|
||||
def _vp_predicate(cfg: dict) -> Tuple[str, dict]:
|
||||
"""规则4 污染主体谓词:嫌疑关键词(大小写不敏感包含式)**减去**来源白名单(豁免优先)。
|
||||
|
||||
@@ -583,25 +621,42 @@ def _vp_predicate(cfg: dict) -> Tuple[str, dict]:
|
||||
return pred, params
|
||||
|
||||
|
||||
def _plan_id_set(db: Session, eid: int):
|
||||
"""已存在的 cash_plans id 集合(规则3 判孤儿用;eid 非 0 时只看该实体)"""
|
||||
sql = "SELECT id FROM cash_plans" + (" WHERE entity_id = :eid" if eid else "")
|
||||
return {row[0] for row in db.execute(text(sql), {"eid": eid}).fetchall()}
|
||||
def _plan_id_set(db: Session, eid: int, excluded=None):
|
||||
"""已存在的 cash_plans id 集合(规则3 判孤儿用;eid 非 0 时只看该实体)
|
||||
|
||||
经营口径(eid=0)下叠加 `scope.excluded_entities` 排除子句(F6 接线)。
|
||||
"""
|
||||
sql = "SELECT id FROM cash_plans"
|
||||
params = {"eid": eid}
|
||||
if eid:
|
||||
sql += " WHERE entity_id = :eid"
|
||||
else:
|
||||
exc, exc_params = _excluded_clause(eid, "entity_id", excluded)
|
||||
if exc:
|
||||
sql += " WHERE 1 = 1" + exc
|
||||
params.update(exc_params)
|
||||
return {row[0] for row in db.execute(text(sql), params).fetchall()}
|
||||
|
||||
|
||||
def _pending_alert_plan_links(db: Session, eid: int):
|
||||
def _pending_alert_plan_links(db: Session, eid: int, excluded=None):
|
||||
"""规则2/3 共用数据:pending cash_plan 预警 → [(alert_id, kpi_id, plan_id, kpi_entity_id)]
|
||||
|
||||
返回 (parsed, raw_rows);Python 侧解析 suggestion JSON(不依赖 MySQL JSON 函数)。
|
||||
经营口径(eid=0)下叠加 `scope.excluded_entities` 排除子句(F6 接线,按 kpi_definitions.entity_id)。
|
||||
"""
|
||||
alert_sql = (
|
||||
"SELECT a.id, a.kpi_id, a.suggestion, d.entity_id AS kpi_entity_id "
|
||||
"FROM kpi_alerts a JOIN kpi_definitions d ON d.id = a.kpi_id "
|
||||
"WHERE a.alert_type = 'cash_plan' AND a.status = 'pending'"
|
||||
)
|
||||
params = {"eid": eid}
|
||||
if eid:
|
||||
alert_sql += " AND d.entity_id = :eid"
|
||||
rows = db.execute(text(alert_sql), {"eid": eid}).fetchall()
|
||||
else:
|
||||
exc, exc_params = _excluded_clause(eid, "d.entity_id", excluded)
|
||||
alert_sql += exc
|
||||
params.update(exc_params)
|
||||
rows = db.execute(text(alert_sql), params).fetchall()
|
||||
parsed = []
|
||||
for r in rows:
|
||||
pid = _extract_plan_id(r.suggestion)
|
||||
@@ -625,48 +680,68 @@ def _dup_groups(parsed, min_count: int = 2):
|
||||
|
||||
|
||||
def _detail_count(db: Session, rule: str, eid: int, manual: bool = False,
|
||||
cfg: Optional[dict] = None) -> int:
|
||||
"""本规则真实总数:一律 COUNT(*)(dup/orphan 由预警分组得出,同样不受 LIMIT 影响)"""
|
||||
cfg: Optional[dict] = None, excluded=None) -> int:
|
||||
"""本规则真实总数:一律 COUNT(*)(dup/orphan 由预警分组得出,同样不受 LIMIT 影响)
|
||||
|
||||
经营口径(eid=0)下叠加 `scope.excluded_entities` 排除(F6 接线)——
|
||||
与 `_run_governance_checks` 的聚合口径逐字一致(两处不得漂移)。
|
||||
"""
|
||||
cfg = cfg or _cfg_values(db, eid)
|
||||
if excluded is None:
|
||||
excluded = _business_excluded(db, eid)
|
||||
|
||||
if rule == "dup_alert":
|
||||
parsed, _ = _pending_alert_plan_links(db, eid)
|
||||
parsed, _ = _pending_alert_plan_links(db, eid, excluded)
|
||||
return len(_dup_groups(parsed, _dup_min_count(cfg)))
|
||||
if rule == "orphan_check":
|
||||
parsed, _ = _pending_alert_plan_links(db, eid)
|
||||
pset = _plan_id_set(db, eid)
|
||||
parsed, _ = _pending_alert_plan_links(db, eid, excluded)
|
||||
pset = _plan_id_set(db, eid, excluded)
|
||||
return len([1 for _a, _k, pid, _e in parsed if pid not in pset])
|
||||
|
||||
ent, params = _entity_clause(eid, "cp.entity_id")
|
||||
exc, exc_params = _excluded_clause(eid, "cp.entity_id", excluded)
|
||||
params.update(exc_params)
|
||||
if rule == "unit_check":
|
||||
sql = "SELECT COUNT(*) FROM cash_plans cp WHERE cp.amount > :amt" + ent
|
||||
sql = "SELECT COUNT(*) FROM cash_plans cp WHERE cp.amount > :amt" + ent + exc
|
||||
params["amt"] = _unit_threshold(cfg)
|
||||
elif rule == "virtual_pollution":
|
||||
if manual:
|
||||
sql = "SELECT COUNT(*) FROM cash_plans cp WHERE cp.source = :msrc" + ent
|
||||
sql = "SELECT COUNT(*) FROM cash_plans cp WHERE cp.source = :msrc" + ent + exc
|
||||
params["msrc"] = _manual_source(cfg)
|
||||
else:
|
||||
pred, vp_params = _vp_predicate(cfg)
|
||||
params.update(vp_params)
|
||||
sql = "SELECT COUNT(*) FROM cash_plans cp WHERE " + pred + ent
|
||||
sql = "SELECT COUNT(*) FROM cash_plans cp WHERE " + pred + ent + exc
|
||||
elif rule == "entity_check":
|
||||
sql = ("SELECT COUNT(*) FROM kpi_values v JOIN kpi_definitions d ON v.kpi_id = d.id "
|
||||
"WHERE v.entity_id IS NOT NULL AND v.entity_id != d.entity_id")
|
||||
if eid:
|
||||
sql += " AND v.entity_id = :eid"
|
||||
params["eid"] = eid
|
||||
else:
|
||||
exc5, exc5_params = _excluded_clause(eid, "v.entity_id", excluded)
|
||||
sql += exc5
|
||||
params.update(exc5_params)
|
||||
elif rule == "kpi_completeness":
|
||||
sql = ("SELECT COUNT(*) FROM kpi_definitions d WHERE d.status='active' "
|
||||
"AND NOT EXISTS (SELECT 1 FROM kpi_values v WHERE v.kpi_id = d.id)")
|
||||
if eid:
|
||||
sql += " AND d.entity_id = :eid"
|
||||
params["eid"] = eid
|
||||
else:
|
||||
exc6, exc6_params = _excluded_clause(eid, "d.entity_id", excluded)
|
||||
sql += exc6
|
||||
params.update(exc6_params)
|
||||
elif rule == "reconciliation":
|
||||
sql = ("SELECT COUNT(*) FROM (SELECT d.id, " + RECON_DIFF_EXPR + " AS diff_pct "
|
||||
+ RECON_BASE_SQL)
|
||||
if eid:
|
||||
sql += " AND d.entity_id = :eid"
|
||||
params["eid"] = eid
|
||||
else:
|
||||
exc7, exc7_params = _excluded_clause(eid, "d.entity_id", excluded)
|
||||
sql += exc7
|
||||
params.update(exc7_params)
|
||||
sql += " GROUP BY d.id HAVING " + _recon_having(cfg) + ") t"
|
||||
params["pct"] = _recon_pct(cfg)
|
||||
else:
|
||||
@@ -675,12 +750,18 @@ def _detail_count(db: Session, rule: str, eid: int, manual: bool = False,
|
||||
|
||||
|
||||
def _detail_items(db: Session, rule: str, eid: int, limit: int, offset: int,
|
||||
manual: bool = False, cfg: Optional[dict] = None):
|
||||
"""结构化明细条目(服务端 LIMIT/OFFSET 分页;字段:record_id/entity_id/source/key_fields/summary/suggested_action)"""
|
||||
manual: bool = False, cfg: Optional[dict] = None, excluded=None):
|
||||
"""结构化明细条目(服务端 LIMIT/OFFSET 分页;字段:record_id/entity_id/source/key_fields/summary/suggested_action)
|
||||
|
||||
经营口径(eid=0)下与 `_detail_count` / `governance-check` 同源叠加
|
||||
`scope.excluded_entities` 排除(F6 接线)——三处口径一律不得漂移。
|
||||
"""
|
||||
cfg = cfg or _cfg_values(db, eid)
|
||||
if excluded is None:
|
||||
excluded = _business_excluded(db, eid)
|
||||
|
||||
if rule == "dup_alert":
|
||||
parsed, _ = _pending_alert_plan_links(db, eid)
|
||||
parsed, _ = _pending_alert_plan_links(db, eid, excluded)
|
||||
items = []
|
||||
for pid, alerts in _dup_groups(parsed, _dup_min_count(cfg)):
|
||||
items.append({
|
||||
@@ -695,8 +776,8 @@ def _detail_items(db: Session, rule: str, eid: int, limit: int, offset: int,
|
||||
return items[offset:offset + limit]
|
||||
|
||||
if rule == "orphan_check":
|
||||
parsed, _ = _pending_alert_plan_links(db, eid)
|
||||
pset = _plan_id_set(db, eid)
|
||||
parsed, _ = _pending_alert_plan_links(db, eid, excluded)
|
||||
pset = _plan_id_set(db, eid, excluded)
|
||||
items = []
|
||||
for alert_id, kpi_id, pid, ent_id in parsed:
|
||||
if pid in pset:
|
||||
@@ -712,13 +793,15 @@ def _detail_items(db: Session, rule: str, eid: int, limit: int, offset: int,
|
||||
return items[offset:offset + limit]
|
||||
|
||||
ent, params = _entity_clause(eid, "cp.entity_id")
|
||||
exc, exc_params = _excluded_clause(eid, "cp.entity_id", excluded)
|
||||
params.update(exc_params)
|
||||
params.update({"lim": limit, "off": offset})
|
||||
|
||||
if rule == "unit_check":
|
||||
threshold = _unit_threshold(cfg)
|
||||
rows = db.execute(text(
|
||||
"SELECT cp.id, cp.entity_id, cp.amount, cp.source, cp.description FROM cash_plans cp "
|
||||
"WHERE cp.amount > :amt" + ent + " ORDER BY cp.amount DESC LIMIT :lim OFFSET :off"
|
||||
"WHERE cp.amount > :amt" + ent + exc + " ORDER BY cp.amount DESC LIMIT :lim OFFSET :off"
|
||||
), {**params, "amt": threshold}).fetchall()
|
||||
return [{
|
||||
"record_id": r.id,
|
||||
@@ -740,7 +823,7 @@ def _detail_items(db: Session, rule: str, eid: int, limit: int, offset: int,
|
||||
scope_text = "污染源(%s)" % "/".join(cfg["virtual_pollution"]["suspect_keywords"])
|
||||
rows = db.execute(text(
|
||||
"SELECT cp.id, cp.entity_id, cp.source, cp.description FROM cash_plans cp "
|
||||
"WHERE " + where + ent + " ORDER BY cp.id LIMIT :lim OFFSET :off"
|
||||
"WHERE " + where + ent + exc + " ORDER BY cp.id LIMIT :lim OFFSET :off"
|
||||
), {**params, **kw_params}).fetchall()
|
||||
return [{
|
||||
"record_id": r.id,
|
||||
@@ -758,6 +841,10 @@ def _detail_items(db: Session, rule: str, eid: int, limit: int, offset: int,
|
||||
"WHERE v.entity_id IS NOT NULL AND v.entity_id != d.entity_id")
|
||||
if eid:
|
||||
sql += " AND v.entity_id = :eid"
|
||||
else:
|
||||
exc5, exc5_params = _excluded_clause(eid, "v.entity_id", excluded)
|
||||
sql += exc5
|
||||
params.update(exc5_params)
|
||||
sql += " ORDER BY v.id LIMIT :lim OFFSET :off"
|
||||
if not eid:
|
||||
params.pop("eid", None)
|
||||
@@ -777,6 +864,10 @@ def _detail_items(db: Session, rule: str, eid: int, limit: int, offset: int,
|
||||
"WHERE d.status='active' AND NOT EXISTS (SELECT 1 FROM kpi_values v WHERE v.kpi_id = d.id)")
|
||||
if eid:
|
||||
sql += " AND d.entity_id = :eid"
|
||||
else:
|
||||
exc6, exc6_params = _excluded_clause(eid, "d.entity_id", excluded)
|
||||
sql += exc6
|
||||
params.update(exc6_params)
|
||||
sql += " ORDER BY d.id LIMIT :lim OFFSET :off"
|
||||
if not eid:
|
||||
params.pop("eid", None)
|
||||
@@ -796,6 +887,10 @@ def _detail_items(db: Session, rule: str, eid: int, limit: int, offset: int,
|
||||
+ RECON_DIFF_EXPR + " AS diff_pct " + RECON_BASE_SQL)
|
||||
if eid:
|
||||
sql += " AND d.entity_id = :eid"
|
||||
else:
|
||||
exc7, exc7_params = _excluded_clause(eid, "d.entity_id", excluded)
|
||||
sql += exc7
|
||||
params.update(exc7_params)
|
||||
sql += (" GROUP BY d.id HAVING " + _recon_having(cfg)
|
||||
+ " ORDER BY ABS(diff_pct) DESC LIMIT :lim OFFSET :off")
|
||||
recon_params = {**params, "pct": _recon_pct(cfg)}
|
||||
@@ -853,10 +948,17 @@ def _run_governance_checks(db: Session, entity_id: int = 0):
|
||||
|
||||
配置化(ASK-20260928-05):阈值/关键词/条数下限/差异阈值与方向/扣分上限与系数/明细条数
|
||||
一律经 `_cfg_values()`(三层解析)取值;级别与判定表达式本体**锁死**(见 dq_rule_config)。
|
||||
|
||||
经营口径排除(F6 / ASK-05 ③-2,2026-09-29 接线):全账套口径(`entity_id=0`)下,
|
||||
`scope.excluded_entities` 名单内账套(演示/测试)的行**不再计入**评分与明细;
|
||||
单账套口径(`entity_id≠0`)保持原判据(可查看,仅标注不计入经营口径)。
|
||||
"""
|
||||
eid = entity_id or 0
|
||||
cfg = _cfg_values(db, eid)
|
||||
resolution = _load_cfg(db, eid)
|
||||
# 经营口径排除名单(F6 / ASK-05 ③-2 接线):仅全账套口径(eid=0)参与过滤
|
||||
excluded = dqcfg.excluded_entities_from_resolution(resolution)
|
||||
exclusion_applied = bool(not eid and excluded)
|
||||
issues = []
|
||||
disabled = []
|
||||
|
||||
@@ -865,11 +967,13 @@ def _run_governance_checks(db: Session, entity_id: int = 0):
|
||||
threshold = _unit_threshold(cfg)
|
||||
limit = _detail_limit(cfg, "unit_check")
|
||||
ent, params = _entity_clause(eid, "cp.entity_id")
|
||||
exc, exc_params = _excluded_clause(eid, "cp.entity_id", excluded)
|
||||
params.update(exc_params)
|
||||
rows = db.execute(text(
|
||||
"SELECT cp.id, cp.entity_id, cp.amount, cp.source, cp.description "
|
||||
"FROM cash_plans cp WHERE cp.amount > :amt" + ent + " ORDER BY cp.amount DESC LIMIT 200"
|
||||
"FROM cash_plans cp WHERE cp.amount > :amt" + ent + exc + " ORDER BY cp.amount DESC LIMIT 200"
|
||||
), {**params, "amt": threshold}).fetchall()
|
||||
unit_total = _detail_count(db, "unit_check", eid, cfg=cfg) # 真实总数(不受明细 LIMIT 截断)
|
||||
unit_total = _detail_count(db, "unit_check", eid, cfg=cfg, excluded=excluded) # 真实总数(不受明细 LIMIT 截断)
|
||||
issues.append({
|
||||
"rule": "unit_check", "level": RULES_META["unit_check"]["level"],
|
||||
"count": unit_total,
|
||||
@@ -881,10 +985,10 @@ def _run_governance_checks(db: Session, entity_id: int = 0):
|
||||
issues.append(_disabled_issue("unit_check", cfg))
|
||||
|
||||
# ── 规则2/3 共用数据:pending cash_plan 预警(实体筛选经 kpi_definitions)──
|
||||
parsed, _alert_rows = _pending_alert_plan_links(db, eid)
|
||||
parsed, _alert_rows = _pending_alert_plan_links(db, eid, excluded)
|
||||
|
||||
# 已存在的 cash_plans id 集合(规则3 判断孤儿用;实体筛选时仅看该实体下 plan)
|
||||
plan_id_set = _plan_id_set(db, eid)
|
||||
plan_id_set = _plan_id_set(db, eid, excluded)
|
||||
|
||||
# ── 规则2 重复预警:同 plan_id 的 pending 预警 ≥ min_count(默认 2)──
|
||||
if cfg["dup_alert"].get("enabled", True):
|
||||
@@ -918,17 +1022,20 @@ def _run_governance_checks(db: Session, entity_id: int = 0):
|
||||
if cfg["virtual_pollution"].get("enabled", True):
|
||||
limit = _detail_limit(cfg, "virtual_pollution")
|
||||
ent, params = _entity_clause(eid, "cp.entity_id")
|
||||
exc, exc_params = _excluded_clause(eid, "cp.entity_id", excluded)
|
||||
params.update(exc_params)
|
||||
pred, vp_params = _vp_predicate(cfg)
|
||||
rows = db.execute(text(
|
||||
"SELECT cp.id, cp.entity_id, cp.source, cp.description FROM cash_plans cp "
|
||||
"WHERE " + pred + ent + " ORDER BY cp.id LIMIT 200"
|
||||
"WHERE " + pred + ent + exc + " ORDER BY cp.id LIMIT 200"
|
||||
), {**params, **vp_params}).fetchall()
|
||||
manual_rows = db.execute(text(
|
||||
"SELECT cp.id, cp.entity_id, cp.source, cp.description FROM cash_plans cp "
|
||||
"WHERE cp.source = :msrc" + ent + " ORDER BY cp.id LIMIT 200"
|
||||
"WHERE cp.source = :msrc" + ent + exc + " ORDER BY cp.id LIMIT 200"
|
||||
), {**params, "msrc": _manual_source(cfg)}).fetchall()
|
||||
poll_total = _detail_count(db, "virtual_pollution", eid, cfg=cfg)
|
||||
manual_total = _detail_count(db, "virtual_pollution", eid, manual=True, cfg=cfg)
|
||||
poll_total = _detail_count(db, "virtual_pollution", eid, cfg=cfg, excluded=excluded)
|
||||
manual_total = _detail_count(db, "virtual_pollution", eid, manual=True, cfg=cfg,
|
||||
excluded=excluded)
|
||||
issues.append({
|
||||
"rule": "virtual_pollution", "level": RULES_META["virtual_pollution"]["level"],
|
||||
"count": poll_total,
|
||||
@@ -952,9 +1059,14 @@ def _run_governance_checks(db: Session, entity_id: int = 0):
|
||||
)
|
||||
if eid:
|
||||
ent_sql += " AND v.entity_id = :eid"
|
||||
ent_params = {"eid": eid}
|
||||
if not eid:
|
||||
exc5, exc5_params = _excluded_clause(eid, "v.entity_id", excluded)
|
||||
ent_sql += exc5
|
||||
ent_params.update(exc5_params)
|
||||
ent_sql += " ORDER BY v.id LIMIT 200"
|
||||
ent_rows = db.execute(text(ent_sql), {"eid": eid}).fetchall()
|
||||
ent_total = _detail_count(db, "entity_check", eid, cfg=cfg) # 真实总数
|
||||
ent_rows = db.execute(text(ent_sql), ent_params).fetchall()
|
||||
ent_total = _detail_count(db, "entity_check", eid, cfg=cfg, excluded=excluded) # 真实总数
|
||||
issues.append({
|
||||
"rule": "entity_check", "level": RULES_META["entity_check"]["level"],
|
||||
"count": ent_total,
|
||||
@@ -974,9 +1086,14 @@ def _run_governance_checks(db: Session, entity_id: int = 0):
|
||||
)
|
||||
if eid:
|
||||
comp_sql += " AND d.entity_id = :eid"
|
||||
comp_params = {"eid": eid}
|
||||
if not eid:
|
||||
exc6, exc6_params = _excluded_clause(eid, "d.entity_id", excluded)
|
||||
comp_sql += exc6
|
||||
comp_params.update(exc6_params)
|
||||
comp_sql += " ORDER BY d.id LIMIT 300"
|
||||
comp_rows = db.execute(text(comp_sql), {"eid": eid}).fetchall()
|
||||
comp_total = _detail_count(db, "kpi_completeness", eid, cfg=cfg)
|
||||
comp_rows = db.execute(text(comp_sql), comp_params).fetchall()
|
||||
comp_total = _detail_count(db, "kpi_completeness", eid, cfg=cfg, excluded=excluded)
|
||||
issues.append({
|
||||
"rule": "kpi_completeness", "level": RULES_META["kpi_completeness"]["level"],
|
||||
"count": comp_total,
|
||||
@@ -997,11 +1114,16 @@ def _run_governance_checks(db: Session, entity_id: int = 0):
|
||||
)
|
||||
if eid:
|
||||
recon_sql += " AND d.entity_id = :eid"
|
||||
recon_params = {"eid": eid}
|
||||
if not eid:
|
||||
exc7, exc7_params = _excluded_clause(eid, "d.entity_id", excluded)
|
||||
recon_sql += exc7
|
||||
recon_params.update(exc7_params)
|
||||
recon_sql += (" GROUP BY d.id HAVING " + _recon_having(cfg)
|
||||
+ " ORDER BY ABS(diff_pct) DESC LIMIT 200")
|
||||
recon_rows = db.execute(text(recon_sql),
|
||||
{"eid": eid, "pct": _recon_pct(cfg)}).fetchall()
|
||||
recon_total = _detail_count(db, "reconciliation", eid, cfg=cfg)
|
||||
recon_params["pct"] = _recon_pct(cfg)
|
||||
recon_rows = db.execute(text(recon_sql), recon_params).fetchall()
|
||||
recon_total = _detail_count(db, "reconciliation", eid, cfg=cfg, excluded=excluded)
|
||||
issues.append({
|
||||
"rule": "reconciliation", "level": RULES_META["reconciliation"]["level"],
|
||||
"count": recon_total,
|
||||
@@ -1039,7 +1161,6 @@ def _run_governance_checks(db: Session, entity_id: int = 0):
|
||||
|
||||
passed = [item["rule"] for item in issues if item["count"] == 0 and item["rule"] not in disabled]
|
||||
|
||||
excluded = list(resolution.get("scope", {}).get("excluded_entities") or [])
|
||||
return {
|
||||
"checked_at": datetime.now().isoformat(timespec="seconds"),
|
||||
"entity_id": eid,
|
||||
@@ -1054,9 +1175,16 @@ def _run_governance_checks(db: Session, entity_id: int = 0):
|
||||
"config_version": resolution.get("config_version"),
|
||||
"config_source": {"table_ready": resolution.get("table_ready"),
|
||||
"entity_config_enabled": resolution.get("entity_config_enabled")},
|
||||
# 演示/测试账套排除(F6):默认空集 ⇒ 不改变评分语义,只做标注
|
||||
# 演示/测试账套排除(F6 / ASK-05 ③-2):**已接线** —— 全账套口径(eid=0)的评分与
|
||||
# 明细已按本名单过滤(见 `_excluded_clause`);单账套查询仅标注不计入经营口径。
|
||||
"scope_excluded_entities": excluded,
|
||||
"excluded_from_business_caliber": bool(eid and eid in excluded),
|
||||
"excluded_from_business_caliber": dqcfg.is_excluded_from_business_caliber(eid, excluded),
|
||||
"business_caliber": {
|
||||
"caliber": "business" if not eid else "entity",
|
||||
"exclusion_applied": exclusion_applied,
|
||||
"excluded_entities": excluded,
|
||||
"included": not dqcfg.is_excluded_from_business_caliber(eid, excluded),
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
@@ -1177,9 +1305,18 @@ def get_rule_config(
|
||||
},
|
||||
"global_params": res["global"],
|
||||
"scope": res["scope"],
|
||||
"scope_note": "演示/测试账套排除(F6)当前只做标注、不改评分语义;默认空集 = 不排除",
|
||||
"scope_note": "演示/测试账套排除(F6 / ASK-05 ③-2):已接线 —— 全账套口径的评分与明细"
|
||||
"不再计入名单内账套;单账套查询仍可查看(展示)。改本配置值即可调整(零代码回退)",
|
||||
"locked": [dict(it) for it in dqcfg.LOCKED_ITEMS],
|
||||
"readonly": True,
|
||||
"write_contract": {
|
||||
"roles": ["ceo", "it"], # 与 PUT /rule-config 的 require_role 一致
|
||||
"platform_roles": ["it"], # open_level=platform 的项仅 it 可写
|
||||
"min_reason_len": dqcfg.MIN_REASON_LEN,
|
||||
"entity_config_entities": list(dqcfg.CONFIG_ENTITIES),
|
||||
"bot_channel_write": False,
|
||||
"endpoint": "PUT /api/cma/data-quality/rule-config",
|
||||
},
|
||||
"generated_at": datetime.now().isoformat(timespec="seconds"),
|
||||
}
|
||||
|
||||
@@ -1239,15 +1376,23 @@ def update_rule_config(
|
||||
if not checked["ok"]:
|
||||
raise HTTPException(400, checked["error"])
|
||||
|
||||
# 平台级项(open_level=platform)落到**全账套模板行**(entity_id=0):
|
||||
# 「平台级」语义 = 一处写、全账套生效。若写在本账套行上,值只对本账套可见,
|
||||
# 而全账套(经营)口径(entity_id=0)读不到 ⇒ 名不符实,且无法治理
|
||||
# `scope.excluded_entities` 这类平台口径项。写权限仍仅 it(上面已校验)。
|
||||
target_entity = (dqcfg.TEMPLATE_ENTITY_ID if level == dqcfg.OPEN_LEVEL_PLATFORM
|
||||
else entity_id)
|
||||
|
||||
rule_name = RULES_META.get(rule_key, {}).get("name") or rule_key
|
||||
try:
|
||||
result = dqcfg.upsert_param(db, entity_id, rule_key, param_key, checked["value"],
|
||||
result = dqcfg.upsert_param(db, target_entity, rule_key, param_key, checked["value"],
|
||||
username=getattr(current_user, "username", None))
|
||||
log_op(
|
||||
db, dqcfg.ACTION_UPDATE, target_type=dqcfg.TARGET_TYPE, target_id=result["id"],
|
||||
user_id=current_user.id, entity_id=entity_id,
|
||||
detail={
|
||||
"entity_id": entity_id,
|
||||
"target_entity_id": target_entity,
|
||||
"rule_key": rule_key,
|
||||
"rule_name": rule_name,
|
||||
"param_key": param_key,
|
||||
@@ -1275,6 +1420,7 @@ def update_rule_config(
|
||||
return {
|
||||
"success": True,
|
||||
"entity_id": entity_id,
|
||||
"target_entity_id": target_entity,
|
||||
"rule_key": rule_key,
|
||||
"param_key": param_key,
|
||||
"before": result["before"],
|
||||
@@ -1285,5 +1431,6 @@ def update_rule_config(
|
||||
"changed_by": getattr(current_user, "username", None),
|
||||
"audit": {"action": dqcfg.ACTION_UPDATE, "target_type": dqcfg.TARGET_TYPE,
|
||||
"target_id": result["id"]},
|
||||
"note": "配置即生效(下次 governance-check 按新口径计算);历史评分不重算",
|
||||
"note": "配置即生效(下次 governance-check 按新口径计算);历史评分不重算"
|
||||
+ (";平台级项写入全账套模板行(entity_id=0)" if target_entity == dqcfg.TEMPLATE_ENTITY_ID else ""),
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user