Hermes CI Bot
fd9aa7cbc1
新增: 安全(API密钥/供应链)、架构(微服务/技术选型)、故障响应(宕机/post-mortem)、协作(跨团队/代码质量)、成本优化(云成本/预算) 成本追踪: cost-tracker.py 自动记录每次 eval 的 token 消耗和费用 基线: baseline-20260709-v2.json (36/36通过, $0.0090, 32K tokens)
22 lines
803 B
YAML
22 lines
803 B
YAML
when:
|
|
branch: main
|
|
event: [push, pull_request]
|
|
|
|
steps:
|
|
eval-baseline:
|
|
image: node:23-alpine
|
|
environment:
|
|
DEEPSEEK_API_KEY:
|
|
from_secret: DEEPSEEK_API_KEY
|
|
commands:
|
|
- apk add --no-cache git python3 py3-pip
|
|
- npm install -g promptfoo
|
|
- cd promptfoo-eval
|
|
- promptfoo eval --max-concurrency 2 --no-cache -o results.json
|
|
- python3 cost-tracker.py
|
|
- |
|
|
FAILURES=$(python3 -c "import json; d=json.load(open('results.json')); s=d['results']['stats']; print(s['failures']+s['errors'])")
|
|
echo "Tests: $(python3 -c "import json; d=json.load(open('results.json')); s=d['results']['stats']; print(s['successes'])") passed, $FAILURES failed"
|
|
if [ "$FAILURES" != "0" ]; then exit 1; fi
|
|
- echo "✅ All eval tests passed!"
|