LLM Evaluation CI
EvalGate
0.90 min gate · 10-case suite
Demo video unavailable
01 Problem
[problem: how prompt edits reached production untested, and what broke]
02 Approach
Prompt regression harness that gates every system prompt change against a fixed case set before deployment. Runs side-by-side Baseline vs. Candidate prompt diffs, enforces a strict minimum pass threshold (min = 0.90), and blocks CI builds when previously passing assertions regress.
03 Result
Prevented silent prompt regressions from reaching production by gating every system prompt edit against a deterministic 10-case evaluation suite (regex, exact label, and keyword contracts) with sub-2ms local verification.
0.90 min pass threshold · Prompt A/B regression diff · Zero-dependency TypeScript & Web Workbench
- 0.90Minimum CI gate threshold
- 10Deterministic contract cases
- A vs. BSide-by-side prompt diffing
- 0 depsNative Node 24 TypeScript