Writing · Field notes
The golden set: how to prove an agent works before it touches a customer.
In a regulated firm, it seemed to work in the demo is not a defence. This is how you build the evidence that it does.
· 6 min read
Every regulated firm has the same fear about AI, and it is a reasonable one. An agent that is right ninety-five times and confidently wrong five times is not a ninety-five percent solution. It is a five percent liability, and the five percent is invisible until it is in front of a regulator, a client or a judge. The demo hides this. The demo shows the ninety-five. The golden set is how you find the five before they find you.
A golden set is a fixed collection of real cases from your own work, each with an agreed correct answer, that the agent must handle before anyone trusts it. Not synthetic examples. Not the vendor's benchmark. Your cases, judged by your experts, frozen so the score means something. Here is how I build one.
Start with the cases that scare you.
Do not sample at random. Ask the people who do the work today for the cases that keep them up at night: the ambiguous contract clause, the payment that looks fine but is not, the email that is polite and fraudulent. A golden set weighted toward the hard cases tells you what random sampling never will.
Fix the answers before you see the agent's.
Each case gets a correct outcome agreed by a human expert, written down, and locked. The moment you let the agent's output influence what counts as correct, the evaluation is theatre. The answers come first.
Score, and keep the score.
Run the agent against the set and record the number. Then keep running it, every change, every model swap, every prompt tweak. The number is the only honest answer to the question the board will ask: is it getting better, or are we guessing.
Name the failures.
A score of eighty-two percent is not a finding. The eighteen percent is. Group the failures, name them, and decide for each whether it is acceptable, mitigable or a stop. At Helionics, the company I founded, our agents were proven against a public corpus of 500,000 real corporate emails precisely because a public, fixed set makes the failures impossible to hide.
Make it a gate, not a report.
The golden set only works if a failing score stops the release. If it produces a slide that everyone admires and no one acts on, you have built a metric, not a control.
The golden set is unglamorous. It is also the difference between an agent you can defend in a regulated firm and one you are quietly hoping never gets tested. In this work, hope is not a control.
The letters
Occasional letters on getting AI into production.
Work email only.