Writing · Field notes

The Production Gate: the 23 checks an AI agent must pass before it ships.

Most enterprise pilots do not fail. They stall at review, in front of someone whose job is to say no. This is the list that gets you through that meeting.

· 6 min read

Every enterprise has a version of the same meeting. The agent works, the demo lands, and then legal, security and operations ask their questions. The pilot that cannot answer them stalls there, sometimes forever.

The difference between a pilot and a production system is not the model. It is whether those questions were answered before anyone asked.

I keep the answers as a checklist of 23 checks, grouped in four gates. It has survived delivery for Shell, Freshfields, JP Morgan and T-Mobile in one form or another; this is the current form.

Gate one, data.

  1. 1.Every data source the agent reads has a named owner who said yes in writing.
  2. 2.Data residency is mapped: where it is stored, where it is processed, where it leaves.
  3. 3.Personal data is inventoried and the lawful basis is written down.
  4. 4.Retention is decided: what the agent stores, for how long, deleted by whom.
  5. 5.The agent's access is scoped to the minimum that makes the workflow run.

Gate two, behaviour.

  1. 6.There is a golden set of real cases with agreed correct outcomes, and the agent's score on it is written down.
  2. 7.The failure modes have names and the top three have mitigations.
  3. 8.Someone tried to break it on purpose and the results are documented.
  4. 9.The agent declines cleanly outside its scope instead of improvising.
  5. 10.Behaviour is reproducible: same input, same policy, explainable difference if any.
  6. 11.Output the business would regret is defined, and blocked, not discouraged.

Gate three, governance.

  1. 12.A human owns every consequential action, and the audit trail shows who.
  2. 13.Every action the agent takes is logged where compliance can read it.
  3. 14.The EU AI Act classification is written down, with the obligations that follow.
  4. 15.The works council or staff representatives heard about it from you, not from a rumour.
  5. 16.There is a kill switch, and the person who can pull it knows they can.
  6. 17.The vendor dependencies are listed with what happens if each one disappears.

Gate four, operations.

  1. 18.The agent has an owner in the org chart, with budget, after the consultants leave.
  2. 19.Support knows what to do when the agent is wrong at 2am.
  3. 20.Cost per case is measured, not estimated.
  4. 21.The metric the board will see is agreed and reported monthly.
  5. 22.The rollback plan is tested, not written.
  6. 23.The next wedge is named, so success has somewhere to go.

Twenty-three checks looks like bureaucracy until you watch a pilot die in a review meeting for want of one of them.

The checklist is not the delivery. It is the shape of the questions that will be asked anyway. Answer them first and the meeting where pilots stall becomes the meeting where yours shipped.

The letters

Occasional letters on getting AI into production.

Work email only.