Human-in-the-loop: where the loop belongs
Approval placed at every step slows everything and is ignored within a month. On putting the checkpoint where a wrong answer would be expensive, and nowhere else.
Human-in-the-loop has become the phrase people reach for to make an AI system sound responsible, and it usually means a checkbox that says a person will look at the output. It is worth being more precise, because where you put the checkpoint decides both how safe the system is and whether anyone keeps using it.
Approval at every step does not survive
The instinct on a first build is to require sign-off everywhere. It feels cautious and it is the single most reliable way to produce a system nobody trusts.
What happens is predictable. Week one, the reviewer reads everything carefully. Week three, the queue is long and most of what is in it has been correct, so they start skimming. Week six, they are approving in batches without reading. The checkpoint is still there in the diagram and has stopped existing in practice, and now the system has the worst property available: it looks supervised and is not.
A checkpoint that is always crossed successfully teaches the person crossing it that it does not need reading.
Put the loop where a wrong answer is expensive
The useful question is not how much oversight, but which decisions actually carry consequence. Two properties matter, and they are independent.
- Reversibility. A draft that goes to a colleague can be fixed in thirty seconds. A message that goes to a client, a price that goes into a contract, or a record written into a system of record cannot. Put the checkpoint before the irreversible edge, not before the drafting.
- Blast radius. One wrong output affecting one record is a nuisance. One wrong rule applied silently across four thousand records is an incident. Volume changes the answer even when the per-item consequence is small.
Applied properly this usually means far fewer checkpoints than the cautious version, placed much later. In a lead system: no review of the research, no review of the qualification reasoning, a hard stop before anything is sent to a human being outside the company. In a quoting system: no review of the draft, a hard stop before a number goes to a client. One checkpoint each, at the edge, where a person is genuinely deciding something rather than rubber-stamping a step.
The reasoning has to be attached
A checkpoint is only real if the person at it can tell whether the answer is right without redoing the work. That means the output has to arrive with what it was based on: which records, which rule, and which part of the input drove the decision. An output with no reasoning attached cannot be reviewed. It can only be trusted or not trusted, and after a few weeks the answer is always trusted.
This is also where most organisations are genuinely behind, and it is measured. Deploying autonomous components has moved much faster than the ability to govern them.
One in five, among the organisations already running agents. The rest are operating systems that take actions with no settled answer to who is accountable when one of those actions is wrong.
What we actually build in
- A hard stop before every irreversible action, and none before reversible ones.
- The reasoning attached to every decision that reaches a person, so review is possible rather than nominal.
- An escalation path for anything the system is not confident about, which is a different mechanism from routine approval and must not share a queue with it.
- A record of what was decided and on what basis, kept whether or not anyone is currently asking.
- A kill switch that a non-technical person can reach without calling us.
The last one gets left out of most designs and is the one that matters most at three in the morning. If the only way to stop a running system is to contact the firm that built it, the business does not control the system. It is renting the illusion of control, and it will find that out at the least convenient possible moment.