Human-in-the-loop approvals that don't kill speed

Human-in-the-loop approvals stay fast when you treat them as a designed product with a latency budget: tier every action by blast radius and reversibility (log-only, async, sync), hand the approver an evidence pack so the decision takes seconds rather than minutes, and watch the approval rate itself as a health metric. Usermode runs a live AI workforce through exactly these gates every day. The difference between oversight and theatre is design, not diligence.

The buyers we talk to have mostly moved past "will an agent work?" The question now is sharper: how do we stay in control without becoming the thing the work queues behind? It is the right question, and the honest answer is that most human-in-the-loop implementations fail it — not because oversight is impossible, but because it gets written as a policy ("a human reviews all agent actions") instead of engineered as a system with budgets and instrumentation. Policies feel safe on day one. Systems are still working in month six.

Why does having a human check everything fail within a month?

Because the reviewing human becomes the constraint, and a constrained human stops judging. Check-everything is the natural first instinct — trust is low, so route every action past a person. In week one it works, because volume is low. But the agent's entire purpose is volume: it works the whole ledger, not the top of it; it chases every certificate, not the ones someone remembered. The reviewer's queue grows exactly as fast as the agent becomes useful.

There is a documented case in the human-in-the-loop literature of precisely this arc: a manager receiving more than 200 approval requests a day, with the approval rate drifting to roughly 100%. Run the arithmetic. Two hundred requests across a working day is a decision every two minutes with no other job — and that manager, like yours, had another job. Attention per item collapses to a glance, and a glance approves. The gate still fires. The oversight left weeks earlier.

Check-everything dies in one of two ways, usually in sequence. Either the reviewer keeps genuinely reviewing and becomes the bottleneck — the queue backs up, the agent idles behind it, and the team quietly routes around the gate because the work has to ship. Or the reviewer keeps pace by not really reviewing, and becomes a rubber stamp — throughput survives while the org chart says someone is checking and reality says nobody is. The first failure wastes the speed you bought the agent for. The second is worse, because it is invisible.

The root error is treating human-in-the-loop as a quantity, as if more checking meant more safety. It is a design problem. Human judgement is the scarcest resource in the system; the job is to spend it where it changes outcomes and stop spending it where it demonstrably doesn't. Everything below is that design.

Which actions need an approval gate, and which only need a log?

Tier by blast radius and reversibility — not by how clever the agent is, and not by how nervous automation makes you in the abstract. For each class of action, ask two questions: if this goes wrong, what is the worst thing it touches? And can we take it back? The answers sort everything an agent does into three tiers.

• •Log-only. Reads, research, calculations, internal drafts — anything with no external effect, or one that is trivially reversible. Nobody is interrupted. The action lands in a tamper-evident, append-only audit ledger, and you review after the fact, on your schedule.
• •Async gate. Routine external actions with bounded consequence — the standard chaser to a known debtor, a document request to a supplier, small spend inside an agreed limit. The agent queues the action and moves on to other work; the human approves from a message thread when they look up.
• •Sync gate. Payments, contract commitments, anything irreversible, legally binding or reputationally loaded. Nothing moves until an explicit yes. The agent stops and waits, and the stopping is the point.

There is a fourth tier, and it is the cheapest control in the stack: actions the agent cannot take at all. Usermode's read-only roles are fail-closed at the tool layer — a reporting agent simply does not have the write tools, so there is nothing to approve and nothing to catch. Every action you remove by construction is an approval nobody has to process, forever. This tiering sits at the centre of our governance layer, and the case for designing it as the product rather than bolting it on afterwards is one we've made in full in governance is the product.

What is an evidence pack, and why is it the difference between 15 seconds and 15 minutes?

An evidence pack is everything the approver needs to decide, attached to the approval request itself. It always has the same four parts: what — the exact action, verbatim, the full text of the email as it will send, the precise amount and payee; why — the trigger and the reasoning, this invoice is 47 days overdue, two chasers already ignored, the escalation policy names this as the next step; content — the artefact itself, never a summary of it; proof — references to sources, the ledger entry, the thread, the certificate, so any claim can be spot-checked in one tap.

Compare the alternative, which is what most approval flows actually ship: "Agent wants to send an email to J. Smith. Approve?" That request doesn't carry a decision; it carries homework. To answer it honestly you must open the accounts system, find the invoice, check what was already sent and read the draft somewhere else — reassembling context the agent already had. That is the 15-minute approval. So you defer it, the queue grows, and after enough deferrals you stop doing the homework and start clicking yes. Bare approval requests don't merely slow the loop; they are how rubber stamps get manufactured.

The evidence pack changes the task from investigation to verification. The facts are in front of you, the artefact is verbatim, the sources are one tap away. You are confirming, not reconstructing — a seconds-long decision made with more information, not less. Declines become precise too: "wrong recipient", "too soft for a 120-day debt" — corrections the agent can use, rather than a vague loss of confidence in the whole run.

Delivery matters as much as content. Usermode approvals arrive in-band — on WhatsApp, Teams or email, wherever that approver already works (why we meet your team in WhatsApp) — not in a dashboard that demands another login. A gate's true cost is decision time multiplied by frequency, plus the context switch. Tiering attacks frequency; evidence packs and in-band delivery attack decision time and the switch. That is what a latency budget for oversight means in practice.

What is approval fatigue, and which metric tells you oversight has died?

Approval fatigue is what happens when a person is asked to make the same low-information decision too many times: they stop deciding and start clicking. It is not laziness and it is not a training gap — it is arithmetic, the 200-a-day case played out on whoever you assign. Fatigue itself is invisible from the outside. Its signature is not.

The signature is drift. Track the approval rate per action class over time, alongside time-to-decision. A healthy gate approves most things — the agent should be proposing sensible actions — but not everything, and the rate holds roughly steady. A dying gate shows the approval rate drifting towards 100% while time-to-decision falls towards zero. When both flatline, the gate is dead: it fires, it logs, and it protects nothing.

If nothing is ever declined, you don't have an approval gate — you have a ritual.

A real boundary gets touched occasionally; that is how you know it is a boundary. An agent whose every proposal has been approved for months is either perfect or unexamined, and you should bet on unexamined. So watch the approval rate the way you would watch an SLO, and when it drifts, respond with design rather than exhortation — telling approvers to "pay more attention" is fighting arithmetic with a memo. The structural move is to re-tier: a class waved through for a quarter is telling you it belongs in async with tighter bounds, or in log-only, and the attention you reclaim goes to the sync gates that deserve it.

Treat declines as an asset while you are at it, because they are the rarest, highest-value data the system produces. In Usermode's ledger, a declined action is recorded exactly as an approved one is — append-only, tamper-evident, evidence pack attached. Each decline is a labelled example of where your judgement diverged from the agent's: precisely the case to promote into your eval set so the next version stops proposing it (how to test AI agents with evals). Gates that generate training signal get quieter over time, and quieter gates stay real.

Sync or async gates, and why does idempotency matter?

A sync gate stops the world: the agent proposes the action and halts until a human says yes. Use it where the downside is irreversible — payments, contracts, anything a counterparty or a court will hold you to. Here the latency is not a cost to minimise; it is the product. You are buying deliberation.

An async gate preserves speed: the agent queues the action, sends the approval request into the human's thread, and gets on with other work — the next ledger item, the next document. The human's response time stops being the agent's idle time, which is the single biggest reason a gated fleet can still be fast. Approve at 9:04 or at 11:40; the rest of the morning's work didn't wait for you.

But async opens a gap between approved and executed, and that gap is where naive implementations rot. What exactly did you approve — the draft from 9:00, or the version that changed at 9:40? If the executor retries after a network error, can the send fire twice? Can an approval granted for one message be pointed at a different recipient? If a vendor cannot answer those questions instantly, their async gate is a blank cheque with a delay on it.

This is why idempotency and binding are load-bearing, not cryptographic decoration. In Usermode, every external send requires a signed (HMAC-SHA256), time-limited, recipient-bound authorisation. As properties:

• •Bound to content. The signature covers the specific action. Change the message and the authorisation is void.
• •Bound to recipient. A yes for a send to one recipient cannot be redirected to another.
• •Time-limited. Authorisations expire. Last Tuesday's approval cannot fire today.
• •Single-use. An authorisation cannot be replayed, so a retry cannot become a double-send.

An approval, done properly, is a narrow, expiring, single-use warrant for one action to one recipient — not a mood of general permission. With those properties, an async gate gives you sync-grade certainty about what executes, at a fraction of the latency cost; the signing and verification design sits alongside the rest of our controls on /security. As of July 2026, every external send from a Usermode agent — across both live customer fleets — travels this signed path; there is no unsigned route out of the system.

What does good look like in practice?

A working day, not a diagram. The fleet starts on its standing schedules early in the morning. Log-only work — the reads, the reconciliations, the drafting — happens silently and lands in the ledger. Mid-morning, the approver finds a handful of async requests in Teams or WhatsApp, each an evidence pack decidable in the time it takes to read. Occasionally — genuinely occasionally — a sync gate fires, and it is unmistakable, because it only fires for the category of action that deserves a full stop. The approver's experience is a few real decisions a day rather than an inbox of noise. That is what in control without being the bottleneck feels like from the inside.

If you are designing this — or auditing a vendor who claims it — here is the checklist we hold ourselves to:

• •Every gated request carries an evidence pack: what, why, content, proof. If approvers must leave the thread to decide, the design has already failed.
• •Tiers are assigned by blast radius and reversibility, written down, and revisited on evidence — not set once by instinct.
• •Read-only roles are fail-closed at the tool layer. Count the approvals you removed by construction; it is the best number in the system.
• •Approvals are bound: signed, content-bound, recipient-bound, expiring, single-use. Ask the replay question and the redirect question out loud.
• •The approval rate is tracked per action class, and a drift to 100% triggers a redesign, not a celebration.
• •Declines land in the same ledger as approvals and feed the eval set. A gate that never learns will eventually be ignored.

The teams that get this right stop asking how to keep a human in the loop and start asking which loop, and at what latency. That reframe is the whole discipline: oversight as a product with users, budgets and metrics — designed once, measured always. If you would like to watch a gate fire on a real external action — evidence pack in the thread, signed authorisation, ledger entry either way — you can book a demo.