How to Design Human-in-the-Loop AI Approvals

// In plain English

When an AI only talks — answers a question, drafts a paragraph, summarizes a PDF — the worst it can do is be wrong, and you just read it and move on. But the moment an AI starts to act — send an email, delete a file, charge a card, push a change to your calendar — a wrong answer becomes a real-world mistake that's already happened. Human-in-the-loop UX is the set of design patterns that put a person between the AI's decision and the action it would take.

Think of it like a self-checkout that still needs a human for the alcohol or the unexpected item. The machine does the fast, boring work — scanning, totaling, bagging — but for the steps that carry real risk, a person glances at the screen and taps approve. You get the speed of automation for the easy 95% and a human gate on the risky 5%. That's exactly the trade human-in-the-loop UX is tuning.

Concretely, it's a handful of moves you wire into the interface: show the user what the AI is about to do before it does it, let them edit the AI's draft instead of accepting it whole, make them confirm irreversible steps, and give them a way to undo when something slips through. The skill is deciding how much friction each action deserves — too little and users get burned, too much and they stop using the feature.

// Why it matters

An LLM is a probabilistic system: it's right most of the time and confidently wrong some of the time. That's tolerable when output is just text on a screen. It becomes dangerous the instant you connect the model to tools that change state — the same model that occasionally invents a fact will occasionally email the wrong person or overwrite the wrong row. You can't make the model perfect, so you design around its imperfection.

Three forces make the approval step matter:

• Mistakes are now actions, not opinions. A hallucinated sentence costs nothing to ignore. A hallucinated DELETE runs. The cost of an error jumps the moment the AI touches the outside world, and the UI is your last checkpoint before that happens.
• Trust has to be earned, not assumed. New users don't know how reliable your AI is. If you let it act silently on day one, the first visible mistake destroys trust permanently. A review step lets users watch it be right a few times and grant it more autonomy as confidence grows.
• Accountability and reversibility. In many domains — money, health, legal, communications — somebody must be able to say "a human approved this." An explicit approval, with a record, is often a compliance requirement, not just a nicety. And actions that can't be undone need a stronger gate than ones that can.

Who should care? Anyone building beyond a chatbot. AI assistants that book meetings or send replies. Agentic tools that run multi-step workflows. "Fix my code and open a PR" developer tools. Customer-support bots that issue refunds. If your AI's output is an action rather than a paragraph, human-in-the-loop UX is the difference between a feature people trust and one they turn off after the first scare.

// How it works

The core pattern is propose → review → commit. The AI never acts directly; it produces a proposed action — a structured description of what it wants to do. Your app renders that proposal as a human-readable preview, the user reviews (and optionally edits) it, and only an explicit confirmation actually executes the action. Nothing irreversible happens between the model and the world without passing through that gate.

1. Separate proposing from doing

The first design decision is technical: have the model return the action rather than perform it. Instead of letting the model call send_email() directly, have it emit a description — recipient, subject, body — that your app holds. This single separation is what makes a review step even possible. The model decides what to do; your code decides whether and when it actually runs.

{
  "action": "send_email",
  "requires_approval": true,
  "args": {
    "to": "dana@acme.com",
    "subject": "Re: Q3 invoice",
    "body": "Hi Dana, attaching the corrected invoice..."
  },
  "reversible": false
}

2. Preview in human terms, not raw output

A good preview shows the effect of the action, not the JSON. "Send this email to Dana Lee" with the actual rendered message beats a code blob. For edits, show a diff — before and after — so the user sees precisely what changes. The rule: the user should be able to predict the outcome from the preview alone, without trusting the AI's summary of itself.

3. Make the response cheap and obvious

Give the user three first-class choices — approve, edit, reject — and make edit as easy as approve. The most valuable interaction in HITL is the user fixing one wrong field and then approving, because it keeps them in control without throwing away the AI's work. After commit, show what happened ("Email sent to Dana") and, when possible, an Undo for a few seconds. Undo is psychologically huge: it lets people approve quickly because they know a mistake is recoverable.

// Calibrating friction to the cost of a mistake

The central craft of human-in-the-loop UX is choosing how much friction each action deserves. Two variables drive it: reversibility (can you undo it?) and blast radius (how bad is it if it's wrong?). High on both means a heavy gate; low on both means let it fly. Putting a confirm dialog on a one-click-undoable draft is as wrong as letting a wire transfer go through silently.

Friction should also decrease as trust grows. A new user might approve every action; an experienced one who has watched the AI be right 50 times wants to auto-approve the safe categories and only gate the scary ones. Letting users set their own autonomy level — per action type — is how you keep power users fast without exposing beginners to risk.

// Patterns and common pitfalls

Patterns that work

• Editable drafts over take-it-or-leave-it. Present the AI's output as a starting point the user shapes, not a verdict they accept or discard. Editing keeps the human in control and produces better training signal about what the AI got wrong.
• Batch review for many small actions. When the AI proposes ten edits, show all ten with individual checkboxes and a select-all, rather than ten separate dialogs. One review of a list beats ten interruptions.
• Undo as the default safety net. For anything reversible, prefer optimistic execution plus a generous undo window over a blocking confirm. It feels fast and safe.
• State what will happen, in the user's words. "This will email 340 customers" tells the user the blast radius. "Proceed?" hides it. The number is the whole point.

Pitfalls that erode trust

• Silent action. The AI does something and the user only finds out later (or never). Even a perfect action shown after the fact is better than a correct one done in secret — visibility is trust.
• Fake review. A confirm button under a wall of text nobody reads is theater. If the preview isn't scannable in two seconds, the gate doesn't actually gate anything.
• All-or-nothing autonomy. Forcing users to either approve every step or trust the AI blindly. Real systems need a middle setting: auto for safe categories, gated for risky ones.
• No record of who approved what. When something goes wrong, you and the user both need to see what was proposed, what was edited, and what was confirmed. Log the whole chain.
• Trusting AI-summarized consequences. If the model says "this will only update one row" but actually wrote a query touching thousands, the preview must reflect the real effect computed by your code — never the model's claim about itself.

// Going deeper

Once the basic propose-review-commit loop works, the interesting problems are about scaling trust without scaling friction. A few directions worth knowing.

Graduated autonomy and policy gates. Instead of a single global "ask me" toggle, define rules: auto-approve emails to internal addresses, always gate payments over a threshold, require confirm for anything touching production. This moves the approval decision from per-click to per-policy, so routine actions flow and only genuine outliers stop for a human. It's the UI counterpart of how serious agentic systems decide when to act versus when to ask.

Trust calibration over time. Two failure modes mirror each other: over-trust (the user rubber-stamps everything and a bad action slips through) and under-trust (the user reviews so paranoidly the AI saves no time). Good UX nudges users toward appropriate trust — surfacing the AI's confidence, flagging when it's operating outside its usual pattern, and loosening gates only as a track record accumulates. The goal isn't maximum trust; it's trust that matches actual reliability.

Asynchronous and long-running approvals. When an agent runs for minutes and hits an approval point, the user may not be watching. You need to pause the run, notify the person, and resume from the exact step once they respond — without losing the agent's intermediate state. This durable "pause for human, then continue" flow is one of the harder engineering problems behind HITL UX, and it's why action-taking agents often need a persisted task queue, not just a chat box.

Latency interacts with friction. An approval step adds a human-speed pause to an otherwise instant flow, so where you place it matters — gate at the end of a fast pipeline and the wait feels jarring. Pairing approvals with good latency design (streaming the proposal as it forms, pre-computing the preview) keeps the gate from feeling like a stall. The durable principle stays constant: the AI's autonomy should never exceed the user's ability to see, understand, and reverse what it does — see what makes good AI UX for the broader frame.