Human in the loop
Keeping humans in the loop on purpose
Where to keep a person in the loop, why judgment belongs with people, and how to design the checkpoint so the machine carries the volume.Plenty of people treat "human-in-the-loop" as a temporary crutch, something you keep until the model is good enough to remove it. That framing misses the point.
Keeping a person in the loop is a deliberate design choice about where judgment belongs.
In a lot of the systems worth building, the checkpoint survives every improvement to the model, because someone still has to answer for the output when a customer or a regulator asks.
Where to draw the line
Volume and judgment belong in different hands.
The useful line runs between the work that repeats and the calls that carry consequences.Volume
The repetitive, high-throughput work: sorting, extracting, routing, drafting, reconciling. Machines are good at it, and handing it over frees people for the part that actually needs them.
Judgment
The call that carries consequences: approving the exception, overriding the default, deciding the case the rules never anticipated. A person belongs there for accountability.
A refund that sets a precedent, a message that goes out under the company's name, a decision a regulator might ask about later: these are not places to let a model act alone, however capable it looks in a demo.
Four things that matter
The real work is placing the checkpoint in the right spot.
The choice is rarely between fully manual and fully automated. When we design a checkpoint, four things matter.Automate the volume
Run the routine start to finish where a mistake is cheap and reversible.
Escalate by confidence and impact
Route anything uncertain or high-stakes to a person. In practice that means a threshold: anything the model scores below a set confidence, or any transaction over a set amount, stops for review.
Make the human's job easy
A good checkpoint is a ten-second decision. It shows the reviewer exactly what to decide, with the context already gathered.
Capture the decisions
The overrides a person keeps making the same way are next quarter's rules, and encoding them shrinks the queue over time.
One workflow, one review
A support inbox a model drafts and a person clears once a day.
Take a support inbox. A model reads every message, sorts it, and drafts a reply in the business's own tone. Once a day a person reviews the batch, fixes what reads wrong, and sends in one pass.
The person decides whether the answer actually fits this customer, and nothing leaves the building until they approve it. The outbox still sounds like a person wrote it.
It stays on purpose
The checkpoint earns its place as the automation gets better.
As the machine absorbs the routine, what reaches a person is a smaller, sharper set of decisions, the ones that carry real weight and need someone to answer for them.
Where to start