The cost of a second opinion

A pattern that keeps coming up in product conversations: have a second model check the first model's work. It sounds free. It is not.

We tried it on a genuinely tedious job — reconciling a device inventory against a licensing export, a few thousand rows, the kind of task that is too fiddly to script and too dull to do properly.

What the second pass caught

Real things. Rows where a convention had been applied silently and wrongly. Two cases where a plausible-looking total had been assembled from the wrong column. A confident summary that did not match its own working.

That is a good hit rate for something that took a few minutes.

What it cost

Roughly double the tokens, which was the part I had budgeted for, and a review step that nobody owned, which was the part I had not.

The reviewer produces a list of concerns. Someone has to read it, decide which concerns are real, and act. When that someone is me, fine. When it is a team, "the AI flagged something" turns into a queue, and queues need an owner, an SLA, and a rule for what happens when the flag is wrong.

Where this landed

We kept the second pass, and we changed what it outputs. Instead of a list of concerns it now returns a decision — safe to proceed or stop, here is the row — with the reasoning attached but collapsed.

Same model, same cost, a fraction of the human time. The change was not technical. It was deciding, up front, what a human was supposed to do with the answer.