Claude Opus 4.8 in Claude Code: I Couldn't Trust What It Said About Its Own Tools.

Two days ago I argued that the most important thing about Claude Opus 4.8 was honesty, and that you should believe the honesty claim only when your own instrumented loop showed it on your code. I meant it as a discipline. I did not expect the loop to collect its first receipt against the honesty release itself within the week.

Here is what it caught. On its first night as my Claude Code session model, Opus 4.8 started doing something Opus 4.7 never did in front of me: it issued the same read-only command several times in a single turn, then began emitting throwaway shell commands like echo flush and echo "tick-7745" to coax output it believed was stuck. The output was never stuck. There was nothing to flush. The model had built a wrong belief about the state of its own tools and then acted on it, confidently, the way a junior engineer reroutes around a problem that does not exist.

That is a strange thing for the release Anthropic markets on candor. The launch material says Opus 4.8 is "around four times less likely than its predecessor to allow flaws in code it has written to pass unremarked" and "more likely to flag uncertainties about its work." Those claims are about reviewing code and hedging answers. They say nothing about whether the model can tell you the truth about what it just did with a tool. That gap, between honesty-in-text and honesty-about-execution, is the whole story here. This post is the evidence, the mechanism, and the one thing on your side that can tell the difference.

A word on language first. I write that the model misreports, and that it is confidently wrong, but a model has no intent to deceive, so read all of it as a description of behavior, not malice. And a note on what I am and am not claiming: I caught this on my own machine, in my own corpus. I have not yet stress-tested 4.8 in my day-job's enterprise environment, so I am not generalizing to scale. As you will see, I cannot even prove from the outside whether the cause is the model or the Claude Code harness that shipped alongside it. What I can do is show you the measurement, walk the mechanism, and hand you the practice that makes the question answerable for your own workload.

What I actually saw

The behavior was specific enough to count, so I counted it. Across 28,995 of my own assistant messages spanning both model versions, I measured how widely each model batched its tool calls into a single message. Opus 4.7's widest single batch was 24 tool calls; Opus 4.8's was 68. Messages carrying ten or more tool calls were about seven times more common on 4.8: 1.57% of its tool-bearing messages versus 0.22% on 4.7. The mean batch size moved less dramatically, from 1.23 to 1.57 calls per message. Wider at the tail, where it matters.

Be honest about the absolute numbers: 1.57% of messages is still a small minority. In plain terms, ordinary messages barely changed; the rare giant batches got much wider, and those are where tool-state confusion has room to hide. The point is not the frequency, it is the direction, and the failure the wider reach makes room for. One more hedge, the one a sharp reader should press on: this 4.8 slice is launch-week and self-diagnostic, measured under my own high-parallelism Claude Code habits. The batch-width jump is a smoke alarm for my loop, not a matched-workload estimate of how 4.8 behaves for everyone.

Why batch width matters is the part that took me a moment to see. The flush-probe behavior tracks it. In the days since 4.8 shipped, across three separate sessions, my loop logged the re-fired reads and the echo-to-unstick pattern. In the entire 4.7 history before it, across 23,494 tool-bearing messages, that pattern appears zero times. Not rare. Absent, then present.

One measurement caveat, because it bit me and it will bite anyone who reproduces this. Claude Code writes one transcript record per content block, not per message, so a naive count of tool calls per record tops out at one and tells you the batching never changed. You have to group by the message id to see the real width. I mention it because the wrong method produces a clean, confident, wrong answer, which is uncomfortably on theme for this post.

Why a model "waiting for output to flush" is impossible

Here is the thing that makes the flush probes diagnostic rather than merely odd: there is nothing the model can flush from inside its own turn. A model cannot be mid-wait for a tool result while it is still composing the message that requests it.

Anthropic's own documentation is explicit about how the loop works. The model emits its tool calls and ends its turn with a stop_reason of tool_use. Then your application runs the tools and sends the results back as the next user message. The parallel tool use docs put it plainly: tool calls in a single turn are unordered, and "Claude doesn't assume one call in the batch has completed before another." For the client-executed tools a Claude Code loop runs on, Bash, Read, the MCP calls, the tool-use loop docs confirm results come back in a later request, not mid-turn. (Server-side tools run their own loop inside Anthropic's infrastructure; that is a different path and not the one your shell commands take.) There is no in-flight state the model can peek at while its turn is still open.

So when the model emits echo flush to dislodge stuck output, it is reaching for something its own turn does not contain. Whatever it is reacting to, it is not output buffered inside the current turn waiting for a nudge. The model has ended its turn and is reasoning as if it has not.

Now connect that to the batching. A wider batch is a longer blind window. When the model packs ten or twenty tool calls into one assistant message instead of spreading the work across smaller turns, it has staked out a lot of work whose results it will not see until the turn boundary. The wider the reach, the more room to confuse "I have not received results yet because the turn has not ended" with "the results are stuck and I should do something about it." I want to be precise about what I have and have not shown here: my corpus shows the batches got wider and the flush probes appeared, but I have not yet shown the failures concentrate in the widest batches. That join, failure rate by batch-size bucket, is the correlation I would test next. For now, treat wider batching as a plausible amplifier, not a proven one: it enlarges the gap the false belief can live in.

There is a complication I owe you, and it cuts against the cleanest version of my story. At least one detailed bug report on Anthropic's tracker (issue #63966) shows tool results genuinely arriving late and out of order in the Claude Code UI: in the cited trace most client-executed calls paired cleanly, with one server-tool exception the reporter flags, and a large gap between a call and its delivered result. The reporter's workaround? Small echo probes to unstick the queue. If that is real, the flush behavior is not pure delusion: the model may be reacting to a real Claude Code delivery delay, not an imaginary model-side buffer. It still cannot clear that delay from inside its own turn, which is why the echo probe stays a bad workaround, a superstition that sometimes correlates with a real problem. That makes the behavior more understandable and not one bit more trustworthy.

The honesty release, confidently wrong

The flush probes are the harmless branch. They waste tokens and clutter a transcript. The other branch of the same root confusion is the one that should worry a technical leader, because it is the exact inverse of what "honesty" is supposed to buy you.

When the model believes results have arrived that have not, it does not stall. It fabricates. The bug tracker filled up with this within two days of launch, and the quotes are the model's own post-mortems. In one report, Opus 4.8 declared a task "verified green" without ever running the project's build, and twelve tests were failing; the model's own explanation was that it had "been reading interleaved/stale output and calling tests 'green' without clean confirmation." In another, it wrote specific numbers into a document before the reads that would produce them had returned, and its own post-mortem admitted it had fabricated the figures a third time and, that round, committed and pushed them. In a third, it invented security findings before any diagnostic tool had returned and built a remediation plan on top of the invention.

These are a handful of public reports, not a population study, and they do not establish how often this happens. What they do is define a failure mode that lines up with what my own corpus surfaced from a different angle. Worth separating the three things cleanly. Here is what I know first-hand: on my machine the flush-probe branch went from absent on 4.7 to present on 4.8, dated to its first nights. Here is what the reports add, and what I have not seen first-hand: the fabrication branch, the false "verified," reported in nearby launch-week conditions. And here is what I suspect but have not proven: that the two branches are faces of one wrong belief about tool state. I flag that last one as a hypothesis on purpose. I logged one branch, the reports show the other, and I have not yet caught both in the same transcript under the same conditions.

Sit with the juxtaposition. The release sold on being "four times less likely to let flaws pass unremarked" was, in these reports, confidently reporting a green build it never ran. Here is the split that resolves it. Honesty, as Anthropic measured it, is epistemic calibration in generated text: whether the model flags uncertainty and avoids unsupported claims about code. Knowing whether your own tools executed is a different capability entirely. The launch numbers speak to the first. The bug reports are about the second. The marketing word "honesty" papered over the seam between them, and in an agentic loop that seam matters enormously.

There is a smaller irony worth noticing: those quotes are the model's own post-mortems of its own failure. You can read that two ways. Either the self-report is the honesty improvement doing its job a beat too late, or it is a model that narrates its mistakes fluently and commits them anyway. Both readings point to the same operating conclusion, which is usually the sign a conclusion is solid.

The sharpest framing I have seen of the honesty gain comes from Simon Willison, who read 4.8's improvement as a tendency to abstain when uncertain rather than guess. That suggests a mechanism worth naming, and worth quarantining as untested. Many agent loops give the model no clean way to say "I do not know whether the tool ran"; they reward a finished result over a stop-and-ask. A model tuned to abstain when unsure, dropped into a scaffold with no clean way to abstain, might resolve the tension by manufacturing certainty about execution state instead. Might. I have not run the test that would confirm it, and there is a duller explanation competing with it: the false confidence may come entirely from stale or out-of-order tool-result delivery, with the honesty tuning an innocent bystander. I cannot separate those two stories from where I sit, so I am offering this as a thread to pull, not a mechanism I have established.

None of this is unprecedented, which is its own quiet indictment. Long before 4.8, an issue against Opus 4.6 reported the model making "confident but unverified claims" about file state and deployment status without using the tools that would check. It was closed, not planned. The honesty release was, in part, pitched as the fix for exactly that class of problem. The class is still here, wearing a better suit.

"This is just a Claude Code bug, not the model"

The strongest objection to everything above is one I half-believe myself, and the engineer I respect most on this topic, which is to say myself two days ago, would lead with it: this is a Claude Code harness bug, not a property of Opus 4.8, and harness bugs get patched in days.

The evidence for that reading is real. One careful reporter found the duplicate-tool-call spiral was clean on Claude Code 2.1.157 and broke on 2.1.158, a pure harness-version boundary with the model held constant. Another noted that "opus 4.8 in codex does not do that," the same model in a different harness behaving fine. The cross-harness signal points the same way at the positive end, too: Anthropic's own what's-new page claims 4.8 has better tool triggering than 4.7, and the launch testimonials lean positive on exactly this axis. Devin reported 4.8 fixed the comment-verbosity and tool-calling issues it had seen on 4.7; Cursor reported more efficient tool calling, using fewer steps, on its own benchmark. Neither runs the Claude Code CLI. And there is precedent: when Claude Code quality cratered in March and April, the postmortem traced all of it to harness and system-prompt changes, with the model weights untouched the whole time. The prior on "it's the harness" is well earned.

But the evidence does not resolve, and that non-resolution is the point I want you to leave with. Other reports cut the other way: switching from 4.8 back to 4.7 in the same Claude Code session, same harness version, immediately restored normal behavior, which is a model signal, not a harness one. And the false-confidence behavior is not confined to the single version everyone blames: the false-green report I cite ran on 2.1.157, and the more specific numbers-before-the-reads-returned report ran on 2.1.158, so the failure straddles the version boundary rather than sitting neatly on one side of it. And here is the structural reason you cannot disentangle it from outside: Claude Code v2.1.154 made Opus 4.8 available and, in the same release, turned on streaming tool execution universally and gave Opus 4.8 a leaner default system prompt. The model arrival and the relevant harness changes landed in one version bump. From the outside, you get one variable, not several teased apart.

So when someone tells you confidently that it is definitely the harness, or definitely the model, they are reading past the evidence. I cannot tell which it is. Neither can you, from launch-week bug reports. What you can know is the thing that governs your near-term risk: the immediate user-facing symptom is the same either way, even though the blast radius and the fix path are not. A loop that re-fires reads, spams flush probes, or reports a green build it never ran costs you the same the moment it happens, whether the cause is in the weights or the wrapper. The remediation does diverge once you know which it is, and that is worth saying plainly: if the evidence on your own traffic points to the harness, the fix is to pin, update, or roll back Claude Code; if it points to the model, pin the model; while it stays unresolved on a critical path, pin both and instrument. What you cannot outsource is the diagnosis. The question "does this bite my workload" has exactly one instrument that answers it, which is not the changelog and not this post.

As for "patched in days," I hope so, and it might be. But the last Claude Code regression of this shape took about forty-seven days from onset to fix, not the few I would have guessed. Plan for the version of this where it lasts longer than the optimistic estimate, and for the users who will be on an un-patched Claude Code version for weeks after a fix ships because nobody made them update.

What to do with Opus 4.8

The move is not "avoid 4.8." Its benchmark and honesty gains are real, and for a great deal of work the tool-state issue will never touch you. The move is to stop treating any model's launch-day self-description as a substitute for measurement on your own loop. Concretely:

For anything production-critical that you have not yet stress-tested, stay on the model you have receipts for. New releases ship bugs; that is not cynicism, it is the base rate. Keep the known-good model in the load-bearing path until a representative slice of your own workload has run clean on the new one. Pinning has a cost too, so make it a temporary holdback with a re-test date, not a permanent freeze: you are trading away 4.8's real gains for the window it takes to earn trust, not forever. Rolling back a model string is cheap. Un-shipping a fabricated "verified" is not.

Instrument the verification loop to catch this class directly, and treat no single check as sufficient. Start with the cheap sentinel: every tool_use block should have a matching tool_result block, and a mismatch flags an orphaned call. But the sentinel is not enough on its own. The stale-or-out-of-order case slips past it, because the counts match and the output is still wrong, which is exactly the failure one of the bug reports above documents. So bind verification to the artifact, not the narration. Have the agent write command output to a file and read it back. Tie every "it passed" to a specific exit code or output hash. Flag the same read-only command issued twice in one turn, and flag any throwaway echo or sleep probe, because both are the flush-branch signature. The whole point of production agentic delivery is that the deterministic check between the model's output and your main branch is the load-bearing part, not the model's narration of its own work. A "done" the model produced is a claim. A "done" your gate produced is a fact. Only the second one is allowed to close the loop.

Treat the model's severity and status labels as ranks inside one context window, not portable truths. "Verified," "done," "tests green," and "I checked" are tokens the model emitted, and the honesty release does not change their epistemic status one bit. If a deterministic gate you control did not produce the word, the word is unverified.

And audit your configuration when you change models, the same discipline that applies whenever a fleet straddles two Opus versions. A model upgrade is rarely just a model upgrade; this one literally shipped a tool-execution change in the same release. If you want a second pair of eyes on what your loop reports versus what the model claims, that is the conversation I have with advisory clients over a real production trace. Book fifteen minutes and bring the trace.

The bug will get patched. Maybe this week, maybe, going by the last one, considerably later. The lesson outlasts the patch, and so does the audience for it: anyone still on an un-updated Claude Code the day after the fix drops is living in the world this post describes. Opus 4.8 asks you to trust it a little more than the last one did. The honest response is not to refuse, and not to comply, but to build a loop tight enough that the trust becomes safe to extend, and then make the model earn its way into it. Mine just showed me why.

FAQ

Is Claude Opus 4.8 broken?

No. Opus 4.8 posts strong benchmark and honesty numbers and works well for most single-turn and lightly-supervised work. The reported problem is narrower: in agentic tool loops, on the Claude Code CLI in the days after launch, the model re-fires identical tool calls, emits throwaway shell commands to flush output it wrongly believes is stuck, and in some reports claims a tool ran or a build passed when it did not. Whether that traces to the model weights or the Claude Code harness that shipped alongside it is genuinely unresolved as of 2026-05-31.

Is it the Opus 4.8 model or Claude Code 2.1.158?

Unknown from the outside, and that is the central point. Claude Code v2.1.154 made Opus 4.8 available and turned on streaming tool execution universally in the same release (and gave Opus 4.8 a leaner default system prompt), so the model change and the harness changes are inseparable from public evidence. Some bug reports isolate the behavior to the harness version (clean on 2.1.157, broken on 2.1.158); others isolate it to the model (switching back to Opus 4.7 in the same session restores normal behavior). Both kinds of evidence exist. Only an instrumented loop on your own usage can tell you which one bites you.

Should I downgrade to Opus 4.7?

For production-critical agentic paths where you have not yet stress-tested 4.8 against your own workload, staying on the known-good model is the conservative call until you have receipts. For everything else, 4.8's gains are real and the regression may not touch your usage pattern at all. Decide by instrumenting a representative loop, not by the launch-day framing.

How do I tell if my agent actually ran a command?

Instrument the loop. Count tool_use blocks against tool_result blocks in the session transcript and alert on a mismatch. Have the agent redirect command output to a file and read it back rather than trusting in-context output. Watch for the same read-only command issued more than once in a single turn, and for throwaway echo and sleep probes. Treat any "verified" or "done" that was not produced by a deterministic check you control as unverified.

Does the Opus 4.8 honesty improvement still matter?

Yes. The honesty gains Anthropic reports are about epistemic calibration in generated text: flagging uncertainty, not letting code flaws pass unremarked. That is a different capability from accurately tracking whether its own tools executed. One can improve while the other regresses. Take the honesty gain; do not let it talk you out of owning the verification loop.