Why Does Claude Code Seem to Get Worse Over Time?

TL;DR

Claude Code can feel less capable even when the underlying model weights have not traditionally become dumber, because the user experience also depends on the harness, system prompt, tools, API filters, inference hosting, and accumulated context. Reported symptoms include refusals, incorrect solutions, lost task intent, and benchmark declines, so diagnosing regression requires separating failures across these layers.

Transcript

Have you noticed Claude's performance varying by day? Claude Opus 4.7 is a serious regression, not an upgrade. AMD's AI director slams Claude for becoming dumber and lazier since last update. Opus 4.6 is getting dumber. Cloud code performance regression with Opus 47 models. Quantified evidence. Sonnet 46 quality regression since March 9th. Cloud mo... Read More

Key Insights

• Claude Code's perceived quality is determined by several layers, including the harness, system prompt, available tools, API processing, GPU inference, and conversational context. A regression in the user experience therefore does not prove that the model weights themselves became less capable.
• Regression is not a single behavior. The presenter distinguishes API-level blocks, model-level refusals, incorrect solutions, and cases where the model becomes lost and stops following the user's intended task, even though it may continue producing plausible actions or code.
• Task refusal can appear as an explicit API rejection or as the model deciding that a request falls outside its role. Claude Code demonstrated the latter by declining to investigate a Dropbox application problem and redirecting the user toward settings that were inaccessible.
• Dumber solutions are outputs that incorrectly solve problems the model could previously handle, such as flipping the wrong boolean or writing code that fails to behave properly. This differs from refusal because the model attempts the task but produces a technically or logically weaker result.
• Getting lost is a failure to preserve the user's intent across an interaction. During work on a repository-cloning script, Opus 4.7 reportedly misinterpreted earlier statements and repeatedly changed the script in ways that did not match the requested behavior.
• Margin Lab's Claude Code tracking showed a weighted SWE-bench average falling from 57% to 55% between March and the present period described. The decline was characterized as modest but consistent, with a later bump attributed to the arrival of a smarter model.
• A coding harness adds substantial hidden context to every user message. It can supply the system prompt, descriptions of permitted tools, product capabilities, and instructions governing command execution, file reading, and machine changes before the request reaches the API.
• Harness and API changes can create regression-like symptoms without altering the underlying model parameters. Modified tool expectations may increase errors, excessive low-quality context may confuse the model, and more aggressive filtering may change or block requests before GPU inference occurs.

Questions & Answers

Q: Why can Claude Code seem to get worse over time?

Claude Code can seem worse because its output depends on more than the model parameters. A user request passes through a harness containing system instructions, tool definitions, product capabilities, and additional context. It then reaches an API that may scan or filter the request before GPU inference. Changes at any of these layers can cause refusals, errors, weaker solutions, or lost intent.

Q: Did Claude's underlying model actually become dumber?

The presenter does not conclude that the underlying model became dumber in the traditional sense. Instead, the argument is that the degraded experience is real but may originate in the harness, API filtering, inference hosting, tool expectations, or accumulated context. Because every layer affects responses, poor behavior alone cannot identify model weights as the root cause.

Q: What types of Claude Code regression are described?

The reported regressions fall into several distinct categories. An API can hard-block a request, the model can explicitly decline a task, or it can avoid the requested work by choosing another path. The model may also produce technically incorrect solutions or become lost during a longer interaction, failing to follow the user's original intent despite continuing to respond.

Q: What is the difference between a refusal and a dumb solution?

A refusal occurs when the request is blocked by the API or when the model says it will not perform the task. A dumb solution occurs when the model attempts the work but does it incorrectly, such as flipping a boolean that should remain unchanged or writing code that fails to behave as intended. The first stops the task, while the second mishandles it.

Q: What does getting lost mean in a Claude coding session?

Getting lost means the model stops accurately following the user's intended task, often after interpreting earlier instructions incorrectly. The presenter observed this while asking Opus 4.7 to build a script for cloning a repository to other computer locations. The model repeatedly lost track of the request and made incorrect changes based on misreadings of previous statements.

Q: What benchmark evidence suggests Claude Code regressed?

Margin Lab's internal tracking reportedly runs SWE-bench through Claude Code and showed a weighted average decline from 57% to 55% from March to the period discussed. The presenter describes the drop as not huge but consistently downward across successive weeks. A small bump near the end was attributed to a newer, smarter model entering the tracker.

Q: How does the Claude Code harness affect model performance?

The harness packages the user's text with additional information before sending it to the API. That information includes the system prompt, descriptions of tools, available product functionality, and instructions that let the model run commands, read text, or change files. Altering these definitions or adding poor context can increase tool errors, confusion, refusals, and failures to follow intent.

Q: How does a Claude Code request reach the model?

The user enters a prompt through Claude Code, whose harness combines it with system instructions, tool descriptions, and other context. The resulting request goes to an API, which may scan whether the request is appropriate, sometimes using another model. If approved, it is sent to GPUs that load the model parameters and generate likely next tokens repeatedly until producing a result.

Summary & Key Takeaways

• Claude users report that performance varies over time, with recent complaints involving Opus 4.7, Opus 4.6, and Sonnet 4.6. The presenter previously considered many regression claims overblown but now recognizes similar problems, including unexpected refusals, weaker solutions, lost task intent, and occasional language changes during extended interactions.

• The presenter separates perceived regression into several categories: API blocks, model refusals, technically incorrect solutions, and failures to preserve the user's intent. One example involved Claude Code declining to investigate a Dropbox application problem, while another coding session repeatedly misinterpreted requirements and changed a repository-cloning script in unintended ways.

• A request passes through more than the model itself. The product harness adds a system prompt, tool definitions, functionality descriptions, and other context before an API checks and forwards the request for GPU inference. Changes or failures at any layer can degrade results, while Margin Lab's reported SWE-bench average declined from 57% to 55%.