Context engineering used to focus on adding information that base models lacked:
But then the models kept getting better. Now, the same context your agents couldn’t function without can make them perform worse.
Our AI onboarding wizard, for example, often landed on the wrong project in monorepos because our scripts pointed to root by default. Now that models are good at inferring repo structure, we updated it so headless runs take advantage of model recommendations.
This is the same reason why Anthropic removed 80% of Claude Code’s system prompt. Labs are going back on previous advice about rules and repetition; judgment, interfaces, and progressive disclosure are the new best practices instead.
The goal of context engineering has since shifted toward subtracting as much information as you can to get out of the models’ way:
But figuring out exactly what to subtract isn’t easy when dealing with unpredictable behavior.
When asked about his approach, Boris Cherny, the creator of Claude Code, recommended deleting your CLAUDE.md every six months to stay on the bleeding edge. Theo recently reported it was worth rewriting his AGENTS.md by hand:
If you'd rather not start from scratch every time, here are three practical ways to maintain your context while "unhobbling" AI, from most hands-on to most autonomous.
1. Run /doctor, but follow up with real eyes
The simplest way to keep your context healthy is to read and edit it yourself.
Most coding agents have a built-in /doctor command to help with this. Anthropic’s version of it runs basic health checks, deletes redundant prompts, catches broken settings, finds unused plugins, and optimizes for lazy loading.
The report also prints a summary of how often you’ve used each file, and how much it can trim. Here’s what that looks like for our website’s main AGENTS.md:
Component: posthog.com/AGENTS.md (= CLAUDE.md symlink)
Type: memory file
Scope: project (checked in)
Uses (total since install): always loaded
Used in window?: yes
Est. resident tokens: ~1,780
Verdict: trim ~350Across the posthog.com repo, it suggested turning off 3 unused plugins and 3 skills across our website repo and estimated saving an average of 6K tokens per session.
This is decent, but you can’t rely on it to catch everything because it doesn’t check for correctness; it acts only based on what it can derive from your code.
For example, last month, we started using merge queues and added this line to our AGENTS.md:
“All merges into master go through the Trunk merge queue. Never run gh pr merge.”
A couple days later, we paused the queue to fix some failing tests and forgot to update that line. Our agents therefore had wrong instructions for 21 hours. During that time, one engineer’s PR got stuck for 10 hours; another lost 45 minutes investigating the cause before updating the instructions.
We didn’t use claude doctor in the posthog monorepo to fix this, but it couldn’t have caught it anyway since merge queue state lives in a GitHub setting rather than in code. Agents don’t know what they don’t know.
Try this: Run
claude doctorafter each upgrade and follow up with a manual pass on yourAGENTS.md. For each line, if you can’t name the failure it prevents, delete it.
2. Test your context like it’s code
Another tool for maintaining and unhobbling your context is evals.
It’s just like adding regression tests whenever you fix a bug in code. Every time you update your context to address an agent’s mistake, capture what caused it in the first place as an eval.
For example, we record technical “gotchas” we see from real PostHog Wizard runs as framework-specific commandments such as:
For versions 15.3+, initialize PostHog in
instrumentation-client.tsfor the simplest setup.For Phoenix or Plug apps, add
PostHog.Integrations.Plugbefore the router so request context is attached to captured events and errorsposthog-rsis the Rust SDK crate; add it withcargo add posthog-rsand construct the client withposthog_rs::client(options).await.
These get combined with instructions from the context-mill (e.g., “reuse event names the project already uses”) and a set of ~40 clean sample apps to create the evals we use in the wizard-ci. Here’s how it works:
wizard-ciruns the PostHog Wizard on all ~40 sample apps and creates one PR for each.Those PRs don’t get merged. Instead, a second agent called the pr-evaluator grades each of them based on the diffs and session logs.
The
pr-evaluatorleaves metrics and reports on the trigger PR and thewizard-ci PR.
These reports help us catch issues like the wizard skipping installing PostHog entirely because the model determined it was already complete:
wizard-ci is just one of many context-as-code tools we’ve built in our wizard-workbench. The setup is complex since it maintains our biggest conversion path, but the core concept is easy to apply: test your context by saving prompts that check if your agents are doing what you want them to do.
Try this: The next time your agent makes a mistake, paste the prompt that caused it into a
failures.md. Re-run those prompts the next time you edit or delete parts of yourAGENTS.mdas a quick test suite for your highest-cost piece of context.
3. Ask agents for feedback directly
Another option for fixing your context is to just ask. Agents are good at providing feedback about how they would improve your context while they’re using it.
Many developers already do this to update their skills, but you can take it further by putting it in your prompts by default.
For example, the final instruction for the PostHog Wizard says to send back a quick remark about any errors they encountered in production:
“What information or guidance would have been useful to have in the integration prompt or documentation for this task? Specifically anything that would have prevented tool failures, erroneous edits, or other wasted turns.”
This gives us a rich feed of live bug reports for cheap – a basic form of AI observability:
We took this even further by feeding them back into our context-mill with a loop, making it self-driving. But since we can't trust agents at face value, we cluster and verify the underlying issues first. Here's what the feedback clusters looked like for the last month:
Half of the other category is just confirmation messages like “succeeded on the first attempt,” or “posthog-js already installed”. The other half is a long tail of issues that weren’t common enough to cluster.
Once the loop identifies a meaningful cluster, it deploys subagents to verify the issue before attempting a fix. For example, to check the notebook create schema/tool issue, a subagent followed the legacy instructions to reproduce the hand-splicing text failure that was addressed in this PR.
This approach works best at scale to cluster remarks, but you can still apply the concept to your workflows by simply asking agents to log what context files they used.
We do this in our posthog monorepo PR template, which prompts agents to name any skills invoked. We’ve seen other agents use this information to catch and fix skill inconsistencies, like in this PR where Claude found an issue while unblocking stalled ClickHouse cleanup PRs.
Try this: Agents face issues all the time. Ask them to provide feedback about your context with a structured prompt about any errors, inconsistencies, or failures they ran into. Over time, you can turn verified reports into a self-driving context system.
Written by Jina Yoon, who would like a word with whoever coined the term “unhobbling.”
🧠 Optimize your HUMANS.md
A field guide to Claude Fable 5: Finding your unknowns – Thariq Shihipar
Our context warehouse knows your data, the semantic layer tells it what to trust – Thiago Rocha Salvatore, Lizzie Epton
The most common questions about developer marketing, answered – Joe Martin
Nobody watches their session replays: A study of 7.7 million views – Natalia Amorim
Treat prompt changes like code deploys – Luke Murray
Deleting you code and rewriting sound like a poor maintenance performance approach. Why not do a skill that finds and defines the diff or model context adjustment than create automated new MD version to test than replace keeping the old version until you are comfortable