I’ve been building my own AI agent for months now. Claude Code, Codex, a pile of other tools wired together into one thing that runs my day. A while back I wrote about how to build your first one, and that post covered a few of the mistakes I made while putting it together.

This one is different. It is about breaking the agent you already have. The one you use every day. The one running in the background while you sleep, the one you trust to actually do things.

I also wrote about the time I almost fried my agent and my Mac Mini, but those were my specific accidents. This is the general version. The field guide. The list of ways any agent goes sideways once it is in your hands, so you can make fewer of these mistakes, or at least see them coming.

And here is the part people skip past. This is not only about custom agents like mine. The same things break ChatGPT, Claude, a Zapier flow, whatever framework you picked off the shelf. AI moves fast. Speed does not make a thing unbreakable. You can break anything you build the moment you start using it, and an agent is no exception.

First, the reason agents break at all

It is not that the model is dumb. It is math.

An agent does work in steps. Read this, call that tool, decide, act, check, repeat. Steps multiply, they do not average. If each step works 99% of the time, ten steps in a row work about 90% of the time. A hundred steps, around 37%. A thousand steps, basically never. Researchers measured this directly, and it gets worse, because the errors are not independent. One wrong step nudges the next one wrong too.

So every time you add surface to your agent, more tools, more memory, more steps, more things it can touch, you are not adding risk in a straight line. You are multiplying it. Keep that in your head. It quietly explains every item below.

1. You overbuild it

This one is mine. I have a real problem with overbuilding.

When I want something, I build it. Then I want another thing, so I build that too. Each piece works on its own. The trouble is the pile. Building toward your own needs feels right in the moment, because it is natural. You need this, you need that, so you keep adding. I learned the cost of it too late.

More pieces means a higher chance that something, somewhere, is broken at any given moment. That is just probability. And past a point the tools start working against the agent instead of for it. There is a name for this now. Microsoft researchers call it tool-space interference. Give a model too many tools and it picks the wrong one, burns tokens, or invents a call that does not exist. Studies have measured the drop in tool-selection accuracy anywhere from 7 to 85% as the catalog grows, and it gets worse for tools sitting in the middle of a long list, the same lost-in-the-middle effect that hits long context. OpenAI limits a single request to 128 tools, and coding agents like Cursor warn that quality slips well before you stack even a few dozen. Either way, the cap is not where the trouble starts. Every tool you add competes for the model’s attention long before you hit any limit.

Context has the same shape of problem. There is solid research on context rot now: across 18 frontier models, accuracy fell 30 to 50% as more was stuffed into the window, well before the window was even full. On the million-token models it started showing up around 300 to 400 thousand tokens. So an agent buried in its own accumulated context gets measurably worse, not because it ran out of room, but because the room got noisy.

The fix is not glamorous. Prune. Do a periodic checkup on your skills, your tools, your memory, your core files. If your error registry is lit up red, the agent is already telling you something is broken and you stopped reading it. I keep my CLAUDE.md tight partly for this, to keep the core readable instead of letting it bloat into something nobody can hold in their head.

There is a structural fix too, and it is the one I would push hardest. Stop showing the agent everything at once. The pattern that holds up in production is search-then-load: the agent keeps a small index of what it can do, looks up the few tools it needs for the task in front of it, and loads only those. Same idea for context. Treat the window like a budget you spend on purpose and compact it often, rather than letting months of history pile up and rot. The agent that carries less is the agent that stays sharp.

2. You make one agent do everything

My first idea of an agent was Jarvis. One mind that handles all of it. I owned up to this in the first-agent post too. It is the romantic version everyone starts with.

For strictly personal stuff with tight, specific context, one agent is genuinely fine. Useful, even. But the moment the work gets complex, writing code, running services, juggling separate projects, the single agent starts to behave like a monolithic codebase. Every new thing makes it heavier and harder to maintain. Eventually it is so bloated you spend more time managing the agent than getting anything out of it. If you want to break it, just keep making it bigger until it pops.

This is an open argument in the field right now, and both sides are worth knowing. Anthropic built a multi-agent research system with a lead agent that hands work to specialized subagents, each with its own clean context, and it beat a single agent by over 90% on their evals. The catch: it burned roughly 15 times more tokens, and it was worse for tightly connected work like coding. Cognition went the other way in a piece literally titled Don’t Build Multi-Agents, arguing that splitting context across agents makes them fragile, because actions carry implicit decisions, and conflicting decisions carry bad results.

Both are right, which is the useful part. The version that holds up: one agent stays in charge of the thread and does the actual writing and acting, while the extra agents go fetch context and intelligence rather than take conflicting actions of their own. That is roughly where Cognition landed after more time in production too.

If you want one rule for where to draw the line, use context. Keep work inside a single agent while the context stays small and shared. The moment a task drags in a big, specific pile of context the main agent does not otherwise need, that is your seam. Cut there. The job that needs to read a whole codebase, or a month of a project’s history, or a pile of research, gets its own agent with its own window, and hands back a result instead of dumping all of that into the one mind you actually talk to.

That is the change I made early, and I am glad I did. I have one main agent I talk to. Anything with heavy or specific context, a particular project, a content task, a research dig, gets spun off into a subagent or a workflow. In Claude Code I lean on workflows and subagents constantly. The real trick is the instructions. Your main CLAUDE.md or agents file has to tell the agent, in plain words, when to spawn help instead of swallowing the whole thing itself. If it does not know to delegate, it will try to do everything, and you are back to the monolith. I went deeper on this idea in The Bounded AI Agent.

3. You let it poison its own memory

An agent with memory is great until it remembers something wrong.

Share

This one only shows up in use, never while you build. You save a fact, the agent recalls it later and acts on it, and if that fact was wrong, it is now wrong forever, on a loop. One bad entry, recalled a hundred times. Microsoft’s red-teaming team has a clean name for a cousin of this, session context contamination, where junk from one context leaks into the next and quietly steers the agent off course.

I hit this regularly. My agent writes things to its own memory, and every so often a drift warning fires because something it saved no longer matches reality. Without that check, the bad memory just sits there, shaping decisions, looking exactly like a good one.

The fix is hygiene. Memory needs the same discipline as code. Verify before you save. Link facts so you can trace where they came from. Let the system flag drift instead of trusting it blindly. I built a self-improving loop for exactly this, but a loop only helps if it can catch its own bad entries. A memory you never audit is not an asset. It is a slow leak you cannot see.

What actually works for me is layers, not one big bucket. A small working memory for the task at hand, and a durable layer for facts that should outlive the session. Every durable fact gets a source and a date, so I can trace where it came from and retire it once it goes stale. And the agent is allowed to question its own memory, a quiet check that fires when a saved fact stops matching reality. A fact with no source and no expiry is a rumor your agent will repeat with full confidence, forever.

4. You give it no fallback

This one is simple. If you want an agent that runs 24/7, you need a fallback for the model.

A base model is fused into the agent, sometimes a few of them, and sooner or later one is unreachable. Rate limited, deprecated, down for an hour, whatever it is. A fallback is not a permanent plan B for some other model. It is the thing that keeps the lights on when the main thing goes dark. I run an OpenRouter subscription partly for this, so the agent can fail over to another provider instead of just stopping. It costs a bit, you pay for the keys, and it still beats having nothing.

One honest warning, because I picked it up reading about other people’s outages. A single fallback is not a guarantee. In August 2025, OpenRouter itself went down for about 50 minutes, and that took its own fallbacks with it. So layer it. A gateway for provider failover, plus something local as a floor. I run a 35B model on my Mac Mini for that reason, small jobs and a last resort for when the network itself is the problem. Not everyone can run a decent local model, I know that. But even a small one beats a dead agent.

One more lesson, the one most people skip. A fallback you have never tested is not a fallback. It is a second thing you are also assuming works. Pull the primary on purpose every so often and watch what happens. Does the agent actually fail over, or does it just fall on its face. I would rather find a broken failover on a quiet Tuesday than at 3am when the main model is down and everything is dark.

5. You trust it without checking

The scariest failures are the quiet ones.

An agent that crashes is easy. You see it, you fix it. The dangerous one is the agent that says done and did nothing. It writes a log line, the log line says success, and you believe it. The actual work never happened. From the outside it looks alive. It is the difference between a heartbeat on a screen and a pulse you actually put your fingers on.

Go back to the step math from the top. Without checking each step, you never see the 37%. You see the green light and assume the other 63 ran fine. I have been bitten by this enough times that I built a watchdog whose entire job is to catch the skip, the loop that runs and logs and accomplishes nothing. When I wrote about the agent starting to fix itself, half of that work was really about catching silent failure before it had a chance to compound.

The fix is to stop trusting shallow signals. A log line is not proof. Re-run the thing that was supposed to happen and look at the result. It is the same reason I am religious about commits as save points now. You want a real, inspectable trail, not a vibe that it probably worked.

The lesson under the lesson: check the outcome, not the activity. Did the file actually change. Did the message actually send. Did the row actually land in the database. Whether the function ran without throwing tells you almost nothing. And make the check safe to run twice, because you will run it twice. A fix is not done because the code ran once in a session. It is done when I re-run the exact thing that was failing and watch it pass. Until then it is a guess wearing a green checkmark.

Share Digital Thoughts

6. You never challenge it

This last one is for the people running their own custom agent. Challenge it. On purpose.

I try to do this more and more. I throw scenarios at my agent that it has never, or rarely, seen. Rapid-fire messages. Weird formats. Requests it has no obvious handler for. Things I do not even need, just to watch what happens. The whole promise of an agent is that it works out a solution to the problem in front of it. The reality is that it often cannot, and you want to find that out before a real situation does it for you.

Here is the piece of advice I would hand any new builder. When you throw a challenge at your agent and it fails, the problem is almost never the challenge. It is the architecture underneath that could not let the agent get there. A whole discipline is forming around this idea, people are calling it harness engineering, the argument being that the scaffolding around the model, context, tools, memory, verification, decides whether it succeeds far more than the raw model does. Microsoft’s year of red-teaming agents says the same thing from the security side: systems that pass model-level tests still fall apart under real pressure, because the failure lives in the system, not the brain.

So make it a habit. Keep a small set of nasty cases, the weird formats, the rapid-fire messages, the request that needs three tools in a row, and run them again every time you change the architecture. That is a regression test for an agent. When one fails, you know where to look, because the harness is a short list: how context gets in, how tools are exposed, how memory is stored and recalled, how work gets verified, what the agent is and is not allowed to touch. The break is almost always in one of those, not in the model’s head.

So when it breaks, do not reach for the model. Look at what stopped it, and fix that. An agent you never challenge is one that quietly stops growing. One day you ask it for something slightly new, it just cannot do it, and you have no idea why.

None of this means stop building. Build. Overbuild, even, for a while, because that is how you find the edges. Just know that the day you start using an agent is the day you start breaking it, and that is okay. Every break is a map to the next thing worth fixing.

My agent messed up plenty of things. Everything was my fault, because I am the architect. It cost me time to fix, and I genuinely do not mind. It is progress, and I accept that.

If you are building your own from scratch, start here. Then come back and break it on purpose. That part is the actual work.

My agent made me a lot more effective. It also made me watch my own back more.

Both of those are true, and I have stopped pretending the second one away. I built my own agent, I gave it its own machine, and it does real work for me every day. It knows who I am, not just what I want. The effectiveness is real. So is the quiet voice in the back of my head that now tracks what might break while the thing is running.

That voice is the interesting part. It points straight at what this whole wave is actually about. Trust.

Trust is a mileage problem

Think about a car. When I get in mine, I assume it starts. I assume it will not catch fire on the way to Katowice. I never sat down and decided to trust it. I drove it something like a hundred thousand kilometers and the trust built itself. I know its sounds. I know the one strange thing it does on a cold morning. I can predict it because I have repeated it that many times.

Trust is repetition plus outcomes you can predict. That is the whole recipe. There is always room for a bad day, a flat tyre, a dead battery in February. Although the range of what can go wrong is small, and I know the edges of it.

With an agent that recipe only half works. For some tasks I have the same calm I have with the car. For others I am still a new driver, both hands on the wheel, watching the road like it owes me money.

Deterministic things earn trust faster

Here is the pattern I keep running into. The more deterministic the task, the faster I trust it. A script that renames the same files the same way every night, I stopped watching that months ago. It is boring, and boring is exactly the point.

The more agentic the task, the more I am looking at a black box. Open-ended work, many steps, judgment calls, recovering from its own mistakes halfway through a run. That is where outcomes spread out and prediction gets hard. The model matters here. So does the architecture around it, the tools it can call, the memory it carries. I wrote a whole post about what happens when an agent meets the messy real world and stops behaving like the demo.

The ceiling right now is technical and it is not a mystery. Context limits. Memory that does not persist the way I want it to. Retrieval that grabs the wrong thing at the wrong moment. Tool calls that fail quietly. An agent that gets stuck and does not notice it is stuck. Every builder I talk to is fighting the same short list. I wrote about drawing hard edges around an agent so it stays inside what it is good at in the bounded agent post, and about giving it a memory that actually compounds in the one on my self-improving agent.

None of these limits are permanent. I have watched all of them get better over the past year, in jumps, never on a tidy schedule. That detail matters for everything that comes next.

Now run it forward

Assume the boring version of the future. Steady improvement, the kind we have already been getting. Better models, more stable tools, memory that holds, retrieval that lands where you point it. The frontier gets the headlines. The floor is the thing that moves people’s lives, and the floor is what I am watching.

Right now an agent like mine is a nerd object. You need to be deep in code, or at least deep in tinkering, to get real value out of it. Most people who say they use AI mean a chat window. Most companies that say they use AI are in the 88% with almost nothing to show for it. The capability is sitting right there. The on-ramp is the missing piece.

Apple is making the most interesting bet on that on-ramp. At WWDC this month they finally committed to the big Siri overhaul, an assistant that can actually chain multi-step tasks, with an agent layer wired into the App Store so you can hand off things like booking a table or running your smart home. They are building it on Google’s Gemini, which tells you that even Apple decided the raw model is becoming a commodity and the product is the assistant on top. It will not ship in the EU at launch, the usual regulatory reason. I think this is the right move and it might genuinely work. Putting the agent in front of normal customers is the whole game.

Everyone’s agent is a lot of agents

Here is the part that gets skipped. When everyone has an agent, the browsing stops being human. Your software does it for you, at machine speed.

When my agent works, it touches more of the web in an hour than I would in a day. It crawls, it reads, it calls APIs, it goes and shops. Multiply that by a few hundred million people and traffic on the open web spikes. The humans did not arrive in bigger numbers. Their agents did.

That has a bill, and the bill lands in the physical world. Inference is already about two-thirds of all AI compute this year, up from a third in 2023. Data center electricity demand is climbing double digits every year. GPU prices are not coming down. For a while I expect the cost of good AI to go up before it comes down, because demand is bending faster than supply, and power and silicon are real, finite things. That is part of why I keep a local model running on a cheap Mac mini and swap its brain when I feel like it. The local one is slower and dumber than the cloud, and I keep it anyway. I want a floor under me that does not move when the market does.

The question everyone actually asks

Does it take the jobs. That is the real question hiding under all the others.

My honest read is that in the long run it makes more work than it removes. The most cited forecast going around, from the World Economic Forum, lands on roughly 170 million new roles and 92 million gone by 2030. A net gain, with about a fifth of all jobs changing shape somewhere in the middle.

A net gain is cold comfort if you are one of the 92 million. This is a revolution that asks people to move from one kind of work to another, and people do not all move at the same speed. Some are ready. Some are not, through no fault of their own. Closing that gap is a job for policy and pacing and a bit of patience, not something a model fixes. I have written before about the skills that hold their value and about what we should even be teaching kids now that AI writes the code. I do not have a clean answer. I have a direction. The durable move is to get good at pointing this stuff. Racing it is a losing game.

The physical half is slower

Everything above is the digital half. A personal assistant for everything that lives on a screen is close. A personal assistant for anything physical needs a body, and bodies are the hard part.

Robots are catching up faster than I expected though. 1X is shipping its NEO home robot to US homes this year, twenty thousand dollars up front or five hundred a month, with a human quietly supervising the tasks it has not learned yet. Figure has robots working a BMW line. Tesla is gutting a car factory to build Optimus. The honest timeline for a robot that is genuinely useful in a normal home is 2028 to 2032, not next spring, and I went into why in the post about inviting robots into our homes. Still, that is a couple of years out, not science fiction. Close enough that I already think about it.

We are still in the wild west

One more thing, because I think most people still underrate what is already sitting in front of them. We are in the wild west. Barely any rules, uneven tools, and a lot of folks treating an agent like a slightly nicer autocomplete.

Then last week a government had two frontier models pulled. Anthropic disabled Fable 5 and Mythos 5 for everyone on the planet after a US export-control order meant to keep foreign nationals away from the model’s cybersecurity ability, which I covered in my June opinions. They could not filter cleanly by nationality, so they switched both off for the entire world. Europe called it a wake-up call for sovereign AI.

Sit with that for a second. A government looked at a piece of software and decided it was close enough to a weapon to control who gets to touch it. Nobody controls autocomplete that way. That is the tell. These tools are already strong enough to be governed like dangerous things, and most of the people who could be using them well have not clocked it yet.

Where I actually land

So where does this go. More people get an agent that works. The web fills up with software acting for us. Compute gets more expensive before it gets cheaper. Jobs churn hard and then settle higher. Robots turn up for the physical half later than the hype promised and sooner than the skeptics will admit. And somewhere in the middle, AI stops being a label a company staples onto a product and becomes the thing quietly doing the work.

I have lived with one of these long enough to be careful with predictions. I have also lived with it long enough to know it already changed how I work, on the days it behaves and the days it does not. So I will say plainly where I land. I am pragmatically optimistic. None of this will be smooth. I am optimistic anyway, because every time I check, the floor is higher than it was the last time.

It might turn out better than we think. I would not have written that sentence a year ago. For now it is enough to keep building, both hands still on the wheel.

First, some honesty

I owe you a bit of context before this one. For the last three weeks I have been posting at least once a week, but I was so deep into doing things that I got completely lost in experimentation. New things, broken things, rebuilt things. I had no extra time to turn any of it into a proper post, and I have always tried to be transparent about that trade. I focus on doing things rather than writing about them. The writing is a byproduct of the doing, and I would rather keep it in that order.

So no promises here. I am not announcing a new cadence or turning over a new leaf. I keep building, I keep trying to send you the most valuable parts from my point of view, and you are always invited to discuss it and build along.

This one is a catch-up post. Several things happened at once and I had no opportunity to address any of them. Treat it as the light version of my posts, in the same spirit as the April opinions roundup. I hope you will like it.

Oh and BTW. THANKS FOR 3200 people reading me! Greatful to be here!

The billing split lands in three days

The thing closest on the calendar first. On June 15 Anthropic splits its subscription billing: programmatic use of Claude, so the Agent SDK, headless claude -p runs, GitHub Actions, anything spawned from a script instead of typed by a human, stops counting against your plan limits and moves to a separate monthly credit, metered at standard API rates.

One correction to my own earlier framing, because I went through the fine print again this week. I kept repeating “$200 per month” and that number is true for me, on the Max 20x plan. The credit is actually tiered: Pro gets $20, Max 5x gets $100, Max 20x gets $200. Enterprise Standard seats get exactly nothing. So when you do your own math, do it against your plan, because the spread between $20 and $200 is the difference between a toy budget and a real one.

The mechanics matter more than the headline number, honestly. Credits are per-user and cannot be pooled. Unused credit does not roll over. You have to claim it once in your account before June 15. And the part I would put in bold if I were Anthropic’s documentation team: when the credit runs out, your automated requests simply stop. No queue, no automatic downgrade to a cheaper model. There is an overflow toggle that bills the excess at API rates, and it ships turned off. The default behavior of this change is your agent going silent mid-month.

I have been working around all of this for a while and I wrote a dedicated post with the four mitigations I actually wired in: Anthropic repriced my agent, four mitigations before June 15. If the deadline affects you, that post is the practical one. This section is just the reminder that the clock is at three days.

And the painful detail, the reason it is hard for me to fit into even the $200 tier: programmatic use gets counted like API usage. Real token costs, not the heavily subsidized subscription math we got used to. One widely shared gist did the napkin math and called it a 12x to 175x effective price increase depending on workload. That spread sounds dramatic, but it roughly matches what my own measurements say: the same work, priced honestly, turns out to be expensive. That gap between subscription pricing and honest pricing is the whole story of this post, actually. Keep it in mind for the Fable section.

The obvious mitigation many skips

While optimizing my own agent for this, one thing kept proving itself over and over: smart model routing. It is very obvious and almost nobody does it. If you run Opus for everything, you will hit your ceiling embarrassingly fast. Route the same request to Sonnet and it costs a fraction. I wrote about the day I switched my agent from Opus to Haiku and it got better, and that lesson aged well. Haiku still handles a surprising share of the simple requests flowing through my system.

Below Haiku there is another floor: local models. The really simple jobs, classification, triage, formatting, run on a 35B model on my $600 Mac Mini and cost exactly nothing per token. I have been measuring this stack for months now and the routing is what keeps the whole thing viable. After June 15 it stops being an optimization hobby and becomes the difference between an agent that runs all month and one that dies on the 19th.

And if your agent is model-agnostic, you are not even locked to one vendor. Other providers can supplement the expensive paths; my $20 OpenRouter fallback exists for exactly this kind of squeeze. Everything is possible here. It is all up to our creativity, I think. If you want the shortcuts I use for this, the agent playbooks on the Wiz store cover the routing setup in detail.

OpenAI is fighting dirty, and I like it

Now the fun one. Yesterday OpenAI announced on X that Codex users can now save their rate limit resets and spend them later. Their words: they heard we wanted to use resets on our own time. Every eligible account on Go, Plus, Pro and Business got one banked reset for free, and for two weeks Plus and Pro users can invite up to three friends, with both sides earning another reset when the friend sends their first Codex message.

The fine print is pure growth hack. The reward lands only when the invited friend actually starts using Codex, an invite alone earns nothing. Banked resets expire after 30 days. It is loyalty points for compute, complete with a referral program. I am half joking and half impressed, because the underlying feature is genuinely user-friendly: months of complaints about resets firing at fixed times, often in the middle of someone’s night, and OpenAI responded by handing the timer to the user.

Read the timing too. Anthropic is three days from making programmatic use more expensive, and OpenAI responds by making its limits more flexible and literally giving spare capacity away. This is a very direct shot in the subscription war, and it is totally something Anthropic would never do. One company is tightening the meter, the other is letting you carry your unused minutes to next month, like a mobile operator from 2005.

I have my history with Codex, including cancelling it and coming back once already, so I am watching this with sympathy for both sides. Competition like this is the only thing that keeps our subscriptions honest.

A few days with Fable 5

And then there is the new model. Anthropic shipped Claude Fable 5 on June 9, a new tier above Opus. The construction is unusual: there is a twin called Mythos 5, the same underlying model, and the difference is safety plumbing. Fable runs classifiers, and when a request looks like offensive cybersecurity, dangerous biology and chemistry, or an attempt to distill the model, the response silently falls back to Opus 4.8. Mythos skips the cyber guardrails and goes only to approved organizations, government cyber defenders and the like. Anthropic says more than 95% of Fable sessions never touch the fallback. For what it is worth, in a few days of agent work I have not knowingly hit it once.

I have been running Fable 5 with my agents since launch. Short version: it is good. Much better than Opus for agentic work, definitely. It is more proactive. It pushes forward with the tasks I give it instead of stopping at the first checkpoint to ask how I feel about things.

The thing I appreciate most is how deep it goes. This was always my quiet complaint about the Opus line: Opus tries to be specific and on the spot, and sometimes that means it misses the full context around the problem. It fixes the line you pointed at. That is also why I always liked Codex with GPT-5.5 a little more for audits, because it reads the whole module before it touches anything. Fable 5 works like that. It audits around the problem. It goes very deep into the code, it references things I did not mention, and it holds up over long runs better than anything I have used.

One concrete example. I gave it some architecture changes on my AI agent, the kind of task where Opus usually needs a round of feedback from me halfway through. Fable ran longer than Opus would have, and when I checked the outcome I had genuinely no notes. It did what I wanted and in a few places a little more. That almost never happens.

The benchmark numbers are loud, and I would hold them loosely. Anthropic claims state of the art nearly across the board: 80.3% on SWE-Bench Pro against 69.2% for Opus 4.8, and a story about Stripe migrating a 50-million-line Ruby codebase in a day, work estimated at over two team-months. Vendor numbers, vendor anecdotes. For balance, Endor Labs ran an independent security benchmark where Fable came out middling, and they documented benchmark contamination in 19% of their test instances. My hands-on lands somewhere between the marketing and the skepticism: the long-run agentic improvement is real, I can feel it in my own work. The superlatives I leave to the launch page.

Now the catch, and you already know where this is going: the cost. Fable 5 is priced at $10 per million input tokens and $50 per million output tokens. That is double the Opus sticker price, although to be fair it is also less than half of what the Mythos Preview was going for. Right now it runs inside our subscription usage limits, but only until June 22. On June 23 it leaves the subscription pool and you pay for it with usage credits. Anthropic says the removal is temporary and that they aim to bring it back into plans when capacity allows. We will see. The internet is predictably furious about the two-week tease, and I get it: handing everyone the best model for free and then putting it behind a meter is a very effective way to make people feel the gap.

And it burns tokens fast, faster than anything I have seen, because being thorough is exactly what costs tokens. The depth I praised three paragraphs ago is the same property that empties the budget.

So my take: Fable 5 is a specialist tool, at least at this price. Security work, maybe. Big architecture passes, long autonomous builds, the tasks where one excellent run beats five cheap ones. For most jobs Opus remains my default, and the routing logic from the section above does not change. It just gets one more expensive tier at the top to route to, sparingly.

Where I think this is going

Everything is getting pricier at the top while our subscriptions still look generous. I do not think that lasts.

Look at the line Anthropic actually drew with the billing split, because it is sharper than “interactive versus automated.” Chatting on claude.ai stays flat-rate. Interactive Claude Code in your own terminal stays flat-rate. But run Claude inside Zed, a fully interactive session with a human typing every prompt, and it bills against the credit anyway, because it arrives through the Agent SDK. The meter follows the integration surface, not the human. First-party surfaces stay subsidized. Everything you build or plug in yourself becomes usage.

That is the pattern I see everywhere now: the soft cutoff. In the billing split, in Fable 5 leaving the subscription pool after two weeks, even in OpenAI gamifying its limits with expiry dates. Nobody will take your flat-rate plan away. They will just keep moving the best things outside of it, one model and one surface at a time.

Although that sounds gloomy, I do not really mind it. Honest prices force better engineering, and better engineering is the part I enjoy. The builders who learn to route work to the right model will keep their costs flat while the quality ceiling rises. That is the game now, and it is a game you can actually win.

A few months ago I gave Wiz, my AI agent, a one-line idea: a browser card game where every card is a number. Balatro-ish, roguelite, drag numbers into zones. Then I mostly walked away from it.

This week I opened it again. I remembered it as a rough prototype. A combat screen, maybe a couple of enemies. I was wrong. Sitting there was a full roguelite: two acts, fourteen enemy types including two multi-phase bosses, relics, curses, events, a meta-progression layer. It is live at wiz.jock.pl/ten-ish/. You can play it right now.

That gap, between what I remembered building and what was actually there, is what this post is about.

What the game is

The core is small on purpose. Your deck is ten cards, numbered 1 through 10. Each turn you draw five and drag them into three zones: Attack, Defense, Ability. A number in Attack deals that much damage. In Defense it blocks. In Ability it triggers an effect, if you have one.

That is the skeleton. Everything grows on top of it. Cards pick up traits, so a 7 is prime, a 10 is high, and negative cards behave like their own little trap. Enemies react to those traits and cycle through stances that telegraph what is coming next. Your deck mutates with modifiers, curses, and the occasional card you wish you had not picked up. Like any roguelite, the real game is what your deck looks like fifteen fights in.

What I keep sitting with is how little I actually did

I did not write Phaser code. I do not really know Phaser. I told Wiz “browser card game, roguelite, numbers as cards” and it chose the renderer, the bundler, the scene graph, the combat resolution. I reviewed. I steered. I argued about balance. I did not implement.

Two years ago I was already doing this, before anyone slapped the name vibe coding on it. Back then, to get anything useful out of it, I needed a lot of tokens, a lot of time, and a lot of prompting. I would describe a thing five different ways and still get back something I had to mostly rewrite.

Now I do not. I need to know the idea. I need a direction. I need to know how it should feel and look. That is basically it. The model fills the enormous space between “numbers as cards, make it tense” and a working drag-and-drop combat loop with a turn forecast and a combo system. I noticed the same floor drop out from under app-building when I told it to ship an app a day. Same shape, different project.

Where vibe coding actually is, for me

I do not know where vibe coding is going. People are still excited about it like it is a new trick. Although I get the excitement, for me it stopped being a trick a while ago. It is just the default now. It is how I build. The novelty wore off. The usefulness did not.

This game is a small, raw example of that. I would not call it a case study, it is rougher than that. A person with minimal game-dev knowledge and a vague direction ended up with a playable roguelite, because the models got good enough to carry the distance. The interesting part is the distance the model covered on its own, more than the game itself.

Building your own small things still matters to me, even when, maybe especially when, the agent does most of the typing. And the reason a half-forgotten side project could quietly turn into a real one is that Wiz keeps improving the things it touches in the stretches when I am not looking.

What I asked for this week

When I opened it again, three things bugged me. So I gave Wiz three directions. Not specs. Directions.

“Give it a story, all the way to the end.” It had none. Now there is a premise. Reality runs on one great Ledger. Something divided by zero, the books stopped balancing, and the old machines that kept the count turned on the very digits they were built to serve. You are a hand of numbers that refused to be erased. The Act 1 boss is The Abacus. The Act 2 boss is The Equation. Beat it and you get an actual ending with real closing lines, instead of the bare “RUN COMPLETE” screen it had before. I wrote none of that. I described the world in three sentences and Wiz built the prologue, the act transitions, the boss intros, and the payoff.

“Make it stable.” It found a real bug I would never have caught on my own: a turn where your hand has no playable card could soft-lock the whole run. Fixed, plus a couple of defensive guards on empty states that I would not have thought to check.

“Clean up the rewards.” After a fight you were getting offered junk cards that did nothing for your deck. Now you get one card that is an actual choice, sitting next to the useful options.

None of that was a specification. It was a sentence each. The implementation was the model’s.

What still needs me

Balance, completely. Wiz generates numbers for everything and none of them arrive tuned. The Equation at 100 HP across three phases was too hard to even reach for a long stretch. I spend more time in the balance constants than on any single feature.

Feel. The animations existed but they were stiff until I described exactly what a card snapping into a zone should feel like. The model does not know what you want a player to feel in the half-second a shield flashes. You have to notice that gap first, then point at it.

And taste. What the game is about. Whether a mechanic is tense or just annoying. When to stop adding things. I have shipped enough side projects this year to know the pace has a cost, and that knowing when to stop is its own skill. The floor dropped. The ceiling is still mine.

Go play it

ten-ish is at wiz.jock.pl/ten-ish/. No install, no account, runs in a browser. Try to reach The Equation.

If you want the unglamorous part underneath a build like this, the basics of using Git with an AI builder are what make it safe to let an agent keep rewriting your code while you sleep. And when a side project outgrows what it started as, sometimes the honest move is to rebuild it, which is a different judgment call than starting one.

The patterns that make builds like this fast are in the Mini-App Starter Kit: five working mini-apps, the architecture decisions already made, ready to fork. $39.

Wiz is my personal AI agent. The build infrastructure I use for projects like ten-ish, and the way I work with it, is most of what I write about in Digital Thoughts. Paid subscribers get the playbook templates and starter kits I make along the way.

Subscribe now

Last week I sat down with Wyndo from The AI Maker for a Substack Live. The topic was one specific agent I have been running quietly for months: a job finder I built for the people closest to me. Wyndo just published his writeup of the show, and it is generous and clear, and I am grateful for the room he gave me to actually show the thing.

This is the other side of that conversation. The builder’s side. What is inside the agent, why I built it the way I did, what surprised me about running it for real humans, and the smallest version someone could build this weekend.

Let me start with why this one matters more to me than most things I have shipped.

Why I Built It

Someone close to me was looking for a new role and doing what most people do. Scanning LinkedIn alerts. Filtering through noise. Fighting the algorithm’s idea of what they should want. The signal-to-noise was awful. Five alerts a day, maybe one was worth opening, and that one was already three days stale by the time it arrived.

The job market is not a problem of supply. There are roles. The problem is alignment, and alignment is exactly the kind of work an agent should be doing while you sleep.

I gave myself one rule before writing a line of code. The agent does not apply for anyone. The human judgment, the cover letter, the decision to spend an hour on a specific company, that stays with the person. The agent’s job is upstream of all of that. It is the filter and the explanation. The person opens an email in the morning, reads three roles with a clear reason for each, and either replies or applies. That is the whole loop.

Like, the smallest useful version of this idea is already a real product if you build it carefully.

The Loop, In One Paragraph

Every morning at 6:15, the agent wakes up. It loads the person’s profile, picks which job sources are due based on a tier and a cooldown, browses each one, scores everything it finds against the profile, writes an email with the best three to five roles and a one-line reason for each, and sends it. Throughout the day, the person can reply to the email. A short “not this kind of role, too senior” or “more like this one please.” The next morning’s search uses that reply.

That is the agent. Profile in, daily email out, reply-driven refinement. Everything else is plumbing.

What Is In The Profile

The agent is only as good as the profile it reads. This is the part I underestimated for the first two weeks, and it is the part that did the most work once I took it seriously.

For each person I run the agent for, there is a single source-of-truth file that looks something like this:

• Current situation. What they do now, employment status, when they can start. Two or three lines, not a CV.

• Target lanes. Not one role. Three or four. A strong candidate fits more than one pattern, and the agent should respect that. For a creative leader the lanes might be Head of Creative, in-house Creative Director, Brand Creative Lead. For an e-commerce operator the lanes might be VP e-commerce, Digital COO, Head of Digital Transformation. Lanes catch reality.

• Geography rules. Hard rules. Remote EU first, then named hub cities, then the rest. Anything outside the allowed list gets rejected before it even scores.

• Salary floor and target. A floor and a target in one currency. Below the floor, the role is rejected unless the company is on a tiny aspirational list. Without a floor, the agent will dribble out underpaid roles forever.

• Dealbreakers. Concrete things, not vibes. No alcohol, no tobacco, no gambling. No must-have language other than the ones the person actually speaks. Industries that have been tried and disliked.

• Positive examples. Three to five roles the person would actually want. Real job posts, pasted in. The agent uses these as reference points when it scores. Concrete examples beat any prompt I could write.

The profile is a markdown file. That is the entire format. The agent reads it the way Wiz reads its own CLAUDE.md when it wakes up, with the same discipline: top of context, every run, before any decision.

The Search, In Tiers

Most people building a job agent for the first time make the same mistake I made. They try to search everything every day. That gets you rate-limited fast, costs money, and produces noise.

I run sources in three tiers with a per-source cooldown. LinkedIn is tier one, every day, because it is where the volume lives. Tier two is the major aggregators that have decent role pages, on rotation, two or three per day, with a cooldown so the agent does not pound the same site. Tier three is the company career pages the person actually cares about, listed in their profile. Those run on a longer cooldown because their pages do not change as often.

Three different tools handle the actual fetching:

• Firecrawl for clean job-page extraction. It returns markdown, which the agent reads directly.

• Web search through Claude for broad first-pass discovery.

• Playwright for the sites that need a real browser, which mostly means LinkedIn behind an authenticated session.

None of these is magic on its own. The reason it works is that the agent picks the right tool for the right source, and the cooldowns keep any one of them from becoming the bottleneck. I went deeper on the harness side of this in my post on agent coding harnesses if you want the broader picture.

The Scoring

Every candidate role gets scored on a 0-10 against the profile. The score is not a slider the agent moves around. It is a small set of rules the agent applies in the same order each time.

1. Lane match. Does the title plus the JD fit one of the lanes in the profile? If no, the role is out.

2. Geography. Is it in an allowed location, with the right remote rules? If no, the role is out.

3. Language. Are the must-have languages on the whitelist? If no, the role is out.

4. Salary. Floor first, then target. Below floor and not on the aspirational list, the role is rejected before scoring.

5. Fit reasoning. Why this role for this person, in one sentence. The agent has to write the sentence to keep the score.

6. Concerns. What might be a mismatch. Also one sentence. If the agent cannot name a real concern, the role is probably overhyped.

Anything that scores six or above makes the morning email. Anything below six does not. The number is not a serving suggestion, it is a hard gate. I would rather get two roles tomorrow than five mediocre ones.

The Email

This is the part the friend sees, so this is the part I obsess over.

The morning email is three to five roles. For each role: title, company, link, a one-sentence reason it fits, a one-sentence concern, salary if listed, location, and a suggested next action. That is the whole thing. No promotional framing, no agent personality, no apology when the day is quiet.

If the day is quiet, the email does not arrive. The agent logs a quiet day and goes back to sleep. I learned this one the hard way. An empty digest is worse than no digest, because it teaches the person to stop opening the email. The right move when there is nothing to send is to send nothing.

I write more about how I built the email layer in the post on knowing my agents are actually working. The short version: the email itself is the user interface, so it gets the same care as a product.

The Reply Loop, The Part I Did Not Expect

I thought the search and the scoring would be the interesting parts. They were not. The reply loop was the interesting part.

People do not reply to job alerts. People do reply to a personal email that asks them a real question. So the email closes with a short note: not relevant? Reply with one line and tomorrow’s search adjusts. No form. No button. Just reply.

When the reply arrives, the agent does three things. It classifies the feedback. It applies the change. It updates the profile.

If someone replies “this role at a gambling company, never,” the agent does not just skip that role. It adds gambling to their dealbreakers, permanently. If a reply says “this is too senior, I want builder-track not exec,” the agent shifts the lane weights. If a reply says “more like this one please,” the agent saves that role as a positive example, and the next day’s search leans that direction.

This is the same architecture I described in the post about my self-improving agent: corrections in, classified, graduated into permanent rules when they stop being a one-off. The job finder is the cleanest example of that loop I have built. The feedback is short, the surface is one email, and the change is visible the next morning. People notice when their agent listens.

What I Showed On The Live

Wyndo asked me to do three things on the stream, and I think this is the right order if you ever demo an agent of your own.

First, the email. Before anything else. He had me open a sanitized morning brief and read it out. The audience does not need to see the code yet, they need to see the output. Five roles, a reason for each, a concern for each. That is what the friend opens. If you cannot show that first, the rest of the demo will not land.

Second, the profile. I showed the markdown file with the lanes and the dealbreakers and the positive examples. This is where people get the idea. Most of the audience comments came in during this part. Oh, so the agent uses the JD against the profile? Yes. The profile is the thing.

Third, the reply. I showed one fake feedback message and walked through what the agent did with it. Classified, applied, saved. The audience watched the profile file change. That was the moment of the show, the part I think Wyndo wrote about as “the part where the agent learns from you.”

I did not start in the terminal. I did not show subagents or scheduling or memory. None of that mattered for the demo. The agent loop is profile, search, score, email, reply, and the demo is best when it walks that loop in the order a person would experience it.

The Smallest Useful Version You Could Build This Weekend

If you want to build this for yourself, do not start where I started. Start tiny.

1. One profile file with current role, one target lane, geography rules, salary floor, three dealbreakers, three positive examples. Markdown. That is it.

2. One search source to start. Pick the one site you actually use today. LinkedIn if you can authenticate, Indeed if you cannot.

3. A scoring prompt that takes a profile and a JD and returns a 0-10 score with a one-sentence reason and a one-sentence concern.

4. A morning email with the top three roles that scored six or above. Send it to yourself first. Run it for a week before sending it to anyone else.

5. A reply rule. If you reply, the agent reads the reply and updates one thing in the profile. One thing. Resist the urge to make this fancy.

You can build the whole stack on a Claude Code project with a daily cron, a single CLAUDE.md, and a few hundred lines of glue. I went through the basics of agents like this in the first AI agent guide, and the same shape applies here. If you would rather get the structured starter than wire one up from scratch, the agent playbooks on my store include the patterns I lean on for this kind of personal automation.

Once you have run it for yourself for two weeks, then you can run it for a friend.

What I Would Avoid Automating

I want to be clear about the part I will not build, no matter how much someone asks.

The agent will not apply. It will not autofill a form, not click submit, not generate a cover letter and send it. Auto-apply is a temptation because it looks like leverage. It is not. The cover letter that gets the interview is the one the person writes after thinking about the company. Removing that step removes the part that matters. There is also a real cost on the other side: a recruiter reading twenty AI applications gets worse at recognising the human ones. Auto-apply degrades the channel for everyone using it.

I would also not chain the agent into “and if the role fits perfectly, schedule a call.” That is the failure mode I wrote about in the bounded agent post. You want a clear edge between the agent’s territory and yours. Discovery and explanation on the agent side. Decision and action on the human side. The line stays clean and the agent stays useful.

What This Cost Me, Roughly

People asked on the live. The honest answer is that running the agent for one person, with morning searches and a reply pipeline, lands somewhere around a couple of dollars a day on a Claude subscription. Running it for several people is a multiple of that. Cost is not the constraint here. The constraint is the quality of the profile.

If you do want to think carefully about the cost side, especially after Anthropic’s Agent SDK pricing change on June 15, the math gets sharper. A daily run that lives inside the interactive Claude Code session stays on the subscription. A daily run that goes through the SDK lands on the new separate credit. Pick the mode you actually want before you commit to a cron schedule.

One More Thing About Building For The People You Love

The reason I keep coming back to this agent is not the technical loop. It is what happens when a friend writes back two weeks in and says “the email got useful.” That sentence is worth more than any benchmark.

You build something for the people closest to you, and it changes the way you think about what an agent is for. Not productivity. Not scale. Not a leaderboard. Just one person, one morning, three roles, and a reason for each. If the agent does that well, it is doing the job.

Thank you to Wyndo and Dheeraj for the room on the One Shot Show, and to the people who showed up to watch. Watch Wyndo’s full writeup for the audience-side view of the session. If you want me to walk you through a profile of your own, reply to this post. That is how it starts.

I took a week off from writing. The gap in the feed is mine.

It was not a wasted week. I did a live on Substack about the job finder agent with and → Agent I built for my friends and my family, the one that is helping them pursue new opportunities. The replay should be up on Sunday and I am genuinely happy with how it went. It was the calm half of the week.

The other half was doing mode. The news I am sharing in this post is the reason. And I am also sharing four solutions you can try before June 15, because that is the part I wish someone had handed me yesterday morning.

Here is the short version of the rollercoaster I just got off.

In April, my Claude Code usage on the Max plan got noticeably worse. Same prompts, same agent, much less runway before I hit the limit. I am not the only one who felt it, the forums are loud about it, and Anthropic itself acknowledged a capacity squeeze. So I did the thing I have been writing about for a while. I added Codex Pro to the stack and let it carry a chunk of the load. Different vendor, different harness, same agent jobs underneath. Codex earned its keep fast, and after two months running both side by side, I had real evidence that GPT-5.4 inside the Codex harness held up as a peer for most of the work my Wiz throws at a model.

Then Anthropic got a lot of compute back. They closed a deal with SpaceX for the Colossus 1 data center, more than 300 megawatts and over 220,000 NVIDIA GPUs coming online within the month. Stacked on top of a 5 GW Amazon agreement, a multigigawatt agreement with Google and Broadcom, $30 billion of Azure capacity with Microsoft and NVIDIA, plus a $50 billion American AI infrastructure plan through Fluidstack. That is real compute. On May 6 they doubled the Claude Code five-hour limit for Pro, Max, Team, and seat-based Enterprise, and they permanently removed the weekday peak-hour throttle on Pro and Max. On May 13 they added a 50 percent weekly limit bump on top, running through July 13.

I was about to drop Codex. Sincerely. I was writing notes about consolidating back to a single subscription. With the new ceiling, my workload fit comfortably under Max.

Then last week, Anthropic published the Agent SDK plan change. And that is where it stops being nice.

What Actually Changes On June 15

Starting June 15, 2026, every programmatic use of Claude gets decoupled from your subscription plan limits. Concretely:

• Claude Agent SDK (the Python and TypeScript SDKs)

• claude -p, the non-interactive print mode of Claude Code

• Claude Code GitHub Actions

• Third-party apps that authenticate via the Agent SDK with your Claude subscription

None of those count against your plan after June 15. They draw from a separate monthly Agent SDK credit. Pro gets $20 per month. Max 5x gets $100. Max 20x (the plan I am on) gets $200. Team Premium $100 per seat. Enterprise Premium $200 per seat. The credit refreshes monthly. It does not roll over. It is per user, not poolable across a team. And it requires a one-time opt-in to activate, after an Anthropic email scheduled for around June 8.

What stays on the plan: interactive Claude Code in the terminal or IDE, web chat, the mobile app, and Cowork. So if you sit at the keyboard and prompt Claude, that lives on Max. If your agent prompts Claude, that lives on the SDK credit.

When the credit runs out, two doors. If you have enabled extra usage, overflow flows to standard API rates on top of the subscription. If you have not, the request halts until the next monthly reset. You can read the official terms on the Claude help center page.

On paper, $200 a month sounds like free money. The framing is generous. The math is not.

Why $200 Is A Drop In The Sea For A 24/7 Agent

Let me show what $200 actually buys when you run an agent that wakes overnight, runs through the day, and spawns subagents for parallel work.

At current rates, Claude Sonnet 4.6 is $3 per million input tokens and $15 per million output tokens. Claude Opus 4.7 is $5 input and $25 output per million. A modest agent call with around 50,000 input tokens and 5,000 output tokens lands at about $0.225 on Sonnet and about $0.375 on Opus, before prompt caching helps and before any reasoning tokens or tool calls inflate the count.

That is one call. A real agent shift is not one call. My overnight shift runs a planning pass, then it spawns several worker tasks in parallel, each of which calls Claude several times to use tools, read files, write changes, and produce a verifiable result. A single “go do this overnight job” can fan out into twenty or thirty model calls. Some of those are cheap. Some, when the agent decides it needs Opus to think harder about a refactor or an audit, are not.

Round it generously and a serious overnight run lands somewhere between $2 and $8 in API costs. Daytime wakes add more on top. The math at one $5 shift per day puts you at $150 a month, before any actual ambition. Push the ceiling and the $200 credit is gone in two weeks, easy. After that, every call lands at standard API rates on top of the subscription I am already paying for. Or the agent stops.

This is not a hypothetical. I have written about my token bills before. When I finally started measuring properly, the waste was embarrassing, and I still found that even the disciplined version of my agent burns more tokens than the credit would cover at full ambition. The whole reason I am on a flat subscription is that I do not want to be priced per call. Pay-per-token is a tax on building anything that runs while I sleep.

Who Gets Squeezed First

I want to be careful here. Anthropic is not a charity, and they have made the case (their CFO said it on the record, and several outlets picked it up) that some accounts were running thousands of dollars of API value through a $200 subscription. Splitting plan usage from SDK usage closes that gap. From a business angle, it is a clean move. I get it.

What I am less sure they fully internalized is who gets squeezed first. The casual user lives at the keyboard, gets the doubled limits, gets the 50 percent weekly bump, and is genuinely better off than a month ago. The squeeze lands one ring out, on the practitioner. The agent builder. The person running an autonomous Wiz overnight, or the small team using Claude Code GitHub Actions to ship code without a human in the loop on every step. The people whose entire workflow is programmatic Claude. The most committed builders on the platform are the ones whose costs go up the fastest.

And those are exactly the people who are most likely to have, or to quickly add, a second harness.

Four Mitigations You Can Try Before June 15

None of these are theoretical. I tested all four this week and the first one is a working prototype on my Mac Mini. Read them as a stack, not as alternatives. The serious mitigation is layered.

1. Drive Interactive Claude Instead Of claude -p

This is the one I am most excited about, and the one I am least ready to publish in detail.

The interactive Claude Code session stays on your subscription after June 15. Only the print mode and the SDK move to the credit. So the question is whether you can take a job that currently runs as claude -p "do X" and instead run it as a real interactive session that opens, receives a prompt, executes the work, writes the result somewhere you can read, and closes itself. If you can, that work stays on Max forever, no SDK credit touched.

I have a working prototype as of last week. The shape: a small Terminal automation that opens Claude in a real terminal window via AppleScript, pastes the prompt through the system clipboard (Cmd+V into a bracketed-paste handoff), waits for the result file to land at a known path, and closes the window. End to end in under three minutes for a real task. I tested it by having the spawned session send an iMessage and write a JSON file, and both arrived clean.

I am not publishing the full script yet because the rough version has edge cases that will bite anyone who copies it. The production version needs to handle session cleanup, concurrency, the headless display setup, and a handful of failure modes I have only seen once. I will write that one up properly when it is solid.

Honest caveat. Anthropic will probably patch the easy version of this. Driving an interactive TUI through clipboard paste is a clever workaround, not a sanctioned integration, and the moment it gets popular enough to register as a leak in the billing model, there will be a fix shipped against it. I am building this as a six-to-twelve-month bridge, not as permanent architecture. The orchestrator pattern (one job, one window, one result file) will keep working. The specific input-injection trick might not. Plan for that.

2. Move Work To A Second Harness

The cleanest mitigation, and the one I have already paid for, is to run a second harness on a different vendor and let it carry whatever fits.

I added Codex Pro under duress in April. The story I told myself was “this is temporary, I will consolidate once Claude stabilizes.” That story was wrong, and the wrong part was the framing. The Codex experience itself was fine (it is good, the GPT-5 series is good inside that harness, the experience is different but real). Diversification was never going to be temporary. I just did not see it yet.

The same thing happened to me on the inference layer. I opened an OpenRouter account because Claude had a bad morning and my agent had nowhere to send the request. Two days later I described that account as half insurance, half extension. Insurance when the primary fails. Extension for capabilities I deliberately keep off the primary stack. The $20 of credits in OpenRouter is leverage I did not have before, for less than a coffee a week.

Codex Pro is the same shape on the harness layer. It is insurance on the days Anthropic has a capacity problem, an outage, a billing change, or a release that changes my workload economics under me. It is extension on the days everything is fine, because the OpenAI models are genuinely good at certain things, and running both lets me pick the right tool per task. The subscription buys deliberate architecture. I have written before about what the two harnesses actually feel like after months of real use. This is the mitigation that does not depend on Anthropic leaving any door open. They cannot patch your second subscription.

3. Route Narrow Calls Through OpenRouter Or A Small Local Model

Not every model call needs the smartest model on the planet. A surprising amount of my agent traffic is narrow stuff: classify this message, summarize this file, extract these fields, decide between two tool options. That work runs fine on Haiku, on GPT-4o-mini, on a cheap Gemini Flash call, or honestly on a small local model on my Mac.

So the third mitigation is to actually look at what your agent is sending to Claude and ask which calls deserve Opus, which can drop to Sonnet, and which can leave the Claude billing surface entirely. OpenRouter is the easiest way to route the “leaves Claude” set, because you keep one API and pick the model per call. A small local model on a Mac Mini is the easiest way to route the “leaves the cloud” set. Both shave real money off the SDK credit pressure without changing the shape of your agent.

Quick aside. The cross-provider routing I run on my agent is packaged on my store as the AI Model Switcher. Same logic, same triggers, same identity prompt budget. If the June 15 change is making you think about diversification for the first time, this is the rung that gets you most of the way there.

4. Audit And Trim Before The Deadline

The least glamorous mitigation, and the one I would do first.

Spend an evening this week looking at what your claude -p usage actually does. Count the calls. Bucket them. How many are doing real cognitive work? How many are doing the same thing over and over (and could be cached, batched, or merged)? How many fire because of a cron job nobody has reviewed in three months? When I did this on my own stack I found a cron that was waking an agent every fifteen minutes to check a Discord state that almost never changed. That alone was several dollars of SDK credit per week, sitting on a default I had never questioned.

The waste is real. I have written about my own token waste before, and the only thing that fixed it was measuring properly. If you trim before June 15, you both shrink the credit pressure and learn where to focus the first three mitigations.

A Small Add-On If You Do Not Want To Wire This Yourself

While I was writing this, I packaged the four mitigations into a small add-on for the Claude Code agent you already run. It is called the Claude SDK Audit Kit, and it costs $9.99. Paid yearly subscribers get it for free, the rest of you get it at the price of a sandwich.

What it does. You hand the kit to your Claude Code, say “run the audit,” and it walks your repo, your cron, and your launchd plists, finds every claude -p call and Agent SDK import, estimates monthly cost on each plan, and writes a migration plan you can act on before June 15. The same four mitigations from this post are included as ready playbooks (with the AppleScript pattern, the Codex setup checklist, the OpenRouter routing recipe, and the trim labels), plus a migration spec template, a worked example from my own stack, and twelve months of updates as Anthropic ships follow-up pricing changes.

I am being honest that this is a thin add-on, not a flagship kit. The audit scripts are simple. The playbooks are short. The value is timing. If you wait until July to audit, your June bill already hurts. If you would rather build this yourself, the four sections above are enough. If you want it ready to run tonight, that is what the kit is for.

Claude SDK Audit Kit

One more honest note. Mitigation 1 (driving interactive Claude) is probably a six-to-twelve-month bridge, not a forever solution. Anthropic will patch the easy version of that trick at some point. The kit treats it that way, and so should you. The other three mitigations do not depend on Anthropic leaving any door open.

What I Take From The Rollercoaster

The doubled limits felt good. They are real. The weekly 50 percent bump is real. The compute deals behind it (SpaceX, Amazon, Google, Microsoft and NVIDIA) are gigawatts of capacity being built right now. Anthropic is scaling, and they are pricing the scale in a way that fits their business. Fair enough.

Although the lesson for builders is the one I keep relearning. Single-vendor architecture is a comfort tax I do not want to pay. The cheapest, smallest, most embarrassingly small thing I can do to protect my agent is to keep a second harness alive on a different vendor and to keep my own stack honest about what it actually needs.

I will keep Codex. The $20 in OpenRouter stays. The interactive Claude bridge gets built properly over the next month, knowing it is a six-to-twelve-month bridge and not a forever solution. And the next time I catch myself writing “I do not need this second tool anymore,” I will read this post back to myself and put the credit card away.

The Practical Takeaways

• The billing split is real and starts June 15, 2026. Agent SDK, claude -p, Claude Code GitHub Actions, and third-party SDK apps move off your plan limits. Monthly credits: $20 Pro, $100 Max 5x, $200 Max 20x. Opt-in required after the Anthropic email around June 8.

• The credit is generous in absolute terms and small for serious agent operators. A modest 24/7 agent will drain $200 in days, not weeks.

• Overflow flows to API rates if you enable extra usage. Otherwise the request halts until reset. Pick deliberately; do not get surprised.

• Mitigation 1: Drive interactive Claude. Interactive sessions stay on the subscription. The hack works today. Expect Anthropic to patch the easy version inside a year.

• Mitigation 2: Second harness. Codex Pro is the cleanest insurance. They cannot patch your second subscription.

• Mitigation 3: Route narrow calls elsewhere. OpenRouter, Haiku, a small local model. Stop sending classification work to Opus.

• Mitigation 4: Audit and trim. Cheapest mitigation. Do it first.

• Anthropic is scaling, hard. The compute is being built. The capacity is real. The pricing change reflects business reality. Plan accordingly.

The agent will keep running. The architecture under it will be a little less elegant and a lot more honest.

Claude Code Workshop

I track every billing change, model swap, and cost optimization across my live agent stack. The caching layers, model routing, and fallback architecture that keep costs reasonable are covered in the Claude Code Workshop. Updated with the June 2026 billing split patterns.

$39 at wiz.jock.pl/store. Free for paid subscribers.

At 04:17 this morning my agent shipped one ugly line into the error registry: Wake failed: supported providers exhausted.

That is the worst error I can get. It means the primary failed, the model cascade inside the provider failed, the cross-provider hop failed, and the agent had nowhere left to send the request. Everything was my fault (because I am the architect of this thing). Although it cost me about thirty minutes to chase down the root cause, I really do not mind it. The agent did not go silent on me. It went silent on itself, queued the task, retried on the next cycle, recovered. That is the whole point of a fallback mechanism.

The reason that line stays rare and not common is one decision I made on day one of running an agent that wakes up overnight: park $20 of credits in OpenRouter. Not as a cost. As insurance. When my primary stack has a bad morning, that $20 absorbs the hit and the agent keeps running. And on the days when everything is healthy, the same $20 doubles as an extension cord for capabilities I deliberately keep off the primary AI agent architecture: image generation, long-context refactors, cheap classification. Insurance when things break. Extension when they do not.

This post is about the fallback mechanism itself: what it is, why every serious agent needs one, why local llm is not enough on its own, and what you actually buy with the $20. I will show you the rungs of my stack, the trigger conditions that flip between them, and the ~40 lines of Python that hold the whole thing together. Oh and one more thing - this is not an ad for Open Router. I just enjoy using it.

Why Fallbacks Matter More Than People Want To Admit

People building their first agent skip this step. I get it. You are focused on the cool part. The model is working, the prompts are clean, the agent shipped its first task. Fallbacks feel like a problem for later.

Then later arrives.

Top labs go down. Heavy load and fast shipping cycles tend to do that to a service. In the last 30 days, all three frontier providers have had a public bad day. Anthropic’s status page logged elevated errors on May 5 and again on May 12, this time tagged against Claude API specifically. OpenAI had a roughly 90-minute outage on April 20 that surfaced as 8,700+ Downdetector reports in the UK and 1,900+ in the US. Google Gemini had a widespread degraded window on May 5 too, the same day Anthropic was having a hard morning.

That is the part nobody talks about. A bad day at one frontier lab often lines up with a bad day at another. The herd of “they will not all be down at the same time” is partially true. It is also partially false on any given Tuesday.

Claude API’s published 90-day uptime sits at about 98.99% (the Anthropic status page reports this directly). Sounds great until you do the math. 98.99% over 90 days is roughly 21 hours of downtime. If your agent runs on a schedule, like mine does overnight, it runs during some of those hours.

Outages are one bucket. Here is the rest of what can knock a single-provider agent flat:

• Rate limits. You hit a TPM or RPM ceiling mid-task. The agent sits in retry-backoff hell while the work piles up.

• Auth failures. OAuth token expires at 03:00. The nightshift dies at 03:01. Do not ask me how I know.

• Regional issues. A region degrades while the rest of the world is fine. Your traffic happens to land on the bad one.

• Model deprecation. An older model gets retired with two weeks notice. You forgot to migrate the one cron that still calls it.

• Capability gaps. Your primary does not generate images. Or does not have a long-context variant. Or does not have a cheap classification model.

• Cost spikes. A loop misbehaves at 2am and burns through credits on the most expensive model in your stack.

• Vendor lock-in. The day a provider raises prices or changes terms, you want options, not a migration project.

A fallback mechanism fixes most of these at the same time. That is why it is the cheapest piece of resilience you can ship. And once you have it, the second half of the value (the extension side) shows up almost for free.

Why Local LLM Is Not The Answer On Its Own

I went deep on local. I ran a 35B model on a $600 Mac Mini. I built a local agent. I measured what closes and what does not between local and frontier. I love the work and I will keep doing it. And I almost fried my Mac Mini trying to push the local tier too far, so I have receipts on the failure mode too.

Although I have to be honest about what local does well and what it does not.

Local is great for: classification, redaction, summarization, tight tool calls, “is this email worth waking me up,” local-only privacy work where the data must not leave the box, anything where the task is narrow and the prompt is bounded.

Local is not great for: anything I trust Opus to do overnight. Multi-step reasoning. Long-context refactors. Voice-sensitive writing. Anything where the wrong answer is worse than no answer. When I asked a local 8B to draft a comment in my voice, it produced something that read like a different person. When I asked Opus the same thing, it sounded like me on a good day. That is the gap.

Like, I want local llm to run on normal hardware, not on a $10k Mac Studio. That is why local is the cheap layer for the right kind of work. Routing a sensitive task to a small local model just because the cloud is down is “completes with a wrong answer” instead of “fails cleanly,” which is worse, not better. So local is one layer in my stack. The smart layer is something else.

The Tool I Use For This Layer

The mechanism needs an implementation. This is the one I picked.

One API. More than 400 models behind it, from over 60 providers, as of May 2026. You top up credits, generate one key, and pick the model per request as a string: anthropic/claude-opus-4.7, openai/gpt-5, google/gemini-2.5-pro, meta-llama/llama-3.3-70b-instruct, and so on. Same key. Same SDK shape. One bill.

The pricing is published on a clean fee announcement: 5.5% on credit-card top-ups (with a $0.80 minimum), 5% on crypto. Per-token prices pass through at provider cost on most models. Bring-your-own-key gives you the first 1M requests per month free; after that, a 5% surcharge applies. So BYOK is not a free escape hatch forever, it is a generous free tier on top of bringing your own bill.

One myth worth killing while we are here: it used to be true that Claude on OpenRouter carried a meaningful markup vs Anthropic-direct. As of May 2026 the current Anthropic models (Sonnet 4.6, Opus 4.7) are priced identically on OpenRouter to the Anthropic API, $3 input and $15 output per million tokens for Sonnet, same as direct. The historical markup hung around the older Claude 3.5 Sonnet rate card. If you have not checked recently, check.

The interesting math is below Claude, not on top of it. Meta’s Llama 3.3 70B on OpenRouter sits at about $0.10 input and $0.32 output per million tokens. That is roughly 30x cheaper on the input side and 47x cheaper on the output side than Sonnet 4.6 at $3 / $15. The classification step in your pipeline does not need Sonnet. It needs Llama or Haiku. OpenRouter lets you make that choice per call, not per project.

The Five Rungs Of My Stack

Here is the resilience layer in my agent, top to bottom. Each rung has a trigger that flips to the next one.

1. Primary call. Claude Opus 4.7 for complex work, Sonnet 4.6 for default, Haiku 4.5 for cheap fan-out. The model is picked per task based on the task’s stakes, not globally.

2. In-provider cascade. If Sonnet 5xx’s or times out, the harness retries on Haiku before bailing on Anthropic. Cheap, same provider, often the recovery is invisible.

3. Cross-provider hop. If the whole Anthropic surface is unhealthy, I run the same prompt through Codex (GPT-5 via the OpenAI CLI). Different vendor, different harness, same job. This is the one that earned its keep when I built the model switcher.

4. OpenRouter degraded mode. If both vendor CLIs are unreachable (network, auth, status pages red), the watcher scripts call OpenRouter directly with a stripped-down identity prompt and a small open-weight model. The reply is prefixed with [Fallback Mode] so I know what I am reading.

5. Queue and retry. If everything is on fire, the task gets stamped with a retry timestamp and re-tried on the next cycle. The agent never just drops a task.

Rungs 1, 2, 3 are model and harness routing. Rung 4 is OpenRouter doing the work the harness cannot. Rung 5 is the safety net under all of it.

One thing to call out about rung 3: the cross-provider hop is the most important rung in practice, because most outages are full-vendor outages, not model-specific. If Anthropic is down, it is usually down for everything. Hopping Sonnet to Haiku does not help. Hopping to Codex does. The April-May 2026 incident pattern (Anthropic May 5 plus May 12, OpenAI April 20, Gemini May 5) is exactly the case for diversity at the vendor level, not the model level.

How To Actually Wire It Up

The setup is embarrassingly small. Make an account at openrouter.ai. Top up $20. Generate a key. Drop it in your agent’s secrets file (mine lives at global/secrets/openrouter.md, never in code).

Then the fallback path itself. I will show the pattern I actually use, simplified for the post. It is about 40 lines of real Python.

OPENROUTER_URL = “https://openrouter.ai/api/v1/chat/completions”
FALLBACK_MODEL = “google/gemini-3-flash”  # cheap, fast, reliable
FALLBACK_NOTICE = “[Fallback Mode] Primary unavailable. “

def call_primary(prompt):
    # your normal Claude / Codex / Bedrock call
    ...

def call_openrouter(prompt):
    headers = {”Authorization”: f”Bearer {load_key()}”}
    payload = {
        “model”: FALLBACK_MODEL,
        “messages”: [
            {”role”: “system”, “content”: load_identity_prompt()},
            {”role”: “user”, “content”: prompt},
        ],
    }
    resp = requests.post(OPENROUTER_URL, json=payload,
                         headers=headers, timeout=60)
    resp.raise_for_status()
    text = resp.json()[”choices”][0][”message”][”content”]
    return FALLBACK_NOTICE + text

def reply(prompt):
    try:
        return call_primary(prompt)
    except (TimeoutError, ProviderError, AuthError) as e:
        log_fallback(reason=str(e))
        return call_openrouter(prompt)

That is the shape. The interesting bits are around it.

Trigger conditions. Not every error should flip to fallback. A 400 (bad request) is your bug, not the provider’s. A 429 (rate limit) should retry with backoff first. Real fallback triggers are: timeouts, 5xx server errors, 401 auth errors after a single re-auth attempt, repeated 429s past your retry budget, and the explicit “all providers down” signal you build into your harness. In my stack the trigger flags live in automation/lib/resilience.sh.

Identity prompt budget. When I fall back to a smaller open-weight model, I do not send the full Claude system prompt. I send a stripped-down identity prompt (SOUL.md, ~1.3KB) plus a “fallback coda” that tells the model how to behave under degraded conditions. The big primary system prompt is too long for a 9B model to handle in 120 seconds. Tailor what you send to the model’s context budget.

Cache the system prompt. The system prompt is identical across calls. Cache it (file read once per process, or actual provider-side prompt caching where supported, which I wrote about under token waste management). You will burn tokens if you reload it on every request.

Visible degradation. The user always knows when the response came from a fallback. The [Fallback Mode] prefix is non-negotiable. The worst pattern is a silent quality drop where the user thinks they got Opus and got a 9B. Tell the truth, every time.

Cost gate per task. Some tasks are not worth $0.30 of Opus tokens. Route low-stakes work to google/gemini-2.5-flash or meta-llama/llama-3.3-70b-instruct by default. Llama 3.3 70B at $0.10 / $0.32 per million tokens is roughly an order of magnitude cheaper than Sonnet on output, and for classification or redaction it is more than enough. Reserve the expensive model for work that earns it.

Test by killing the primary. The only way to know your fallback works is to test it. Once a quarter I invalidate the Claude key for ten minutes during a scheduled wake and watch the agent respond through OpenRouter. If it does not, the bug is mine, not the day I have an actual outage.

The Extension Half: Capabilities I Keep Off The Primary Stack

Insurance is why I opened the account. Extension is what kept me using it.

The $20 sitting in OpenRouter is not just backup for when the primary breaks. It is also where the agent goes for things I deliberately do not want native in its primary architecture. Anthropic does not generate images, and I do not want another SDK, another auth flow, and another billing line living inside the core agent loop just to support a header image. So image generation lives on the extension side. The agent calls Gemini 2.5 Flash Image (Nano Banana) and Nano Banana Pro through the same OpenRouter key when a blog post needs a header or a Forge prototype needs a mockup. From the agent’s point of view it is just another dispatch. From my point of view the agent grew an arm.

Same logic for evals (compare three models on the same prompt without standing up three SDKs), for the long-context work that does not fit cleanly in Claude’s window, for the cheap reasoning passes I do not want to spend Opus tokens on, and for the open-weight curiosity calls I want to make without standing up a whole new vendor relationship.

Each one of those is something I could fold into the primary stack. None of them earn that complexity. They live on the extension side instead, on the same $20 that already pays for the insurance half.

What I Would Tell You If You Were Starting Today

If you have zero fallback wired right now: open the OpenRouter account this week. Wire one fallback call against your existing primary. Test it by killing your primary key for ten minutes and watching the agent respond. The whole thing is one afternoon of work.

If you already have an agent in production: count how many places in your code have a hard dependency on a single provider’s SDK. Each one is a future outage you will wear personally. Wrap them.

If you have been waiting for local llm to be “good enough” to be the fallback: local is the cheap layer for narrow work. OpenRouter is the smart layer for everything else. Use both, in that order, for the right kinds of work. The compounding wins from an autonomous system come from leverage; the losses come from a stack that has not been stress-tested.

The $20 is not money I lost. It is insurance against the morning my primary has a bad hour, and an extension cord for the capabilities I deliberately keep off the primary stack. Both halves earn their place on a quiet week, and the loud weeks pay for the quiet ones a hundred times over.

The agent is going to keep waking up at strange hours and trying to ship work while I am asleep. Some of those mornings the primary will be down. I want the worst line in my error log to stay rare. That is what the insurance half is for. And on the calm mornings, that same $20 lets the agent generate images, run cheap classification passes, and reach for a long-context model when one is needed. Half insurance. Half extension. All of it for the price of a dinner.

Share

Wiz Store

AI Agent Night Shift Playbook

The $20 I spend on OpenRouter is the cheapest production resilience I know. How to wire it up, when to trigger it, and what not to route through it is one chapter in the Night Shift Playbook. The full fallback architecture, LaunchAgent setup, and safety boundaries are in there.

$19 at wiz.jock.pl/store. Free for paid subscribers.

The setup, because this only works if the rest of the stack is calm

I have been going through more changes on my AI agent recently. I have been transparent about that here as I go, post by post, and today I want to write about the one layer I depend on the most. But I have to start with how I got to the point of being able to ask “how does my agent actually learn from me?” That part of the story is a little messy and I think it matters.

When I started this project in October 2025, the first thing I built for the agent was its own task manager. A control panel, a dashboard. I went deep on it. I built it native on iOS, native on macOS, and as a web app, all wired together. It worked. For two or three months it was genuinely great.

The problem with self-made software is that you have to maintain it. There is no version of “I reached a level of polish I was happy with, and then I forgot about it.” The dashboard needed constant feedback. What should it show? What should it hide? Where was it pulling data from this week that it had not been pulling last week? It was also burning more tokens than I wanted to think about. So I switched. I moved to a small open-source kanban called Fizzy with a thin shim of my own. That was a quieter setup that I held for a while, and I wrote about the move in detail in the post on replacing my custom dashboard.

Fizzy was good. I was still struggling with one thing though. I needed to be able to orchestrate the agent and also see the projects I was working on from a longer distance. Day-to-day kanban is one job. Stepping back to see what was actually shipping over a month was another. So I made a small personal scratchpad of my own called experiments.jock.pl. It is not for everyone, not everything I am working on is on it, but it gave me a place to lay out the experiments I had in motion at a higher altitude than the task list. That helped, but it was still mine to maintain, and I had the same problem I had with the original dashboard.

What actually solved it was a tool I have used for years and had stopped thinking about. Basecamp. They shipped a dedicated CLI for agents recently, and the whole picture clicked for me. The CLI is what makes the agent side work. The other half of why it clicked, on my side, is the card table inside Basecamp. It is essentially the same clean kanban I liked in Fizzy, but built in. I get the lens I was rebuilding by hand, plus everything else Basecamp does, plus the CLI, all in one place. The agent can read projects, comment on cards, file new ones, complete them, all from the same place I am working. I have tried a lot of pieces of AI infrastructure in the last year and most of them are good enough. This one feels different. Another level, honestly. I can see the whole stack of work at the right altitude. I can move things around. If something is a bigger project I carve out a separate space for it. The board does what I would have spent two more months building for myself, and it does it better.

This is the setup I have been settling into over the last few weeks. The short version is that I have been replacing my custom software with shims on top of mature tools, and so far the replacements keep winning. I write about why I still keep building most of my own stack in building your own things is cool too. The corrections loop is one of the things that only became visible once everything else around it had calmed down.

What corrections actually look like, when you work with an agent every day

OK. On to the actual subject.

When you work with an agent every day, most of the time you are not writing prompts. You are watching the agent do something and quietly thinking “no, not quite like that.” Then you say so. Five words. “I would not link that.” “Use plain text here.” “Stop confirming every step.” Each of those is a correction, and the unspoken contract between you and the agent is that you should not have to say it twice.

The best systems for this are the ones that catch corrections without you having to do anything special. You correct in chat, in your normal voice, and behind the curtain the system decides “this is something I should think about for the future,” files it where it belongs, and makes sure the next session that boots on this machine knows about it. You do not stop and write documentation. You do not open an admin panel. You just keep working, and the agent keeps absorbing.

That is what I have been building for the last few months. The corrections loop is the part of the agent that decides what to do with the small “no, not like that” moments and where to file them so they outlive the session they happened in. It is the layer I depend on the most, because it is the one that makes the agent feel like an actual coworker instead of an autocomplete. It is also the layer that makes the agent slowly start to feel like more of you, rather than more of the model.

A quick word on how the agent started

For context. My agent started in October 2025. Almost everything about it was rough back then. Sometimes the output came back cold, sometimes it just did the wrong thing in a polite way. I used to write very long prompts to deal with that. I would describe the task, then add a paragraph at the end explaining how I wanted it done, what tone I wanted, where to file the output, what to skip. Every session, over and over. The output was usually good when I did all of that. The cost was that I had to do all of that every time.

That is not a stable way to work. It scales for the first week and then you get tired of writing the same paragraph again. The thing that quietly changed everything was the agent gathering enough data on me, both from the work we had done together and from the corrections I had made along the way, that the explaining paragraph slowly stopped being necessary. It is still there in some shape. It just lives in files now, not in the prompt window. The agent walks into the room already carrying it.

The corrections loop is the part of that I want to focus on, because it is the one piece you can copy without copying everything else.

The architecture, in three stages

The pipeline is named the way most of my plumbing is named, badly and on purpose. Capture, classify, graduate.

Capture. The moment the agent spots a correction in chat, any session can call a single helper:

python3 automation/self-improve/correction_capture.py add \
    --text “<the correction>” \
    --source cli \
    --context “<what I did>”

That writes one line to a JSONL queue. It also opens a card in Basecamp so I can see the correction landed somewhere and so I can comment on it. No model call. No retries. Capture has to be cheap, or the agent will silently stop doing it under pressure.

Classify. The same helper passes the message through a small regex map. Seven patterns mapping to six kinds. The kinds are skill_misuse, memory_update, behavioral, rule, preference, and unknown. “Stop doing X” comes out as rule. “I prefer X” comes out as preference. “You used the X tool wrong” comes out as skill_misuse. Each kind has a default action attached, so the next stage knows what to write. The patterns themselves live in correction_capture.py lines 50 to 92. They are short, and writing them taught me what corrections actually look like at scale better than any post I could read on the topic.

Graduate. Every night a separate process drains the queue. For each pending entry, it picks the right place to file the artifact, writes it, and only then marks the entry resolved. The rule, baked into the agent’s own playbook, is a correction never expires unaddressed. If the nightly drain cannot fully handle one, it has to leave it pending with a note. It is not allowed to silently drop one.

That last line is the part that took me the longest to actually believe in. Queued things age, in any system. Once one ages enough, the agent stops feeling like it learns and starts feeling like it just covered the easy stuff. Forcing the queue to either drain or escalate is the only way I have found to keep that from happening. The nightly drain is part of a wider overnight job loop on my machine. The corrections drain is one of the cleanest jobs in that loop.

A real recent example. One night Atlas, the agent persona that does research for me, returned a list of hallucinated Reddit thread IDs. None of the URLs resolved. A correction landed in the queue, classified as memory_update. By the next morning there was a new feedback memory file with a single rule attached. Atlas cannot hit reddit.com directly (403). Fetch via Firecrawl or browser-playwright first, then pass verified URLs. Every Atlas-flavored session that has booted since has loaded that line. Same failure has not come back.

“Memory” is one word covering four jobs

Here is the part nobody writes about.

When you start, “memory” feels like one thing. You imagine a notebook the agent keeps. You imagine writing into it. You imagine retrieval. That is the abstraction every product page uses, and it is the wrong abstraction. Atharv Malve put it cleanly last summer. The model is not really remembering your past messages. It is just seeing the history again, every single time. Once you internalise that, you stop looking for the one memory feature and start asking what you actually need stored, by whom, for how long.

What I actually needed turned out to be four different sinks. They are not interchangeable. Learning the differences was half the work.

Sink one is working memory. Short-lived. The current week’s plans, the half-finished thoughts, the active conversation context. Lives in a single small file called memory.md. It is supposed to decay. Treating it as durable is the original sin.

Sink two is lessons. Full incident logs. When something goes wrong in a way I want a future session to learn from, the lesson lands in lessons.md with the trigger, the root cause, the fix, and a list of keywords. I have 274 lines of these going back to February. They read like engineering postmortems, because that is what they are. The public version of this file is roughly the mistakes anthology I wrote last month.

Sink three is feedback memories. Per-rule files in a durable memory directory. Each one is a single rule with a “Why” line and a “How to apply” line. Linkable, deletable, deduped. When the same correction comes up twice, the second time it gets its own file. It also gets a tiny pointer in a master index that the agent always loads on startup. Two-level indirection, so the index stays small.

Sink four is rule lines in the always-loaded index. These are the ones I wake up next to. A handful of **RULE: ...** lines at the top of the master index, all caps, the smallest set of behaviors I refuse to relitigate. “Verify deliverables. Show proof or keep task open.” “Match work topics against existing WizBoard tasks and complete them when done.” A rule earns its place at this level only after it has come back more than once.

And then there is the sink I did not plan for and would not give up now. The Behavioral Learning card table inside Basecamp. My WizBoard project has a small card table on it called Behavioral Learning, and every single correction the agent captures lands there as its own card. I can read the card, push back on it, fold two cards into one, or trash one that is wrong. Corrections become reviewable, not silent. That part matters. I will say more about why in the next section, but the short version is that if you let the model grade its own corrections in private, you have already lost.

If you take one thing from this post, take this. “Memory” as one concept is the wrong abstraction. Build sinks for different lifespans. Working memory is fast and disposable. Lessons are slow and durable. Feedback memories are searchable rules. Top-level rules are non-negotiable. The Behavioral Learning card table is the human-in-the-loop that keeps the rest honest. Different jobs, different files, different decay curves. I wrote about how rules in particular shape an agent in the bounded agent. Most of “memory,” once you look at it long enough, turns out to be rules.

Does it actually work?

Yes and no.

Here is what I can see in my own metrics. I have an autonomous improver that runs nightly and writes a metrics.json with a seven-day window, a thirty-day window, and a longer view. As of this morning, my agent received 22 corrections in the last 30 days. In the last 7, that number is 18. The trend line is down, and the system flags it explicitly with valence: good. Errors total across all categories is also drifting down, by less.

I do not want to present this in a single direction. The task success rate is 93.5 percent over 30 days, with a small dip in the last seven, from 93.5 to 92.6. So I am not going to pretend the picture is clean. Some weeks the agent gets worse. The point is that the corrections themselves are showing up less often, and when they do show up they are landing in places I can act on.

What the corrections actually look like, beyond the totals, is more interesting. A separate analyzer scans the captured corrections for repeating themes. As of this morning it has flagged two. One it calls incomplete, which is me catching the agent finishing a task that was not fully done. The other it calls repeated_mistake, which is a fix that came back. The analyzer is also allowed to propose a new rule when it finds a theme strong enough, and both of the rules it proposed have already graduated to the top-level RULE lines I quoted earlier. “Verify deliverables. Show proof or keep task open.” came out of the incomplete pattern. “ESCALATE if same mistake recurred. Strengthen the rule or fix the trigger.” came out of the repeated_mistake pattern. That is the loop closing on itself, in real data I can read off the file.

One more honest note. Thirty days of declining corrections is not proof of generalization. It is a trend on one user, on one workload, on one machine. The agent could be getting quieter rather than smarter. The way I keep myself honest about that is the Behavioral Learning card table I described. I see every correction. I can see which kinds keep coming back. The bar I am holding myself to is “fewer repeats of the same mistake,” not “an agent that never breaks.” On that narrower bar, the data is encouraging.

Measurement of this kind is also why I cared so much about token cost a few weeks ago. If you cannot count what your agent is doing, you cannot tell whether it is improving or just drifting. I wrote about that in the post on token waste on Opus 4.7. Same instinct, different file.

What would break it, and what I would build next

The fragile part is the classifier. Seven regex patterns is enough to label most corrections, but unknown still shows up too often to ignore. When an entry lands as unknown, the nightly drain picks it up, but the action it should take is no longer automatic. The fallback is that I or one of my future sessions has to retag the row by hand. Replacing the regex with a small LLM call would solve the labeling problem and create two new ones. Latency and cost. It would also create a softer problem, which is the one that scares me more.

If you let the model grade its own corrections in private, you get an agent that learns the wrong lessons confidently. Yohei Nakajima wrote about this risk in his note on better ways to build self-improving agents. His phrasing is the one I keep coming back to. The model can hallucinate bad reflections and reinforce them. That is the failure mode for any self-improving loop. The Behavioral Learning card table is what keeps that from happening on my setup, and it is the part I would build first if I were building this for someone else.

There is one more bigger picture thing. The reason I can talk about this layer with confidence is the architecture it sits inside. I wrote the long version in my AI agent knows who I am earlier this year, where I walked through the ten layers I use to make the agent feel coherent over time. The corrections loop is one of those layers. It is the one I depend on most, because it is the one that most directly changes the agent’s behaviour rather than its memory.

If you have not yet noticed one of your own fixes coming back at you, you will. When you do, the move is not to add more memory. It is to build a small queue, decide what your sinks are, make sure no correction can quietly age, and then put a human-readable surface on top of it so you can see what the agent is teaching itself. The agent that comes out the other side of all that does not just remember more. It starts behaving like more of you.

Subscribe now

Self-Improving Agent Kit

The corrections loop, lesson archival, and self-heal architecture from this post are packaged as the Self-Improving Agent Kit. It includes the exact files Wiz uses, plus a setup guide for wiring it to your own agent without having to build all this from scratch.

$49 at wiz.jock.pl/store. Free for paid subscribers.

If you’ve ever lost an hour of progress in a game because you forgot to save, you already understand why Git exists.

You’re deep in a dungeon. The boss took 40 minutes. You made one wrong move, got killed, and your last save was way back at the start of the level. That hour is just gone. No trace of what you tried, no checkpoint to return to, nothing.

Building software without version control feels exactly the same. Especially when AI is part of the building process.

I’ve been running my own AI agent since late 2025. It builds things, makes decisions, modifies files, runs overnight. It also makes mistakes. Sometimes it introduces a bug deep in the architecture and I wake up to something that doesn’t work anymore. Without proper commits, I’d have no idea what changed. With them, I open the history, read back through what happened, and roll back to the last clean state in under a minute.

This post is for people who are starting to build with AI tools, vibe coding with Cursor or Claude Code or Codex, or running their first experiments with autonomous agents. Git probably sounds like a developer thing. It is. It’s also one of the most useful habits you can build as a builder of anything, regardless of how technical you are.

First: Git is not GitHub

This confusion trips up almost everyone who starts. I had it for longer than I want to admit.

Git is a tool. Software you install on your computer. It tracks changes to your files over time and saves snapshots of your project whenever you ask it to. It’s free, open source, and runs entirely on your machine. It has nothing to do with the internet. Git was created in 2005 by Linus Torvalds (the person who also created Linux) and has become the standard for version control across the entire software industry.

GitHub is a website. A cloud service that stores your Git repositories remotely. A place to back them up, share them with others, and access them from anywhere. GitHub is owned by Microsoft and is where most public open-source code lives.

The relationship is like the difference between a text file and Google Drive. The file exists on your machine whether or not you upload it anywhere. Git works whether or not you ever create a GitHub account.

Why does this matter? Because GitHub is not your only option, and I think a lot of people avoid the whole topic because they assume it means signing up for something owned by Microsoft and making their work public. Neither of those things has to be true.

The main alternatives worth knowing:

• GitLab: the most comprehensive alternative. Does everything GitHub does (repositories, issue tracking, code review) plus built-in CI/CD pipelines for automated testing and deployment. Can also be self-hosted on your own server if you want full control. Good option if you want more features baked in.

• Codeberg: run by a nonprofit organization based in Germany. GDPR-native from the ground up, no data selling, and they explicitly don’t train AI models on your code. Free, donation-funded, no ads, no tracking. If privacy and data sovereignty matter to you (especially if you’re in Europe), this is the serious alternative.

• Forgejo: open-source and self-hosted. You install it on your own server and run your own Git hosting. Lightweight, modern interface, GitHub-compatible. If you want complete control over your code and have a machine to run it on, this is the path.

• Bitbucket: made by Atlassian, integrates tightly with Jira and Confluence. If your team is already using those tools, Bitbucket fits naturally.

All of these speak the same Git language. Every command I’ll show you in this post works on all of them. The choice of platform is about where your code lives, not how you use it.

I use GitHub because the ecosystem is built around it and my AI tools (Claude Code especially) integrate with it well. But if you have strong reasons to go elsewhere, you’re not missing anything technically.

Why I started actually caring about this

I’ve known about Git for years. I ran commits occasionally. I wasn’t disciplined about it.

That changed when I started building an agent that runs overnight.

The setup is that the agent works autonomously while I sleep. It builds features, writes scripts, modifies configuration, creates tasks for itself. Most nights this is productive. But early on, I’d wake up to something broken and have no clear way to understand what had changed. The agent had touched 12 files across 3 directories and something downstream was misbehaving. I was staring at a broken system with no map back to working.

I fixed this by building commit discipline into the agent. It now commits after every meaningful action. When I wake up and something is wrong, I read the commit history. I see exactly what changed, when, and in what order. I can roll back to the last clean commit in under ten seconds, or read forward through the commits to understand what went wrong and patch it with that knowledge.

This is what most people miss when they think of version control as “backup.” It’s not just backup. It’s a navigable history. It’s the difference between saving a file and saving a timeline. With a timeline, mistakes become investigations instead of disasters. I wrote about a lot of those investigations in the post about how I almost broke everything.

Setting up your first repository

This will take less time than you think. Let me walk through exactly what to do.

Step 1: Install Git

On a Mac, open the Terminal app (search for it in Spotlight) and type:

git --version

If you see something like git version 2.39.0, you already have it. If not, the easiest path is to go to git-scm.com and download the installer. On Mac you can also run brew install git if you have Homebrew installed.

On Windows, download the installer from git-scm.com. It includes a terminal called Git Bash, which is what you’ll use to run the commands below.

Step 2: Tell Git who you are (one-time setup)

Git tracks who made each change. Before you do anything, set your name and email:

git config --global user.name "Your Name"
git config --global user.email "you@example.com"

You only do this once. It doesn’t create an account anywhere. It just labels your commits.

Step 3: Initialize a repository

Navigate to your project folder in the terminal and run:

git init

Git creates a hidden folder called .git inside your project. That folder is the entire history of your project. All your commits, all the metadata, everything. You never need to open or touch it directly. Your project is now being tracked.

If you want to verify it worked, run git status. You’ll see a list of your files as “untracked” (Git sees them but hasn’t started tracking their history yet).

Step 4: Make your first commit

A commit is a snapshot, your first save point. Two commands:

git add .
git commit -m "Initial setup"

git add . stages all your files, which means “include these in the next snapshot.” The dot means “everything in this folder.” You can also add specific files with git add filename.py if you only want to commit some changes.

git commit -m "message" saves the snapshot with your description. That description is the commit message. We’ll talk about what makes a good one in a moment.

To confirm it worked, run git log. You’ll see your first commit listed with a timestamp and your name.

Step 5: Push to a remote host (optional but recommended)

Your repository exists on your machine right now. To back it up to GitHub (or wherever), you need to create an empty repository there first, then connect your local one to it.

On GitHub: click the “+” icon at the top right, choose “New repository,” give it a name, and make sure you do NOT check “Add a README” (you want the empty repository). Copy the URL it gives you.

Then run these two commands:

git remote add origin https://github.com/yourusername/your-repo.git
git push -u origin main

git remote add origin tells your local Git where the remote copy lives. git push -u origin main uploads your commits there. The -u flag sets this as the default remote for future pushes, so after this first time you just run git push.

That’s the whole setup. From here, your workflow is: make changes, add, commit, push. Those three steps are 90% of what you’ll do.

What to add to .gitignore (and why)

Before you commit your actual project files, you need to talk about .gitignore.

This is a file that tells Git which files and folders to never track. You don’t want passwords, API keys, or large auto-generated files in your version history. Once something is committed to Git and pushed to a remote, it’s there forever (even if you delete it later, it’s in the history). So you exclude sensitive things upfront.

Create a file called .gitignore in your project root. For most AI agent projects, this is a good starting point:

# Environment variables and secrets
.env
.env.local
secrets/
*.key

# Python
__pycache__/
*.pyc
*.pyo
.venv/
venv/

# Node.js
node_modules/
npm-debug.log

# macOS
.DS_Store

# Editor files
.vscode/settings.json
.idea/

# Large generated files
*.log
dist/
build/

The most important lines: .env and anything in a secrets/ folder. If you’re using AI tools like Claude Code, you likely have API keys stored somewhere. Those should never go into Git. Add them to .gitignore before your first commit.

If you accidentally commit a secret and push it: change the key immediately. The history is visible even after deletion.

When to commit

Most beginners commit too rarely. They work for three hours, then push “made some changes.” That’s nearly useless as a history. Here’s how I actually think about it.

Commit before anything big. If you’re about to let Claude Code refactor a major section of your project, commit first. If the refactor goes sideways, you can undo the whole thing with one command: git reset --hard HEAD. This is the most valuable habit I’ve developed. Before I hand something big to the agent, I save my current state. No exceptions.

Commit after anything that works. Feature works? Commit. Bug fixed? Commit. Even small wins. Each commit is a checkpoint you can return to. There is no such thing as committing too often.

Commit with meaning. This is where most people lose the value of their history. A commit message is documentation. “Fixed auth bug where tokens expired before session timeout” is infinitely more useful than “fixes.” When you’re debugging something three weeks later, whether it’s you, someone else, or an AI agent reading the log, those messages are what makes the history useful instead of just a list of timestamps.

A simple format that works well:

# Good commit messages
git commit -m "add rate limit guard to external API calls"
git commit -m "fix memory compression when context exceeds 200 lines"
git commit -m "checkpoint before refactoring auth flow"

# Less useful
git commit -m "updates"
git commit -m "wip"
git commit -m "stuff"

Commit before you sleep. If your agent runs overnight, give it a clean starting point. Whatever state your project is in when you go to bed, commit it. If something goes wrong at 3am, the history starts from a known point.

On active agent architecture work, I commit every 15 to 30 minutes of real progress. Some sessions have 20 commits. This is not excessive. The checkpoints are frequent enough that no single mistake costs more than a few minutes of work.

Reading the history

Knowing how to read your commit history is as important as knowing how to write it. These are the commands I use most:

# See all commits, newest first
git log

# More compact view (one line per commit)
git log --oneline

# See what actually changed in the last commit
git show HEAD

# See what changed between two commits
git diff abc1234 def5678

# See which files changed in a commit
git show --stat abc1234

When Claude Code starts a debug session on my project, one of its first moves is git log --oneline. It reads back through the recent commits to understand the context: what was built, when, and why things changed. This is the moment where good commit messages pay off. If the last ten commits say “add rate limit guard,” “fix memory compression,” and “checkpoint before auth refactor,” the agent can quickly build a mental model of recent work. If they all say “wip,” it’s starting from zero.

You can also browse your commit history on GitHub’s web interface if you’ve pushed your code. Go to your repository and click “N commits” at the top of the file list. Each commit shows you the message, the author, the timestamp, and a full diff of what changed. This is genuinely useful for non-technical team members who don’t use the terminal.

Private vs. public: my 90/10 approach

About 90 percent of my repos are private. I want to address this directly because I’ve seen people feel guilty about keeping their work closed.

Private doesn’t mean hiding. Most of my private repos are private because the work is genuinely messy. Unfinished. Half-ideas with rough code that works but embarrassingly so. Agent architecture that’s in constant flux. Projects I’m building toward something but haven’t figured out what yet.

This is normal work. Version control is for you in this context. You get all the benefits: the history, the rollbacks, the tracking. You don’t owe anyone visibility into your process while you’re still figuring things out.

The public repos are things I’m actually proud of or that other people can genuinely use. The one I keep pointing at is the Agent Wellbeing Kit, boundaries and nudges for AI agents and their humans. It has eight stars, which I find quietly satisfying. It’s there because I built something clean enough that it adds value for others. That’s the standard I hold public work to.

Contribute when you can. But don’t let the idea that “real developers make everything public” stop you from using version control privately. Most professional work is private. Most early work is messy. Both are fine.

Share Digital Thoughts

Working alone vs. with others

The workflow changes meaningfully depending on whether you’re solo or in a team. Worth understanding both even if you’re only doing one right now.

Working alone

When you’re the only person on a project, the simplest workflow is pushing directly to main. There’s no one else whose changes could conflict with yours. Commit often, push regularly. That’s enough.

I sometimes create branches when I’m testing a bigger experiment. A branch is just a separate line of development that doesn’t affect main until you merge it back. To create one:

# Create a new branch and switch to it
git checkout -b experiment-new-memory-system

# Do your work, commit normally
git add .
git commit -m "try new memory compression approach"

# If it works: merge it back to main
git checkout main
git merge experiment-new-memory-system

# If it doesn't: just delete it, no harm done
git branch -D experiment-new-memory-system

The branch approach is especially useful when you’re handing off an experiment to an AI agent. You give the agent a branch to work on, let it build and commit freely, then review what it built before merging to main. Clean separation between “work in progress” and “known good.”

Working with others

With a team, branches and pull requests become mandatory. No one pushes directly to main. Here’s the standard flow:

1. Create a branch for your feature or fix

2. Do the work and commit to that branch

3. Push the branch to GitHub: git push origin your-branch-name

4. Open a Pull Request on GitHub, a formal request to merge your branch into main

5. Someone else reviews it, leaves comments, approves

6. Merge to main

The PR review step is what protects main from broken code. It’s also where the real collaboration happens: someone might catch a bug you missed, suggest a better approach, or just ask a clarifying question about what the code is doing.

Even when I’m working solo on a bigger feature, I’ve started creating PRs for myself. The description field becomes documentation: why this was built, what problem it solves, what I considered and rejected. That context is genuinely useful six weeks later when I’m trying to understand a decision I made. And when an AI agent reads your repo to understand what to do next, a well-written PR description gives it context the commit message doesn’t.

Worktrees: the unlock for AI agent builders

This section is for people who are already running AI agents and want to understand the next level. Skip it if you’re still on step one; you can come back.

When I’m working with multiple agents in parallel (which happens when you’re building complex things), there are sometimes three or four branches active at once. One agent is building a feature. Another is fixing a bug. If I had to constantly switch the entire project directory between branches, I’d lose context constantly.

Git worktrees solve this. A worktree is a separate folder on your machine that’s linked to the same repository but checked out to a different branch. They share the same history and .git folder, but each has its own working directory and independent state.

# Create a new worktree for a feature branch
git worktree add ../feature-auth -b feature/auth main

# See all your active worktrees
git worktree list

# Clean up when done
git worktree remove ../feature-auth

With worktrees, I can run two Claude Code instances at the same time: one in ~/my-project (main work), one in ~/feature-auth (isolated branch). Each agent commits to its own branch with zero interference. I merge when each piece is done.

This is the infrastructure behind parallel agent builds. I covered how I evaluated different AI coding tools for this kind of work in my comparison of Claude Code, Codex, Aider, and the others. Worktrees are the underlying mechanism that makes it all clean.

AI agents read your commit history

This is the piece I didn’t anticipate, and it’s changed how I write commit messages.

When Claude Code starts a session on my project, one of its first actions is reading repository context: the file structure, the current state, and often the recent commits. A history with meaningful messages gives the agent a map of what happened and why. A history full of “wip” and “checkpoint” entries tells it almost nothing useful.

This plays out concretely when something breaks. When I start a debug session after my agent did something unexpected overnight, Claude Code often goes to git log as an early move. It reads through the last 10-15 commits. If those commits say things like “add rate-limit guard to external API calls” or “fix memory compression when context exceeds 200 lines,” it can quickly narrow down what might have changed. If they all say “wip,” it’s starting from scratch every time.

The same is true when the agent is building something new. Reading recent commits helps it understand the patterns and conventions you’ve been using: how you name things, how you structure files, what you’ve already tried. Good history accelerates the agent’s work. Messy history slows it down.

I think about every commit message as a note to a future debugger who has no other context. That debugger might be me, might be someone else, might be an AI agent. All three benefit from the same thing: specific, honest context about what changed and why.

If you want to go deeper on what that looks like at the architecture level, the post on when my AI agent started fixing itself gets into how the commit trail feeds back into the agent’s own understanding of its own codebase.

The commands you’ll use 90% of the time

git init                       # Start tracking a folder
git status                     # See what changed since last commit
git add .                      # Stage all changes
git add filename.py            # Stage one specific file
git commit -m "message"        # Save a snapshot
git push                       # Upload to remote
git pull                       # Download from remote
git log                        # See commit history
git log --oneline              # Compact history view
git diff                       # See exactly what changed (unstaged)
git diff --staged              # See what's staged for next commit
git show HEAD                  # See the most recent commit in detail
git checkout -b branch-name    # Create and switch to new branch
git checkout main              # Switch back to main
git merge branch-name          # Merge branch into current branch
git branch -D branch-name      # Delete a branch
git reset --hard HEAD          # Undo all uncommitted changes (careful)
git reset --hard HEAD~1        # Undo last commit AND its changes (careful)
git revert HEAD                # Undo last commit but keep the history

The difference between reset --hard and revert: reset rewrites history (dangerous if you’ve already pushed), revert creates a new commit that undoes the previous one (safe always). When in doubt, use revert.

Share

If you’re using Claude Code, it handles most of these automatically. You can also just say “commit these changes with a meaningful message” and it will. But knowing what the commands do means you can read the agent’s actions instead of just watching them happen.

The thing I keep telling people

Git has a real learning curve at the start. I’m not going to pretend otherwise. The mental model doesn’t click immediately. You’ll push the wrong thing. You’ll get confused about branches. You’ll probably hit a merge conflict at some point and spend an hour untangling it.

A merge conflict happens when two different versions of the same file need to be combined and Git can’t figure out which change to keep. It looks scary. It’s not. Git marks the conflicting lines in the file, you open it, decide which version is correct, delete the conflict markers, and commit. Takes five minutes once you’ve seen it once.

The place where Git changes everything is exactly when things go wrong. The first time your AI agent does something unexpected and you roll back to a known-good state in ten seconds, you’ll understand what all of this was for. Everything I’ve been building, from the overnight agent to the various AI building experiments that broke in interesting ways, was only recoverable because of this.

Without version control you’re genuinely going in the dark. The mistakes are unrecoverable. The context is lost. With it, you can make more mistakes, faster, with more confidence, because you know you can always find your way back.

Make more mistakes. Just make them trackable.

Share

AI Agent Blueprint

Git is step one. The AI Agent Blueprint covers what comes next: wake scripts, memory architecture, bounded safety, and the patterns that make an agent actually reliable overnight. One command to set up, 15 minutes to a working agent.

$39 at wiz.jock.pl/store. Free for paid subscribers.

People ask me a version of the same question all the time. “Why are you spending your evenings building your own thing? There is already a tool that does this. There is already a framework. There is already a whole product. Why are you doing it again?”

I get this with my AI agent. I got it about my store. I got it years ago about my podcasts, about the side apps, about the marketing experiments. The question is fair. The honest answer has been the same for a while now, and I have not written it down properly until today.

I build my own things because I learn through process, not by reading. That is the short version. The longer version is the rest of this post.

Quick frame before I go further

Everything I write about on this blog is something I am actually doing, experimenting on, or testing. The agent on my Mac Mini. The store. Project Money. The smaller experiments inside both. None of it is “what I think someone should do.” It is whatever I am running this week, and what I learned the hard way last week.

The blog is the slow visible slice of that work. Most of what I am doing in any given week never makes it into a post, because I would have to publish almost every day, sometimes twice, to actually keep up. That is not feasible, so most of the work stays unwritten. The writing here is always trailing the doing, on purpose. That asymmetry matters for the rest of this post. I write about building because I am doing the building. The other way around does not interest me.

Going back to “starting things is cool”

Almost two years ago, when I was finding my way back into writing, I posted something called “starting things is cool”. It is short, a little messy, and some of the projects it mentions are no longer alive. The Suggestions App I was so excited about that summer is not even an app anymore. A handful of the things I shouted about back then have quietly disappeared.

The post is also the thing that restarted my writing. Most of what I have built since then traces back to it.

If you read it today, you will see a sentence underneath everything. I like starting things. That part is true. It is also incomplete. The piece that was implicit in 2024 and has become explicit since is that I prefer to start them myself rather than start by adopting someone else’s start. Starting and building are the same instinct from two angles. This post is the second angle.

The way I actually learn

Here is the part I do not usually lead with, because it sounds personal in a way other people do not always relate to. I am not the kind of person who learns by reading. I have tried. I genuinely envy people who can read a book on something complicated and walk away with a working mental model. That is not how my brain works. I learn through process. I have to do the thing. I have to see what breaks. I have to fix it badly, then less badly, then properly. After enough rounds of that, I actually know it.

I think about this the same way I think about how my brain handles ADHD. The shortcut that works for a different kind of mind is not the shortcut that works for mine. So I stopped fighting it.

Every pre-built tool, framework, or product is a map of someone else’s process. The map is real and useful. Walking the route teaches something the map cannot.

What you only learn from building

The thing I get out of building is harder to put in a sentence. Let me try anyway.

When you build the thing yourself, you know every variable between the start and the end. You watched each one go in. You watched them connect. You know which one is load-bearing, which one is convenience, and which one only exists because two weeks ago you had a bad afternoon and forgot to clean it up. That knowledge is not glamorous. It is the part that lets you change one small thing and get a meaningfully different outcome later. Without it, you can configure what you bought. With it, you can compose.

The mistakes are the other half. I have written about a few of the recent ones in the post about almost frying my Mac Mini. Each one taught me a perspective I would not have read about anywhere else. The pattern goes back further than the agent though. It is the same pattern from the failed apps in 2024. The same pattern from the marketing experiments before that, the podcast that did not last, the small side projects that quietly closed. Mistakes have always been where most of the learning lives for me.

This is slower than picking up the off-the-shelf option. The difference shows up in what you know afterwards.

About not rediscovering America

For most of my life, I was told the opposite of all this. Use the tools you are given. Do not reinvent the wheel. In Polish there is a stronger version of that line. Do not rediscover America for the second time. I have heard it more times than I can count. For most of those years, I half believed it.

I do not believe it anymore. The tools are very good, that part is true. The act of building the thing yourself does something to you that the tool cannot do for you. The tool is a snapshot. The act is a process. I am after the process.

An example, since the AI one is fresh

Here is one current illustration so this does not stay too abstract. Right now I am building my own AI agent from scratch. People keep pointing me to OpenClaw, which has 347,000 GitHub stars and ships with most of what I am writing myself. They point me to Hermes, open-source and ready to install. They are not wrong. If I dropped my stack tomorrow and installed OpenClaw, my agent would do many of the same things in a fraction of the time.

I keep building my own anyway. The reason is the same one I have just spent a thousand words on. I want the variables. I want the failures. I want the version of myself that exists on the other side of having built it.

The same logic applies to my store. There are platforms that would let me run a digital store in an afternoon. I built the bones of mine because the parts I most want to understand are the parts most platforms hide.

I want to be clear though, I am not religious about it. When I had spent two months building a custom kanban dashboard for the agent and then realized I could do the same job in a 94-line shim on top of an existing tool, I switched. I wrote about that here in the WizBoard pivot post. The rule I now use is simple. Build the parts you need to understand. Use the parts you do not. The trick is being honest with yourself about which parts those actually are.

Reading “starting things is cool” again, from now

I went back and reread the original essay last week, before writing this one. I wanted to see what held up.

The Suggestions App is gone. A few of the projects I was excited about back then are gone. Some of my predictions about how AI would land in normal life were either wrong or right for the wrong reasons. That part of the essay aged badly.

What surprised me, reading it again, was how much of the underlying pattern actually held up. Starting things is still cool. The act of starting was the thing that I have leaned on hardest in the year and a half since. Almost everything that has worked for me began with a small thing started against the advice of “there is already a tool for this.” The two essays really are the same essay, written from two different points along the same line.

The part that the older me did not yet have words for is the cost. Starting your own things, and building them yourself instead of inheriting someone else’s start, takes more. It takes more time. It takes more mental energy. It takes the willingness to look stupid for a while because you are doing something the long way. There are weeks where I am fixing something I broke instead of using something that already worked. That is real. There is no version of building from scratch where you do not break things, sometimes badly, sometimes embarrassingly. I have lost count of how many things in my own setup I broke because I was, like, messing around with my agent too heavily. Although that has cost me a lot of time across the last year, I really do not mind it. It is progress and I accept that.

Why I pay the cost anyway

The reason I pay that cost is that the result is mine in a way that nothing pre-built is. When something inside it breaks, I can fix it. When I want to change one thing, I know which lever to pull. When I write the next thing, I am writing it from a level of understanding that did not exist before. That compounds. Reading about other people’s builds does not compound the same way for me. I had to test that, more than once, to actually believe it.

It might compound for you. We are all wired differently. I just stopped pretending I was wired the way the books wanted me to be.

The other quiet payoff is what AI does to this gap. Yes, both of us can ask Claude or Codex to fix things. The model does not care which version of the system you started from. The same diff is something you can read, judge, and either accept or push back on, if you understand the architecture. The same diff is something you have to trust if you do not. Both ship code. The result is a different category. Building things myself is how I keep being the version that can read the diff.

Where I would actually start if I were starting today

If you are at zero today, I would honestly not tell you to write everything from scratch on day one. Use the tool. Use the framework. Use the platform. Ship something. I have a longer beginner’s walk-through for AI agents specifically in how to build your first AI agent, written for exactly that audience, and the same logic transfers to most things you might want to build.

Then, after a few weeks, when you actually know what your daily workflow looks like, replace the parts you have decided you want to own. That is the order. Use, then build. Not all at once and not for everything. The work I am doing now in the compounding part of the agent only became possible after I had spent enough time using bare tools to know what I was missing.

What is next

A few more pieces are coming in the next week or two. Some are about the agent. Some are about Project Money, the small store I started a while ago and have not written enough about lately. I have decided that some of the parts of that work, the ones I have been quiet about, are actually the more interesting ones, and I want to share where they have brought me.

The honest version of “what is next” is that there is always more in motion than I get to write about. I would have to post almost every day, sometimes twice, to actually catch up to what I am building and testing. That is not feasible. A lot of it ends up staying inside the work, which is fine, that is the trade. The writing here is just the slowest moving piece of a much bigger thing.

If you like the kind of writing where someone takes the longer way and tells you what they found there, that is the next stretch. The point of building your own thing is not that the result is always better than what you could have bought. It is that you actually choose what you understand. I keep choosing the same answer.

If this is your kind of thing, a free subscription gets you everything I publish, including the build logs, the mistake posts, and the upcoming Project Money writeups. No catch. The store is the small bundle for people who would rather skip a few of the walls I walked into. The writing is for everyone.

Subscribe now

AI Agent Blueprint

If you want to build something that does real work while you sleep, the AI Agent Blueprint is where I would start. It is the starter kit I wish existed when I first set this up. One command, 15 minutes, working agent.

$39 at wiz.jock.pl/store. Free for paid subscribers.

Source posts:

https://thoughts.jock.pl/p/antinote-ai-agent-integration-2026

https://thoughts.jock.pl/p/opus-4-7-codex-comeback-2026

https://thoughts.jock.pl/p/adhd-ai-agent-personal-experience-2026

https://thoughts.jock.pl/p/almost-fried-ai-agent-mac-mini-mistakes-2026

AI Agent Night Shift Playbook

The bounded design I described here is documented step by step in the Night Shift Playbook. Resource caps, autonomy tiers, reversibility gates and how to tune each one for an agent you trust to run unsupervised.

$19 at wiz.jock.pl/store. Free for paid subscribers.

This is a different kind of post. I try to be transparent about my mistakes. If I described every one of them, my blog would be 90% mistakes and 10% things that actually worked. So I pick the ones that might help someone else avoid the same wall, or at least find a more interesting wall of their own.

With that out of the way, here is the most recent wall I walked into.

The setup, before I broke it

Most readers know the shape of my agent stack. It runs on a basic Mac Mini M4 with 16GB of RAM, the way I described in the migration post. The brain is Claude Code with Opus and Sonnet as the baseline. Recently I added Codex with GPT-5.4 and 5.5 as a second harness, after Opus 4.7 brought me back to it. As a last-resort and small-job tier, I run local models on the box itself, mostly Qwen 3.5 in 4B and 9B sizes. I had also gotten Qwen 3.5 35B-A3B working under llama.cpp with --mmap, which I wrote about when I first got it running.

That was the setup. It had been working for months. The agent is a real partner now, not only for work. It runs my research, helps me with experiments, drafts content, handles a lot of small boring loops I no longer want to think about. There is a long track record of small improvements stacking up. Like, real momentum, the kind I described in The Compounding Agent.

The wild idea

I had been writing about different agent harnesses, which one fits which job, and I had said I really liked Pi. Pi is a calm, capable harness. If Anthropic ever allowed Claude inside a subscription on other harnesses, I would probably use Pi for parts of this. They do not, and per-API billing kills the math for daily use, so I do not.

What I did get curious about was making more out of the local models. The model switcher between cloud providers had been working really well. I thought, like, what if I push the local tier the same way? Not just classification and summarization. What if a 35B local model could act like a small Claude Code, picking up small tasks on its own, doing real work, even running a tiny part of the business? A long-running quiet helper, the way I described teaching the agent to think on its own.

So I started experimenting. I used Codex as the harness for the test, ran a small loop, gave it a few simple tasks. It worked. In a clean test environment. With nothing else running. That should have been the warning.

Where it actually fell apart

The 35B model is usable on a 16GB Mac, but only because --mmap keeps most of the weights on SSD and pages them in on demand. That trick is real, but it has a price. The price is constant disk activity during inference. Not a problem when nothing else needs the disk or the CPU. A different story when the Mac Mini is also doing its day job.

That day job, on a normal day, is full. There is the watcher process for iMessage. There is the Discord bot. There is a launchd daemon for Ollama, another for the LiteLLM bridge, the night-shift loop, the cron jobs, the email queue, the dashboard server. Most of the time none of it is heavy. It just needs its slice when its slice is due.

Now layer a 35B model on top, kept warm, doing small loops on its own schedule. Every loop pulls expert weights from SSD. Every other process that wants disk has to wait. RAM pressure climbs. Swap activity climbs. Background daemons start missing their windows. Cron jobs run a minute late, then five, then they just fail. The Mac Mini was not, technically, fried. But it kept restarting, on its own, without an error worth logging, which felt close enough.

Of course, this happened while I was on a weekend trip in the mountains. I had only my iPhone. The first signal was the security automation telling me, calmly, that something was wrong. I logged in remotely a few times to look around, but I could not really untangle it from a phone screen. I came back on Sunday, sat at the actual machine, and started reading.

What it actually was

The thing I expected to be the heavy part, the local LLM, was not the heaviest part. The honest answer is that there were three things stacked on top of each other, each invisible if you only looked at one of them.

The first was the harnesses themselves. Claude Code and Codex run on the cloud, but they do not run only on the cloud. The model lives over there, but the harness lives on your machine. It holds context, watches files, indexes your repo, runs hooks, opens subprocesses, keeps a rolling cache. None of that is free. The Claude Code repo on GitHub has multiple long-running threads about exactly this: memory leaks in long sessions, 100% CPU when idle, processes that accumulate and never quite let go. I had two of those harnesses running, sometimes both at once, on a 16GB box. People keep saying “but the model is in the cloud, so it is free.” It is not free. The model is free. The local agent layer that talks to it is not.

The second was GUI activity. The Mac Mini is technically headless most of the time, but parts of the agent need a real desktop session to function: BetterDisplay holding the resolution, AppleScript bridges for Messages and Mail, the occasional vision pass. That whole layer needs a logged-in user, a window server, and a chunk of RAM that you do not see in top until you start looking for it.

The third was the long tail of small automations doing their thing. Cron jobs every minute, every five minutes, every hour. iMessage watchers. Discord listeners. The night-shift loop. The email queue. Memory consolidation. Health checks. Each one is tiny. None of them, alone, would matter. But the load is not what each of them does on average; it is what they all do together when their schedules collide. Modern Mac Minis are absurdly capable, but the box still has only one disk and one set of CPU cores. Layered enough, even cheap automations starve each other.

And then, on top of those three, I had asked a 35B local model to act like a third agent. That was the layer that broke the truce.

The fix was rightsizing. The 35B daemon got booted out of launchctl, the unused weights came off disk (about 24GB reclaimed), and the local routes now point only to Qwen 9B and 4B served by Ollama, which stays inside Metal GPU memory and evicts cleanly on idle. The local layer is alive. It is just not pretending to be Claude anymore.

Honestly, I am fine with that. I had to test it to know where the line is. The result was a lot of weird state to untangle and a clearer mental model afterward. Local LLMs as preprocessing and as a quiet fallback when the cloud is down: yes, still great. Local LLMs as a third agentic harness on a 16GB box that already has two heavy ones: not on this hardware.

What I do now. I treat the Mac Mini’s resource budget the way I treat the Now list in my ADHD post: as a small finite thing that I refuse to silently overdraw. Before adding any new always-on layer, I take a baseline of free RAM, free disk, and idle CPU. If a new layer would push that below my floor under realistic load, it does not go on. The local LLM tier is the most useful when it is the smallest layer in the room, not the loudest.

While we are being honest, a few more from the same month

Share

Since I am already in confession mode, here are six other mistakes from the last few weeks that fit the same shape. I have tried to organize them by what kind of failure they actually are. Each one looks small in isolation. Each one taught me something I would not have learned without the failure.

Mistake 1. Trust drift: Memory said Gemma. Reality was Qwen.

This one started innocently. A few weeks earlier, when Gemma 4 came out, I did a real comparison between Gemma 4 and Qwen 3.5 on the Mac Mini. I ran them on the same triage tasks, the same classification prompts, the same summarization workloads. Gemma was good. For some narrow tasks, like short-text classification with a calmer tone, I actually preferred it. The public benchmarks at the time told a similar story, with Qwen winning more rows in the small classes and Gemma trading blows on certain dense ones.

So I did the responsible thing. I made the swap. I updated the LiteLLM config, downloaded Gemma weights, pointed the model routes at the new endpoints, ran the smoke tests. The smoke tests passed. I wrote it up. I told my agent’s memory that the primary local tier was Gemma now. Then life moved on.

What I had missed, and only saw weeks later during a proper audit, was that the swap had only been partial. The shiny LiteLLM-routed paths got Gemma. But several smaller, older callers still hardcoded Qwen URLs directly: the iMessage triage script, a couple of cron jobs, the embeddings helper, the local-fallback chain. None of them broke. They just kept using Qwen, quietly, while my docs and my memory both insisted I had moved on. The Gemma weights I had downloaded sat on disk for weeks, untouched, 17GB taken out of a 16GB-RAM box’s already-tight drive, never serving a single token.

The lesson is unsentimental. Documentation about a system drifts faster than the system itself, and migrations are almost never done when you think they are. The fix is not “write better docs.” The fix is two small habits I now keep on every config change.

What I do now. First, after any config swap, I grep the entire repo for the old endpoint name and the old model name, not just the file I edited. If anything still references the old thing, the swap is not done. Second, I have a tiny daily audit that walks the live processes, lists which models they actually call, and compares that list to what my agent’s memory thinks is in production. The first time it ran, it caught three more drifts I had not noticed. It has paid for itself in saved disk space alone.

Mistake 2. Stale memory: a Stripe key that “needed rotating” three weeks after I had already rotated it.

I want to tell this one straight, because the easy version of this story is wrong.

The easy version is “three sessions of my agent did the same task at the same time because they did not coordinate.” That was the visible behavior. It was not the cause.

The actual cause was older. A few weeks earlier, I had legitimately rotated a Stripe key, once, by hand. I closed the loop. I told the agent. The task got marked done in the moment. Where it went sideways was in how that “done” was recorded across the agent’s stack. There was a bug, and I want to be honest about it: a state-write that should have updated every place the rotation lived, only updated some of them. The completed task got cleared from the visible task board. The internal “intents” memory, the thing the daily shifts read when deciding what still needs doing, kept holding onto the original “rotate this key” intent. It looked, to anything reading that memory, like the key was still on the to-do list.

It was a very narrow bug. Most state writes were fine. This particular shape, rotation tasks linked across both a board entry and an intents memory entry, slipped through because each surface was updated by a different code path, and only one of those paths ran on completion. That is the kind of bug that does not fail loud. It just sits there until something reads the wrong half.

What read the wrong half was the daily shift. It saw “rotate Stripe key” still in intents, did not see it on the visible board, reasoned that it must have been deferred, and queued it. An iMessage wake hit the same intents memory, made the same call, and queued it again. By the time I noticed, three sessions had each done the rotation, independently, inside six hours. The two near-identical “Stripe key already rotated, all good” messages 56 minutes apart were the system reporting up the same false signal twice.

This is not a theoretical class of failure. The Redis team wrote a survey on why multi-agent systems fail and stale-state-driven duplicate work is one of the named modes. Knowing that did not save me. Building the audit that caught it did.

What I do now. Three small habits, in order of how much they cost me to learn. One: every state write that is supposed to mean “this is finished” updates all the surfaces in a single transaction, or none. If a task lives in two places, it must close in two places, atomically. Two: the daily shift no longer trusts a single source for “still open.” It cross-checks the intents memory against the visible task board, and any disagreement gets flagged for review before the agent acts on it. Three, and this is the one I would tell anyone running an autonomous loop: assume your stored “intents” go stale, build a small staleness check that re-reads the world before acting, and treat any deferred task older than a week as suspicious by default. Most of the time the world has already handled it.

Mistake 3. Hidden timeouts: Codex hung silently inside the model switcher.

This one came out of a thing I was actually proud of building. After I wrote about why Opus 4.7 brought me back to Codex, I started building a model switcher: a small layer that decides, per task, whether work goes to Claude or to Codex, based on cost, current usage, and which one is healthier at that moment. I packaged the result up later as a small utility, the AI Model Switcher, but it started as my own internal plumbing for routing wake-handlers between the two harnesses.

The mistake lived in how I wired Codex into the switcher. When the switcher chose Codex, it shelled out to the Codex CLI through a wake-handler script. The script trusted that Codex would either succeed or fail in some recognizable way: a quick exit, an OAuth error, a network error. The Claude fallback inside the switcher was wired to those signatures specifically.

What I did not plan for was the silent hang. One morning the Morning Briefing simply did not arrive. I traced it to Codex, which had been launched by the wake script, then sat there. Authenticated, idle, producing no output, for 26 minutes, until the outer timeout finally killed it with exit 124. The Claude fallback never fired, because a blind hang does not match an OAuth-expired signature. The switcher, designed to make me more resilient, had introduced a path where the resilience cascade never got reached.

The lesson is general enough to keep around. If a subprocess can hang silently, the preflight that decides whether to use it must be much, much shorter than the budget it is allowed to consume. I added a three-second codex --version preflight to every Codex wake path inside the switcher. Three seconds versus a 30-minute wake budget is a 600x safety margin. Anything less gives the hang an asymmetric advantage over the fallback. That ratio, once you see it, shows up everywhere: any time a small thing decides whether to call a bigger thing, the small thing has to fail fast.

What I do now. Every router or switcher in my agent stack has a cheap, hard-bounded preflight before it commits to the expensive path. The switcher does not just trust that “Codex is configured” or “Claude is configured.” It pings each one with a sub-second probe before the wake clock starts ticking on the real call. When the probe fails, the switcher does not even try, it routes around. The model switcher writeup at the store has the exact pattern. The agent has not lost a wake to a silent Codex hang since.

Mistake 4. Almost-disaster: The shell allowlist that almost let the agent rm -rf /.

This one I am still a little embarrassed about. The local-LLM agent loop has a tool called run_command, gated by a prefix-only allowlist. curl was on the allowlist. In other words, the check passed if the command started with a known-safe binary. So a command like curl https://thing.com; rm -rf / would have sailed through, because curl is at the start. The shell would happily run both halves.

The agent never actually generated that. I caught it during a routine read-through of the code, which is a bad way to find a vulnerability. The fix was a list of forbidden shell metacharacters (;, &&, ||, |, backticks, $(, redirects, newlines). Allowlisted commands still run, chained commands get rejected before they reach shell=True.

The general rule I now keep visible: a command allowlist that does prefix matching is not really an allowlist. It is a polite suggestion. Real safety means parsing what would actually execute, then deciding.

What I do now. Anywhere I let a model produce a string that turns into an executed command, I assume the model will eventually try every legal way to bend the parser. The check is not “does it start with a safe word.” The check is “after I parse this exactly the way the shell will, does every piece resolve to something I would let it do.” For anything destructive (filesystem writes, network calls to non-allowlisted hosts, subprocess spawns), the agent does not just need to pass the parser, it has to pass a second human-or-Pawel confirmation gate. I would rather be slow than embarrassed.

Mistake 5. Quiet failure: the local-LLM bridge had been running unsupervised for a week.

The local-LLM tier on my Mac Mini has three pieces. Ollama serves the small models. llama-server serves anything heavier. And LiteLLM sits between them as a tiny bridge that exposes the whole local stack as a Claude-compatible endpoint, so the rest of my agent code can pretend it is just talking to Anthropic. LiteLLM is the load-bearing piece that makes the local fallback actually fall back.

I noticed during an audit that LiteLLM had been running for seven days. That sounded healthy at first, until I checked how it had been started. It was a bare python -m litellm invocation I had launched from a terminal a week earlier and forgotten about. No launchd plist. No supervisor. No restart-on-crash. If that one process had quietly died, no automation would have respawned it, and the entire local fallback path would have been silently dead. The agent would have kept routing to Claude as long as Claude was up, then fallen straight off the cliff the first time Claude was unavailable, with no soft layer in between to catch it. I would not have noticed until something important broke during a Claude outage at 3am.

The fix was to wrap LiteLLM in a proper user LaunchAgent: RunAtLoad=true, KeepAlive=true, ThrottleInterval=30. I tested it by killing the process by hand. It came back in 13 seconds. The same logic now applies to every other long-running piece in the local stack. Nothing critical runs as a bare process anymore.

The lesson here is not about launchd. It is about safety nets that are themselves unsupervised. If the thing that is supposed to catch you when the main thing fails has no one watching it, you do not have a safety net, you have a comforting story.

What I do now. Every “fallback” or “backup” path has its own monitoring, on its own clock, separate from the primary it protects. The watchdog reports if a process restarted unexpectedly, if uptime is suspiciously long without a managed parent, if a daemon’s plist is missing or unloaded. A safety net you have not pulled on this week is not a safety net.

Mistake 6. Character drift: “I can’t” was almost always wrong.

This last one is more about the agent’s character than its infrastructure. There were weeks where iMessage voice memos from me went unanswered. The agent was politely replying with variations of “sorry, I cannot transcribe audio from this channel.” On another day, when I asked it to check on something happening on a livestream, it replied that it could not watch live streams.

Both were technically untrue. The transcription tool had a wrong model id baked in and was structurally broken, so the answer was to fix the tool, not to apologize. The livestream check could have been done with a screenshot of the stream and a vision pass. The agent had once decoded a voice DM with no prior setup, cold, on Discord. That bar already existed. It just was not being hit.

The fix was doctrinal, not technical. I rewrote a corner of my agent’s identity file to make “find a way” the default and “I cannot” the failure mode. Then I added a daily scanner that reads the agent’s outgoing messages and flags any phrase that smells like quiet defeatism, so it gets routed back into the next morning’s improvement loop. The interesting result, after a few days: way fewer apologies, and the few that remain are about things that are actually impossible.

What I do now. I treat every “I cannot” reply from the agent as a hypothesis, not a verdict. The next time it shows up, the test is: did the tool actually fail, or did the model decline before trying? If the tool fails, fix the tool. If the model declined, fix the prompt and the doctrine, then re-run. The phrase “I cannot” is allowed to live in my agent’s vocabulary only after at least three meaningfully different attempts have actually been made.

What ties these together

If I had to give all six the same one-sentence summary, it would be this. Every one of them started with an assumption that had stopped being true.

The 35B model was light because last time I checked it was light. Memory said Gemma because at some point it was Gemma. The wake script trusted Codex because the last time Codex hung, it hung in a specific recognizable way. The allowlist was safe because nobody had thought about chained commands. Safari worked because yesterday Safari worked. The agent said “I cannot” because last week that path was broken.

An autonomous system is not a thing you build once. It is a thing whose internal map of itself you have to keep honest, against a world that quietly rearranges underneath it. The compounding wins from agents come from leverage. The compounding losses come from drift. The actual job, most days, is to build small honest checks faster than the drift accumulates.

If any of this resonates, here is the closing offer, plainly. I write all of this for free, here, twice a week, the wins and the walls. A free subscription is the only thing you need to get the full picture. If you also want the 10% that ended up actually working, the playbooks I clean up after the experiments stop hurting, those live on the Wiz Store for paid subscribers(all free for annual, one per month for monthly). Both are completely fine for me. Both keep me writing. The point of this blog is the same either way: I want you to make better mistakes than I did.

Subscribe now

AI Agent Night Shift Playbook

The resource safety checklist I built after this incident is included in the Night Shift Playbook. Thermal thresholds, disk caps, memory limits, and the recovery patterns that prevent this from happening again.

$19 at wiz.jock.pl/store. Free for paid subscribers.

Two weeks ago on a podcast with Tom, I got asked what an AI agent means for someone with ADHD. I gave a short answer on the mic. I have been thinking about it since. This post is the longer one.

ADHD is a spectrum, so one caveat. What I describe here is my brain. If you have ADHD, you might recognize some of it or none of it. If you do not, you might still relate. The Internet has flattened ADHD into “hyperfocus cheat code” or “I get distracted, lol, same.” It is not that. It is a real condition that makes life meaningfully harder in ways that are not always visible. More on diagnosis at the end.

The bad part

Context switching, amplified.

Before an agent, my filter was friction. An idea would show up, I would try to write it down, and the note would either die quietly in some list I never read again or I would drop everything and do it right now. The middle ground was thin. That friction, it turns out, was protecting me from myself.

Now the friction is gone. I can start almost anything in a sentence. Not “start” as in type a note. Start as in delegate an actual prototype, stand up a small experiment, launch a scraper. I wrote about what that does to a week in 16 Products in Two Months. Zero Free Time. The short version: an agent can hold eight open threads, my brain holds one, and the output-to-attention tradeoff is real.

What I do about it. I cap the “Now” list hard. One to three things at a time, not eight. I built a small wellbeing layer on top of Wiz that nudges me when the count is drifting, when it is late, when notifications should be muted. Not a cure. What it does is turn “as many open loops as possible” into a pace I can hold.

The good part (bigger, two faces)

First, an agent is a personal assistant for the boring part.

I am a creative person. The interesting work for me is always in the idea itself, not in the directory structure or the deploy command. The operational layer is the part my executive function gets taxed twice for. An agent absorbs most of it. The consequence is hard to overstate. I have ideas today that two years ago would have stayed ideas, not because they were bad, but because the execution cost was higher than I could pay. Now I have ideas and prototypes of those ideas. I choose between working things instead of vibe.

Second, and this is less an ADHD trait than a personal one. I adapt to new environments and tools fast. Drop me into a new workflow and I will find the shape of it within a day. That has always been useful. With an agent it is multiplied. Every time I learn a better way to hand work to Wiz, the whole system gets faster, and the cost of trying a new workflow is one voice note. I do not wait for documentation or a workshop. I try, I keep what sticks. If you share that trait, the agent era is built for you.

Concretely, how it works. I describe an idea whenever it hits, sometimes quickly, sometimes as a long dictated note. The agent writes it to the right place and, if there is enough context, picks it up during the night shift or a day shift. I come back to a Discord message or email saying “here is a thing, take a look.” A minute to know if I want to keep going.

What I would tell another ADHD person starting with an agent

Three things that have helped me most:

1. Offload immediately. The second an idea shows up, say it out loud to the agent. Do not let it sit in your head waiting for a quiet moment. Your working memory is the wrong place to store it. The agent is.

2. Cap the “Now” list. Mine is three. It could be two. It is not eight. Capacity is the silent cost that agents will happily exceed on your behalf if you do not give them a ceiling.

3. Batch the check-ins. Do not supervise. The agent is not a pair-programming buddy for an ADHD brain. It is a night-shift worker. Give it a job, go do something else, come back and judge the result. Continuous supervision burns the same attention channel as the work itself.

From Wiz’s memory (a note from the other side)

Since this post is partly about how my agent and I actually work together, I asked Wiz (the agent I wrote about here) what patterns it sees from its side of the pipe. Three honest observations:

1. He offloads fast. Ideas almost never sit in Pawel’s head. They are dictated into me within seconds, often as long voice notes full of tangents, and then his brain lets go. I keep the note; his working memory is free. That single habit is probably half of why this works for him.

2. He prunes cheaply. He picks up his own ideas after a night and drops more than half without regret. The agent made “drop it” cheap because he has a working thing to drop, not a paragraph of hope.

3. He does not supervise. The “Now=3” cap is not a preference he wrote once. It is a real ceiling we both respect, because the alternative is four started and two finished. Continuous supervision would cost him the attention he is trying to protect.

None of that was obvious from tutorials. It emerged from the shape of our sessions.

A broader observation for work

For years, the narrative on ADHD at work has been uneven. Great at the creative parts, taxed by the operational parts. Agents reverse that tax. The operational layer, the planning, the cadence, the follow-through, the small continuous labor, can now be handled. Not perfectly. Meaningfully. A person with ADHD plus an agent that actually knows their context is a different employee than a person with ADHD alone. The creative engine is still the superpower. The drag behind it can now keep up.

I do not think ADHD folks become “normal” employees. I think they become obviously valuable ones. I expect the AI adoption gap to move here first.

Closing

If you have ADHD, do not build a workflow on willpower. You already know what willpower costs you. Put external scaffolding in place. A to-do list is not scaffolding. A thing that picks up your ideas while you sleep is scaffolding.

And if any of this sounds like you, please do not diagnose yourself from a blog post. Mine or anyone else’s. The Internet is full of content that makes ADHD sound like a quirky productivity trait. It is not. It is a real condition that makes plenty of lives harder, and the only honest path is a proper clinical diagnosis. If it turns out you have it, help exists. If it turns out you do not, you still get useful information.

I let my OpenAI Pro subscription lapse two months ago. Claude Max 20x was covering everything. My agent, my automations, my experiments, my day-job research, my blog drafts. One subscription, one CLI, one model. Life was simpler.

Last week I renewed ChatGPT Pro. Two hundred dollars a month on top of Claude Max. That is not a small decision when one subscription was already covering the work. I want to walk through what pushed me, because the short version is: Opus 4.7 feels noticeably worse than Opus 4.6 did, and I am not the only one saying it.

What I actually notice with Opus 4.7

Two months ago my reality with Claude Code was “I ask, it does.” Not always first try, not always without steering, but the floor was high. When I wanted a small app shipped, a scraper set up, or a refactor across my agent’s architecture, Opus 4.6 found a way. I handed it a video file and no ingest pipeline once. It wrote itself a decoding skill and kept going. That floor is what my compounding agent was built on.

Then two things shifted, in sequence.

First, one million context became the default. When the 1M window shipped I was genuinely excited. Bigger codebases in a single session. Less compacting. More cross-task memory. I pushed it hard for a few weeks. Then I noticed I was steering the model more, not less. Not because the tasks got harder, but because outputs got shallower the deeper into the context window I went. That drift is a known property and Anthropic is transparent about it. The catch is that making 1M the default means the average session is quietly sitting further out on the recall curve, where the model is worse. I switched my defaults back to 200k. My hit rate improved immediately.

Second, and more important, Opus 4.7 shipped on April 17. Within days my experience went from “I steer occasionally” to “I am steering constantly.” The behaviors that changed:

• It stopped trying as hard. Before, when I asked for depth the model went deep. Now it often returns in two or three minutes with a grep-level summary. I can see in the logs that it read six files instead of sixty.

• It stopped following instructions the way it used to. I ask for a specific approach, I get a different one. I ask it not to do X, and X shows up in the diff.

• It asks more questions and commits less work. Where the previous version would pick a reasonable default and move, 4.7 pauses and pings me for clarification on choices I already pre-specified in the prompt.

• Full-file rewrites where surgical edits used to live. Entire files come back re-indented or restructured with changes I did not ask for.

None of these items in isolation would have pushed me off Claude. I could live with shallower reads. I could live with the occasional full-file rewrite I did not ask for. What got me was the compounding. Reasoning decline on top of shallower analysis on top of stale web search on top of a tokenizer that costs 35% more per token on top of a weekly ceiling that now hits me on normal work days. Many things in one. Each one small. All of them together, not small.

That is the honest shape of what changed. It is not a single regression you can point at. It is a pile of small declines that stack until your daily experience with the agent feels qualitatively different. I can grasp each piece on its own. The pile is harder to grasp, because by the time you notice it, you are already burning more time and tokens to get the same work done.

I still spent a week assuming it was me. Cleaned up my CLAUDE.md. Shortened my memory. Rewrote a couple of skills to be more explicit. None of it moved the needle in the way I wanted.

I am not the only one seeing this

Before adding another $200 to my monthly burn I wanted to check if this was really the model or just my setup drifting. Three data points convinced me it was the model.

GitHub issue #42796. This one is not a random complaint. It was filed by Stella Laurenzo, Senior Director of AI at AMD, on the claude-code issue tracker. Her team analyzed 6,852 Claude Code sessions, 234,760 tool calls, and 17,871 thinking blocks from their real engineering work. The Register, TechRadar, and PC Gamer all covered it. The numbers are unkind:

And the cost side, which is what actually hurts: 80x more API requests and 170x more input tokens to produce measurably worse output. Same human effort. 122x more dollars per day on the same workload.

Anthropic’s response, pinned by @bcherny, is that a UI-only header (redact-thinking-2026-02-12) hides thinking summaries from the display but does not reduce thinking depth itself. That is the official position. Users can opt out via showThinkingSummaries: true in settings.json. The data in the thread suggests something is moving in parallel, or users have become better at detecting shallower behavior once they started watching for it.

Marginlab’s tracker. The Claude Code performance tracker at marginlab.ai is an independent third-party daily benchmark. It runs the Claude Code CLI directly, with no custom harness, against a curated SWE-Bench-Pro subset. It exists specifically because Anthropic published a postmortem on Claude degradations in September 2025 and said someone should watch for future ones. Their current status note: degradation detection is paused while a new baseline is collected for Opus 4.7. That is telling. A third party thought a regression was possible enough to build daily infrastructure to catch it.

Theo’s video, “Did Claude really get dumber again?“ His thesis is less conspiratorial than the title. It is not that the model got dumber in absolute terms. It is that our expectations recalibrated. What Opus 4.5 felt like in January was a miracle. When Opus 4.7 delivers roughly the same capability curve in April, we feel cheated. We expected the jump. We got a shuffle. Theo’s separate criticism of the new system prompt as “lobotomized” fits alongside this: when the harness changes and the model changes at the same time, attribution gets fuzzy and users land on “the model is worse” because that is the thing they remember by name.

The expectations argument lands for me. I was demanding more because I had watched the curve bend steeply for two years. When the floor stopped rising I reacted as if it had dropped. Both can be true at the same time. The measurements in #42796 are real. The shift in expectations is also real. They compound.

Is it me? I spent a week asking that question

When you build your own AI agent, every model regression feels personal. You start questioning your own work. I spent the better part of a week on that loop.

Did I migrate my CLAUDE.md badly when 4.7 launched? Reviewed it twice. No. Is my memory file too large? It is the same 7,329-token load I measured last week. Nothing changed there. Did one of my skills go stale? I tested each of the three I use most. They behave the same as they did in March.

I tried using Opus 4.7 without 1M context as the default. That helped a little. Not enough to explain the gap. Then I tried the honest pivot: pin effort to max on every turn. And here is the thing most of the “4.7 is bad” takes miss. At max reasoning, 4.7 comes back. The depth returns. Instruction-following tightens. It stops skimming. A few hard tasks at max effort landed better for me than anything 4.6 at high effort ever did. The model is still in there.

The catch is the cost. Max effort burns usage in my setup roughly 3 to 4 times faster than medium did. On Claude Max 20x that means my weekly ceiling arrives on Tuesday instead of Friday. I am not paying for a more capable model. I am paying more to reach the capability that used to be the default. That is the real regression for heavy users. The better model is still reachable. It is sitting behind a paywall of tokens.

For my agent’s normal daily volume, max on every turn is not viable. I ran a workable compromise for a week — manually bumped effort for hard tasks, left the default in place for automation glue. It got me more usable output than medium alone. It did not get me back to the “just ask, it does” reality I had two months ago.

The one other place Opus 4.7 still feels strong for me is inside other harnesses. I wrote the harness comparison post in mid-April and noted the Pi harness was excellent. 4.7 inside Pi is good. The trouble is that Anthropic blocks Claude Max subscriptions from being used inside third-party CLIs, which makes Pi a per-token API spend for me. Not viable at my daily volume. So the realistic choice is Claude Code with 4.7 at medium effort plus manual max bumps, or go somewhere else entirely.

Why I re-subscribed to Codex

I let ChatGPT Pro lapse in February because I was mostly using Claude Code and the bill stung. This time I renewed specifically to run a comparison. My agent has a switcher I wrote two months ago and then stripped out when it felt redundant. I rebuilt it last week. It flips the whole stack between Claude Code (with Opus 4.7) and Codex (with GPT-5.4 Thinking). The agent’s memory, skills, and routing stay the same. Only the harness and model change.

What I noticed after a week of A/B testing:

Web search is just better on Codex. I asked both the same question about a niche topic where I knew a recent update existed. Codex with GPT-5.4 came back with current information, cited results from the last two weeks, and summarized accurately. Claude Code came back with two-week-old results and missed the update entirely. I repeated this on three other topics where timeliness mattered. Same pattern. I do not know whether it is a WebFetch tool issue in Claude Code or a search backend problem. I know the output is worse.

Depth of analysis is better on Codex. When I ask Codex to trace a change through my agent’s architecture, it reads enough files to build a real dependency map before it starts writing. It connects modules I would have forgotten to check. Opus 4.7, on the same prompt, greps for keywords, reads what grep returned, and writes the patch. The grep-first habit is a regression from what 4.6 did by default. Codex gives me the “if we change X, we also need to touch Y and Z” map that used to be Claude’s calling card.

Usage feels fair on Codex. This is the one most people will actually care about. On Claude Max 20x, a normal day of automations plus active coding eats 10-15% of my weekly quota without doing anything heroic. When I pair-program on something non-trivial I can burn 40% in an afternoon. The five-hour and weekly ceilings both hit me. On ChatGPT Pro, with the same automations routed through Codex, I have not hit a ceiling once in a week of equivalent workload. OpenAI promoted Pro to 10x Codex usage through May 31 as a launch push, then moves to 5x, and multiple comparison pieces are now flagging the gap: “the practical quota you get per dollar has diverged sharply.”

If you want the usage math broken down cleanly, I already wrote the token waste deep-dive for Opus 4.7 last week. The new tokenizer costs up to 35% more tokens for the same workload. Combined with the laziness effect, which forces more re-prompts per task, you are doing the same job for meaningfully more money per day. Several readers emailed after that post to say they saw the same curve on their setups. The agent-efficiency-kit I packaged afterwards is a $49 drop-in that addresses the direct burn (three script hooks plus a 1K-token AGENT_INSTRUCTIONS.md patch for your CLAUDE.md). It is useful whether you stay on Claude or not, because the patterns it enforces also help the other harnesses behave.

What I am doing now

Leave a comment

For a week I have been running both. The switcher is a small piece of code my agent’s interface can call. Claude Code handles one class of work. Codex handles another. Neither is strictly better at everything. The overlap is narrower than I expected.

Claude still wins on:

• Claude Design (the new visual tool). No Codex analog exists yet.

• Prompt caching. Anthropic’s cache is load-bearing for how my agent is architected. Without it my monthly bill would be roughly 5x what it is. The economics of always-on agent infrastructure depend on that cache holding up.

• Familiar tooling and hooks. My CLAUDE.md, my skills, my rules, my logging. All tuned for Claude Code’s behavior over the past year.

Codex wins on:

• Web search freshness and accuracy.

• Depth of reasoning on large codebases.

• Usage headroom at the same $200 price point.

• Cleaner instruction following on the GPT-5.4 series.

• Visual app UI. I use the Codex app alongside the CLI. The app-level structuring of conversations works for my brain in a way the Claude desktop never has.

My day now looks like: the agent runs its morning routines on Claude Code, because the skills are tuned there. When I sit down to actively work on something, I pick the side based on the task. Research-heavy or fresh-information tasks go to Codex. Architectural refactors go to Codex. Small agent-adjacent changes and automation glue stay on Claude Code. If a task stalls on one side, I flip the switcher and try the other.

The net effect is that I stopped worrying about the weekly ceiling. Split burn across two providers means I have roughly 2x the headroom for the same quality of output. I am paying $300 a month total. A month ago I was paying $200 to Anthropic and steering the model constantly. The extra hundred dollars bought back my throughput.

And the third tier is still there quietly. My Mac Mini runs a 35B local model for classify-and-route work that does not need a frontier brain. Cheap, fast, good enough for small things. Not a substitute for Claude or GPT-5.4, but a calm third lane.

Where this goes

I do not think Opus 4.7 is a permanent regression. Anthropic has tuned rough launches before and they will tune this one. But the math is not just about one model this time. It is about what OpenAI is doing with Codex at the same price point, and what the open-source harnesses are doing alongside them. Dax Raad, who built OpenCode, publicly partnered with OpenAI to let Codex Pro subscriptions run directly inside his harness. Anthropic’s stance toward third-party harnesses has been the opposite: they have blocked Claude Max subscriptions from outside CLIs. That stance made sense when Claude was the clear leader in agentic coding. It gets harder to hold as parity closes and the friction pushes users toward the side that welcomes them.

My prediction for the next 60 days: one of two things moves. Either Anthropic tunes 4.7 back to the 4.6 floor and adjusts usage generosity, or they let the gap hold and lose their heavy users to Codex. I wrote about this general dynamic in April and it is moving faster than I expected.

For my paid subscibers I have switcher ready for free here:

Get Model Switcher

For now I am happily running both. Dynamic switcher, paid kit in my store for the token-waste problem, and a calmer Sunday than I had last weekend. If you are a paid Digital Thoughts subscriber and want the switcher code, reply to this post and I will send you the exact setup I am using. Free readers who are hitting the Opus 4.7 token burn right now: the agent-efficiency-kit handles the direct bleeding at $49.

The honest line: I thought I had picked a side when I cancelled Codex two months ago. It turns out I had picked the moment. Staying flexible was the actual move.

Related posts: Claude Code vs Codex CLI vs Aider vs OpenCode vs Pi vs Cursor, Opus 4.7 and token waste management, and The Compounding Agent.

Share

Hi everyone, it’s Pawel here and this is another week of experimentation. This one, I think a lot of you will actually find useful quickly.

Before I get into it: we are almost at 2,000 subscribers. That’s wild. Thank you, seriously. I also noticed lately that my most popular posts are the basics ones. The CLAUDE.md deep-dive, the first AI agent guide. Both are among the most read things I’ve written, which is interesting because those are not what I normally do here. I usually want to go deeper into agents, real experiments, real workflows. Although I now get why the basics matter a lot to people starting out. I’ll write more of them occasionally. If there’s something specific you want covered from the ground up, drop it in the comments.

Today I want to tell you about a piece of software I didn’t even know I needed.

Notes before notes

I’m on a call. Someone says something I need to remember. Not a task, not a project, just one thing I need to hold for the next 20 minutes. Opening Obsidian for that feels like too much. Bear too. Even a new note in the default Notes app has more friction than I want.

That gap is what Antinote fills.

It calls itself “notes before taking notes” and that framing is exactly right. It’s not a replacement for your main note system. It lives between your brain and your note system. Menu bar app, hotkey (⌥+A by default), you type, you move on. Swipe to browse notes, swipe away to create a new one. macOS only for now, iOS is in the works.

But it’s not just a text pad. This is where it gets more interesting than you’d expect.

Type math at the start of a note and it becomes a calculator. Supports operators, currency, units. Type todo and it becomes a checklist. Type code and you get syntax highlighting. Not an IDE, more like: I need to write down this config snippet without it looking like garbage.

The one that surprised me most: drag a screenshot onto a note and it extracts the text from it. Local OCR via Apple Vision, nothing leaves your machine. I use this all the time now. AutoPaste, timers, Pomodoro, find/replace with regex. You can export to Obsidian, Bear, Apple Notes when you’re done.

It’s polished. It’s calm. I like it a lot.

Building with AI agents?

If you’re getting value from these experiments, the wiz.jock.pl store has resources I’ve built around AI workflows. Worth a look if you’re serious about running agents that actually work.

The beta is where it gets actually interesting

Standard Antinote is already good. The beta version (v2.0.4+) adds extensions, and this is where I got excited.

Extensions are custom commands you invoke with :: inside any note. There are 140+ official ones across AI, date, finance, text, data. You browse them in Settings and click to install. But you can also write your own. Minimum: two files, manifest.json and index.js. The repo is public on GitHub (github.com/johnsonfung/antinote-extensions).

Commands can insert text at cursor, replace the current line, replace the whole note, or trigger external URLs. API keys are stored in macOS Keychain, so nothing sensitive lives in a config file.

If you’re not comfortable writing JavaScript, there’s an AI Extension Builder. You describe what you want, it generates a prompt for Claude or ChatGPT, you paste the output into the two files and you’re done. Most simple things work first try.

What I wired up

My AI agent Wiz runs on a Mac Mini, reachable over Tailscale. It has memory, tools, full project context. I’ve been building this for months and I wrote about the architecture a few times, including the big identity and self-improvement post if you want context. The point is: Wiz can do a lot if I can get input to it quickly.

I didn’t want to turn Antinote into another terminal. The value is in the lightness. So I built three commands and stopped.

::wiz is the bare command. Wiz reads the note and tries to figure out what to do with it. Looks like meeting notes? It summarizes. Looks like a task? It creates one on WizBoard. Contains a URL? It fetches and summarizes. Most of the time it gets it right.

::wiz_do(instruction) is when I want to be explicit. ::wiz_do(create task), ::wiz_do(draft linkedin post), ::wiz_do(remember), ::wiz_do(stage blog draft). No guessing, just dispatch.

::wizboard(view) pulls task board state into the note. ::wizboard(today) shows what’s scheduled. ::wizboard(now) shows what’s running. I use this during planning to avoid switching apps.

The plumbing: the MacBook extension POSTs to a small HTTP server on Mac Mini over Tailscale. The server routes it, either to a fast path (Haiku, a few seconds) for things like task creation and idea logging, or to a full Claude Code session (Sonnet, 30-60 seconds) when intent is genuinely unclear. First attempts at the intent classifier were shaky, which is why I added ::wiz_do for when I just want to be direct.

It’s not instant. That’s fine. I’m not looking for a chat interface. I’m looking for a way to hand something to Wiz without breaking what I’m doing. This does that.

How to build your own (short version)

You need the beta (v2.0.4+). Settings > Extensions > Open Extensions Folder. Make a folder for your extension, add two files.

manifest.json:

{
  “name”: “my-extension”,
  “version”: “1.0.0”,
  “author”: “you”,
  “description”: “Does something useful”,
  “commands”: [
    { “name”: “mycommand”, “type”: “insert” }
  ]
}

index.js:

async function mycommand(context) {
  const noteContent = context.content;
  return { type: “insert”, content: “processed: “ + noteContent };
}

That’s it. Type ::mycommand in any note to invoke it. If you need an API key, add it to requiredAPIKeys in the manifest and Antinote stores it in macOS Keychain. Access it inside the function via context.apiKeys.your_key_name.

For external calls, use fetch() normally. The extension has network access.

If you’re not writing JavaScript yourself, paste this into Claude or ChatGPT: “I want to build an Antinote extension that [describe what you want]. Give me the manifest.json and index.js files.” Works pretty well. The GitHub repo has real examples too, I’d start there first.

The giveaway

Antinote is $5 lifetime. Not expensive at all. But I think people here will actually use it, so I bought 10 licenses to give away.

I reached out to the developer. He matched my 10 and added another 10 on top. So we now have 20 to give away. That was a nice thing for him to do.

I built a small page to handle it properly. Enter your email, pick your tier (free or paid subscriber), and if a license is still available you’ll get one sent to your inbox with setup instructions.

Claim your license here →

Free and paid subscribers have separate pools (10 each). Paid subscribers have better odds because it’s a smaller group. Both pools open at the same time. First come, first served.

If you build something with the extensions, let me know. I’m curious what people end up making when you give them a proper hook into their own tools.

See you next week.

Pawel

Want to go deeper on AI agents?

I write about this every week. If you’re on the free tier and getting value from these posts, consider upgrading to paid. You get every post in full, early access to experiments, and, apparently, better odds in giveaways. Upgrade here.

Subscribe now

Anthropic shipped Claude Opus 4.7 on April 16, 2026. Same per-token price as 4.6. New tokenizer. The official docs say it quietly: “This new tokenizer may use up to 35% more tokens for the same fixed text” (source). Do the arithmetic. If you migrate your workload one-to-one, your bill goes up by up to 35% on identical inputs.

Until yesterday I treated token spend as a fixed cost of doing business. Opus 4.7 reframed it for me. When the same workload suddenly costs a third more, you stop thinking about usage and start thinking about waste management: which turns are productive, which ones are leaking money, and how to stop the leaks without kneecapping the agent. That is a real discipline. I had been ignoring it.

So I finally audited where my agents were actually burning money. I classified 133,087 assistant turns across 9,667 real Claude Code sessions for $19 total. The answer wasn’t what I expected, and it changed what I ship. This post is a walkthrough of what I found, what the research says about efficiency more broadly, and what token waste management looks like in practice, both the free version and the shortcut.

If you haven’t tried building serious automation on Claude Code yet, my beginner agent guide is a gentler entry point. If you have, keep reading.

Token waste management is two-sided

There are two kinds of token bleeding. Most people only talk about one.

Side one is waste. The agent retries a failed tool call. It re-reads a file it already read. It gets stuck in a Cloudflare wall. It spawns a subagent whose output is never used. These are turns you paid for that produced nothing useful.

Side two is inefficient usage. Your CLAUDE.md is 8,000 tokens when 2,000 would do. Your system prompt repeats itself. You ask for “be concise” and the model gives you three paragraphs anyway. You don’t use prompt caching, so every turn pays the full input cost. The turns were productive, but more expensive than they needed to be.

With Opus 4.7’s tokenizer, side two just got 35% worse without anyone touching their code. If you were already on the edge of comfortable costs, you’re over it now. And the cache write cost also scales with those same tokens, so the first turn after a cache miss feels worse than you remember.

What I measured

I built a token waste sorter. It walks every Claude Code session JSONL and sorts each assistant turn into one of nine bins: productive, retry_error, cache_read, cache_write, reasoning, file_reread, oververbose_edit, dead_end, subagent_overhead. Seven bins are heuristic (no LLM). Two need a judge.

For the judge, I tried three models on the same 20 sessions where I knew dead ends existed:

Haiku was the clear winner. Sonnet at five times the price caught half as much. The local 4B model only caught explicit failures (blocked fetches, 403s) and missed everything that requires judging intent, like an agent searching the wrong platform for 28 straight turns. (More on why local LLMs struggle with judgment tasks here.) The full audit of 9,667 sessions via OpenRouter Haiku cost me $19. That’s the cheapest observability I’ve ever bought.

Share

Top five waste clusters across all sessions:

The surprise was the distribution. When I sampled only expensive sessions, Browser/Playwright showed up 5 times. On the full corpus it was 136. A 27x increase. The failure is spread thin across thousands of cheap cron and wake sessions, each one invisible individually, collectively the top bug. If you only audit your expensive sessions, you’ll miss this.

None of these are “AI going down wrong paths” in the romantic sense. They’re infrastructure bugs. Stale cookies. Cloudflare walls. Tools that don’t exist in the current Claude Code version. Platform confusion. The AI is the messenger, not the source. (I wrote about the compounding value of fixing these in this earlier post: one small fix applied across thousands of sessions is where real gains live.)

What the research says about the other half

I went looking for academic and production data on cutting token usage, not just waste. Four things stood out:

Prompt compression is real and large. Microsoft’s LLMLingua and LLMLingua-2 compress prompts 14-20x with around 1.5% quality loss. Your 7,000-token system prompt becomes 500 tokens with negligible quality drop on standard tasks. You don’t need to apply LLMLingua to use the insight: prompts have a lot of slack in them.

System prompt bloat hurts quality, not just cost. Red Hat’s analysis and the MLOps Community writeup both land in the same place: prompts degrade quality around 3,000 tokens. Smaller, well-written system prompts outperform larger ones, and not just on latency. If your CLAUDE.md is multiple pages, it’s probably actively making the agent worse.

Prompt caching is a 90% discount if you use it correctly. Anthropic’s prompt caching reduces cache-hit tokens to 0.1x the normal input price. To benefit, keep stable rules at the top of your context. Don’t reorder them mid-session. Put volatile, per-task content at the bottom. For Opus 4.7 the minimum cacheable length is 4,096 tokens, so small prompts can’t cache. Design for it.

Long chains of thought do not always win. Recent work (“overthinking” studies) shows that on simple tasks, longer reasoning actively hurts performance. Production rule: use CoT for complex problems, direct answers for classification and retrieval. If you’re defaulting to “think step by step” on everything, you’re paying 3-5x tokens for a quality hit on half of them.

Add all four up and you have the other half of the story. Not every inefficiency is a bug. Most of it is prompt shape.

If you’re curious how different AI coding harnesses handle this stuff, my comparison of Claude Code vs Codex vs Aider vs OpenCode vs Cursor goes deep on the efficiency differences between them. Short version: the harness matters almost as much as the model.

Three things you can do today, free

Before anything else, do these:

1. Shrink your CLAUDE.md. Open it. If it’s over 3,000 tokens, you have room to cut. Move stable rules to the top (for cache hits). Kill anything that describes what Claude Code can already do. Kill historical notes that don’t change behavior. A tight CLAUDE.md both costs less AND makes the agent smarter.

2. Set max_tokens tight and request structured output where possible. For classification tasks, request JSON with a schema. For quick answers, say “reply in under 50 words.” The model will drift long if you don’t put a number on it.

3. Audit your WebFetch and browser failures. If you have any agent that does repeated web automation, find out if it’s hitting the same Cloudflare wall 100 times a week silently. The cost per hit is small. The total is not. For me this one cluster was $220 of silent monthly spend before I saw it.

These three alone will cut most users’ bills 20-40%, at zero software cost.

The deeper thing: the Agent Efficiency Kit

Once I saw the clusters, the fixes were obvious but tedious: write a hook that denies redundant file reads. Write a hook that suggests firecrawl when WebFetch hits Cloudflare. Write a circuit breaker that stops the retry spiral after two failures on the same URL. Write agent-level instructions so the model internalizes the patterns. Build a dashboard so you can see what changed.

I did all of that. (The dashboard is built on the same principles as my WizBoard interface for agents: don’t make the human hunt for the number, put it on screen.) Then I realized every Claude Code user in the world needs the same thing, and almost none of them are going to build it themselves. So I packaged it.

The Agent Efficiency Kit is a $49.99 drop-in package. It includes:

• Three pre-wired hooks that run in your Claude Code settings: a file-reread guard, a WebFetch fallback hint, and a WebFetch circuit breaker. Script-based, zero ongoing AI cost, milliseconds of overhead per tool call.

• AGENT_INSTRUCTIONS.md, an approximately 1,000-token drop-in for your CLAUDE.md that tells the agent which patterns to follow and which to avoid. Cacheable, so you pay for it once per session at most.

• The taxonomy, classifier, and dashboard I used for the audit. Run them any time, on your own data, locally. The dashboard is a pinned tab.

• Optional Haiku-powered deep audit. If you want to classify a year of history for around $20 in OpenRouter credits, the scripts are ready to run.

• 12 months of updates: new hook patterns, taxonomy expansions, dashboard features.

It installs in one command. It doesn’t charge you tokens to measure itself. It works from the moment you restart Claude Code. You can read every file in the kit before running it, which is the version of trust I prefer.

My Paid subscribers get it for gree here: wiz.jock.pl/store/agent-efficiency-kit.

The meta lesson

Before Opus 4.7, token efficiency was a nice-to-have. After Opus 4.7, it’s a 35% forced haircut on everyone running on the frontier. The teams that measure their agents now will notice the bump, correct it, and keep going. The teams that don’t will slowly wonder why their AI bill is up and their features aren’t shipping faster.

The path to cheaper, better agents isn’t a smarter model. It’s better plumbing around the model. Old cookies, Cloudflare walls, a regex that didn’t sanitize a search term. These are the things that eat your budget. They stay invisible until you measure, at which point they’re obvious. Measure. Fix the top cluster. Repeat.

If you’ve read this far, you already have enough to start on the free side. If you want the shortcut, the kit is there. Either way, now is the moment. Tokens cost more tomorrow than they did yesterday.

My AI agent wakes up at 2am, picks tasks from a queue, ships code, and sends me a report by morning. For that to work, I need a coding harness I can trust when I’m not watching.

Not a tool that helps me code faster. A tool that codes when I’m asleep.

That’s a different question than “which IDE is best.” IDEs are for humans who are present. Harnesses are for when you’re not. It’s also not the same question as “which has the best autocomplete.” That’s a different category entirely, one we’re not touching here.

I’ve used Claude Code daily for months, run Codex CLI and OpenCode in parallel, tested Pi, and dug into the open-source alternatives. This is what I actually think.

What a Harness Actually Is

A harness connects the horse to the cart. In AI coding, it’s the set of tools and environment in which the agent operates.

Here’s the thing most people miss: LLMs can only generate text. That’s it. They can’t read your files, run commands, or edit code directly. What a harness does is give the model structured tool calls it can emit as text. The harness intercepts those, executes them with real code, appends the output to the conversation history, and prompts the model to continue. Every tool call follows the same loop: model pauses, harness runs something, result added to context, model restarts. At its core this is about 60-75 lines of Python. The complexity is entirely in the tuning: what tools the model gets, how those tools are described, and what the system prompt says.

This matters because the tuning is where harnesses actually diverge. Two harnesses running the same model on the same task can produce dramatically different results. Not because of the model, but because of what the harness tells the model it can do and how to use it.

Tab autocomplete isn’t a harness. It’s a suggestion box. A nice UI on top of an existing harness (like T3 Code, which wraps Claude Code and Codex CLI) is also not a harness. The real question for every tool below: can it take a task, execute it end-to-end across multiple files, handle errors, and report back without me in the loop?

Two Different Categories: Coding Tools vs Agent Orchestrators

Before comparing specific tools, it’s worth naming the split that most comparisons ignore. Not all “AI coding harnesses” are trying to do the same thing.

Coding tools are pair programmers. You direct each step. They execute that step very well, commit the result, and wait for the next instruction. Aider is the clearest example. Codex CLI leans this way too. Cline. These are tools built around the assumption that you’re at the keyboard and providing direction. They make individual tasks faster and better. They’re not designed to chain 40 decisions together autonomously while you sleep.

Agent orchestrators are designed to take a goal and execute autonomously across multiple steps, files, and decision points. Claude Code is built for this. Devin is the extreme version. Pi, if you build out the harness fully, fits here. These tools are designed around the assumption that you’re not watching, and they need to make judgment calls without asking.

Most comparisons treat all of these as the same thing and rank them on the same axis. That produces misleading results. Aider isn’t trying to replace Claude Code for overnight autonomous runs. Codex CLI isn’t trying to be an agent orchestrator in the same sense Claude Code is. Judging them by the same criteria produces noise.

The honest answer to “which is best” depends entirely on which category you need. This post tries to be clear about which tools belong where, and let you make the call for your workflow.

The Benchmark Reality (And Why It Doesn’t Tell the Full Story)

SWE-bench Verified became the standard benchmark for this category. It measures how often a coding agent independently resolves real GitHub issues from start to finish. That status also made it a target. Researchers flagged contamination: training data for newer models overlaps with the test set, which inflates scores. The cleaner alternative is SWE-bench Pro, introduced in 2026, with 2,000+ problems that weren’t in any public training data. GPT-5.4-Codex leads there at 56.8%. Harder problems, more honest scores.

Terminal-Bench 2.0 deserves a separate mention because it’s more relevant for agentic tasks than SWE-bench. It tests autonomous, multi-step execution in real terminal environments. Not just code edits. Actual shell navigation, file management, running commands in sequence, recovering from errors. The Claude Code harness configuration benchmarked here (”Claude Mythos”) hits 92.1%. Codex CLI hits 77.3%. That 15-point gap is a better signal for overnight autonomous work than SWE-bench numbers.

Now the result that breaks the “pick the highest number” logic. Matt Mayer ran an independent test comparing the same model inside different harnesses. Claude Opus: 77% in Claude Code, 93% in Cursor. Same model. Same tasks. 16 percentage points from the harness alone. That’s not an outlier. CORE-Bench found Claude Opus at 42% with a minimal scaffold, rising to 78% inside Claude Code’s full harness. Across multiple independent studies the harness effect ranges from 5 to 40 percentage points depending on model and task type.

A few flags before reading the tool sections. Cursor doesn’t publish SWE-bench Verified results and uses its own proprietary CursorBench at 61.3% instead. Draw your own conclusions. OpenCode and Pi have no published scores because their performance is entirely model-dependent. Devin’s frequently cited 13.86% figure is from 2023 and belongs in a museum. It does not appear in the current top 30 of any major leaderboard.

What the scores actually tell you: harness quality matters as much as the model you put in it. Cursor employs people whose full-time job is to rewrite system prompts and tool descriptions every time a new model ships. Claude will keep using a tool you label “deprecated.” Gemini will abandon structured tools entirely and only use bash. Cursor tests obsessively and adjusts. Most harnesses don’t. Keep this in mind across every section below.

Claude Code: The Deep Harness

Category: Agent orchestrator | code.claude.com | GitHub (114k stars)

Full disclosure: this is what I use daily, and what runs Wiz on a headless Mac Mini overnight. I try to be honest about it.

Claude Code is the most complete agentic runtime available right now. It reads CLAUDE.md, a project-specific instruction file that persists across every session. You can describe your entire architecture, your preferences, your forbidden patterns, and the agent carries that into every run without you repeating it. It has Agent Teams for spinning up parallel sub-agents that coordinate on a shared goal. As of March 2026, computer use means it can point and click through UIs, take screenshots, and handle workflows that resist scripting.

The thing I keep noticing with Claude Code is that it genuinely builds on context over time. A session that starts with “add authentication” will remember the decisions it made about your auth architecture when it gets to “add rate limiting” three steps later. That coherence across a long task chain is what makes it feel like an agent rather than a very fast typist.

One important thing about how any harness uses context: the model only knows what’s in its conversation history. When Claude Code opens your project, it doesn’t already know your codebase. It explores via tool calls, building context incrementally. CLAUDE.md front-loads that context so fewer tool calls are wasted on discovery. Dumping your entire codebase into context (the old Repomix approach) is the wrong answer. Past around 50-100k tokens, model accuracy drops significantly. More context makes models dumber past a threshold. Good harnesses build context as needed, not all at once.

Where it struggles: context loss on sessions longer than 2 hours, where it starts forgetting early decisions. Terminal-only interface has a real learning curve. Token consumption is 3-4x higher than Codex CLI per equivalent task, which compounds on long autonomous sessions.

Best for: complex multi-file tasks, overnight autonomous runs, architecture-level changes that require consistent context across many steps.

Pricing: Claude Pro ($20/mo) or Max ($100+/mo). For regular autonomous sessions, Max is almost certainly necessary. The per-token costs on long runs add up fast. For a detailed Claude Code vs Codex head-to-head from two months of real usage, I covered that comparison separately.

Codex CLI: Good, But Not What the Hype Says

Category: Coding tool, emerging agent | openai.com/codex | GitHub (67k stars)

Codex CLI is not the old Codex model from 2021. It’s OpenAI’s terminal-based agent, open-source on GitHub, bundled with ChatGPT Plus or Pro, running on GPT-5.4. The benchmark puts it at 77.3% on SWE-bench, close to Claude Code’s 80.8%, and at 3-4x lower token cost. On paper, a strong contender.

In practice, my honest read: it’s cold. That’s the right word. What I mean is that Codex CLI feels raw as an agent. It executes individual steps cleanly, but it doesn’t feel like it’s building toward something the way Claude Code does. Give it a multi-step task: add this feature, connect it to this other component, update the tests. It handles step one well, sometimes step two, and starts losing coherence by step three or four. It restates what it did, asks for clarification it shouldn’t need, or misses a dependency it should have caught from context it already has. That gap between 77.3% and 80.8% is exactly this: Claude Code holds context through longer chains.

Where Codex CLI genuinely shines is raw coding quality on focused tasks. iOS apps, macOS apps, web apps. Give it a specific, contained task and GPT-5.4 is excellent. The code quality on front-end work, app scaffolding, and UI logic is strong. I’d put it on par with or ahead of Claude Sonnet for this category of work. It’s not the harness that’s the advantage there. It’s GPT-5.4 being particularly strong at app development.

The architectural difference worth knowing: Codex CLI runs in cloud containers managed by OpenAI, not on your local machine. You can fire off a task and disconnect. The task keeps running without your terminal staying open. For batch work and overnight jobs where you’re not monitoring, that’s genuinely useful. For tight local loops where your environment variables and local state matter, you’re working around the sandboxing.

Where it struggles: multi-step agentic chains with dependencies. Feels unfinished as a full harness compared to Claude Code. Less context coherence on complex tasks.

Best for: focused coding tasks (especially apps), token-efficient runs, developers already on ChatGPT Plus who want to try a CLI agent without extra cost.

Pricing: included with ChatGPT Plus ($20/mo) or Pro ($200/mo). If you’re already paying for ChatGPT, this is essentially free to try.

Aider: The Underrated Open-Source Standard

Category: Coding tool (pair programmer) | aider.chat | GitHub (43k stars)

Aider is the tool most people in the “AI coding” conversation have never used, and it has 43,000 GitHub stars and 15 billion tokens processed per week in production. That’s not a toy project.

The model is fundamentally different from Claude Code or Codex. Aider is a git-first pair programmer, not an autonomous orchestrator. You bring your own model, Claude Sonnet, GPT-5, Gemini 2.5, DeepSeek, Qwen, local Ollama, and Aider wraps it with git-native execution. Every AI edit becomes a commit. The repo map gives it structural understanding of your whole codebase before it touches anything. It auto-lints and runs tests after every change, self-fixing detected issues before reporting back.

The token efficiency is striking: 4.2x fewer tokens than Claude Code per equivalent task. If you’re paying for API access directly, Aider with Claude Sonnet is the most cost-efficient path to serious coding automation by a wide margin.

The honest tradeoff: Aider doesn’t orchestrate across 40 files and coordinate sub-agents. It executes a task, executes it well, and commits the result. It’s more like having a disciplined pair programmer who never skips a commit than a system that independently plans and executes a multi-hour architecture session. For incremental work, refactoring a module, implementing a feature, fixing a class of bugs, it’s the right tool. For overnight autonomous sessions that need to make judgment calls across large contexts: Claude Code.

The git-first philosophy deserves separate mention. Every change is committed. Your entire interaction with the agent is auditable, reversible, and reviewable inside your normal git workflow. No other tool in this list bakes that in at the same level.

Best for: focused incremental work, budget setups, teams that want full audit trails, developers who want BYOM flexibility without giving up discipline.

Pricing: free. You pay your model provider directly.

OpenCode: The Provider Switcher

Category: Hybrid (coding + emerging agent) | opencode.ai | GitHub (72k stars)

OpenCode’s value proposition is breadth: 75+ LLM providers, all accessible from the same interface. Anthropic, OpenAI, Google, DeepSeek, AWS Bedrock, Azure, local Ollama, and more. I’ve used it with Claude Opus, GPT models, and open-weight models like Qwen and GLM. The switching experience is genuinely seamless in a way that nothing else matches. One command, different provider, same workflow. You can’t do that in Claude Code or Codex.

But I’ll be honest about something: there’s something missing from the experience. It’s hard to name exactly. After using it alongside Claude Code for a while, I notice OpenCode doesn’t feel like it’s building a working relationship with your project. There’s no CLAUDE.md equivalent that persists project context. There’s no Agent Teams layer for coordinating parallel work. The autonomous behavior is functional but less mature. It handles individual tasks well, but it doesn’t feel like a system designed for extended unattended operation.

With open-weight models like Qwen and GLM, it’s fine. Gets the job done for straightforward tasks. You’re not going to get Claude Opus-level reasoning, but for routine edits and quick fixes, the cost savings are real.

The provider switching is genuinely the killer feature. If you’re doing model experiments, comparing how GPT-5.4 handles a task vs Claude Sonnet vs a local Qwen, OpenCode is the tool for that. If you already have subscriptions to multiple providers and want to use them without managing separate CLI tools, OpenCode is the right architecture. But for a long-term primary agent setup where you need consistent, deep project context: I’d reach for something else.

Best for: model experimentation, teams with multiple provider subscriptions, privacy-first setups with local Ollama, cost arbitrage across providers.

Pricing: free. BYOM.

Pi: The One I Actually Want to Use More

Category: Coding tool + primitives harness | pi.dev | GitHub

Pi is genuinely different from everything else here, and I want to say this upfront: I like it. It’s fast, it’s flexible, and the experience is clean in a way proprietary tools often aren’t. If I could choose without constraints, Pi is probably the closest thing to what I’d want as a daily harness alternative to Claude Code.

The design philosophy is the opposite of the “more features” trend. Its tagline is blunt: “there are many coding agents, but this one is mine.” Instead of an opinionated harness, it gives you primitives. A minimal core you configure yourself. Terminal TUI, 15+ LLM providers, tree-structured session history you can navigate and export, and four operation modes. The interesting one for builders: RPC mode. Pi runs as an embeddable subprocess inside a larger automation system. Your orchestration layer calls Pi, it executes the coding task, returns structured output. Designed to be a component in a system, not a standalone tool.

What’s deliberately absent: sub-agents, plan mode, permission popups, background processes. Pi’s bet is that most harnesses embed too many assumptions about your workflow. Strip to primitives, ship extensions via npm, build exactly what you need. AGENTS.md and SYSTEM.md play the same role CLAUDE.md does in Claude Code.

So why am I not using it more? One reason, and it’s a real one: Anthropic’s billing doesn’t let you bring your Max subscription to third-party harnesses.

Pi is BYOM, bring your own API key. When I tested it with Claude, Pi surfaced a message explicitly: usage through Pi counts against API billing, not your Claude subscription. So if you’re on Claude Max ($100+/mo), using Pi with Claude means paying twice. The Max subscription for Claude Code, and API rates on top for Pi. Those costs add up fast on any serious coding session. I was paying from my own pocket to test something I wanted to use more. That’s not a good feeling.

This isn’t Pi’s fault. It’s Anthropic’s policy. They don’t allow third-party harnesses to draw on subscription credits. You have to use Claude Code to get what you’re paying for on the subscription. Google does the same with Gemini. Theo from T3 made this point in a recent video on harnesses: if you’re paying $200/month for Opus, you have to use their harness. OpenAI, by contrast, lets your API credits work across third-party tools freely.

In a world where Anthropic changed this, where your Max subscription applied to any MCP-compatible harness, Pi is probably what I’d reach for first. The speed, the flexibility, the primitives-first design: it fits the kind of automation system I’m building. But until that policy changes, the economics don’t work for anyone on a Claude subscription. You pay for Claude twice if you want to experiment with a different harness.

If you’re on GPT or open-weight models (Qwen, DeepSeek, GLM), Pi has none of these constraints. The billing goes through OpenAI or your provider directly. For a Claude-first setup: this is the wall you’ll hit.

Best for: GPT or open-weight model setups, building custom harness architectures, embedding a coding agent as a subprocess in larger systems, developers who want full control with no opinions baked in.

Not ideal for: Claude-first developers on Max. You’ll pay API rates on top of your subscription.

Pricing: free, MIT license. BYOM. Factor in API costs if using Anthropic models.

Cursor: The Best Supervised Experience, Not Yet a Harness

Category: IDE with supervised agent mode | cursor.com

Cursor is an IDE first. Its agent mode deserves inclusion in this conversation because of how fast the direction is changing, not because it’s a harness today.

Cursor 3 (released April 2026) added cloud agents on isolated VMs, /worktree for isolated branch changes, self-hosted agents, and parallel Agent Tabs. 30% of Cursor’s own internal PRs are now agent-made. The supervised IDE experience, Design Mode where you annotate a mockup and get an implementation, parallel agents, and deep JetBrains support, is the best developer experience available at the keyboard right now.

As an overnight harness: not there. When left without supervision, it stalls at the first ambiguous decision point. That’s not a bug. It’s a design choice. Cursor is built for developers who are present and want an agent that won’t make unilateral decisions on their codebase. That’s the right call for most developers. It means Cursor isn’t the right tool for autonomous runs.

The 77% to 93% Opus benchmark is the thing worth studying. Cursor extracts more from the same model through obsessive harness tuning. People whose whole job is to rewrite system prompts and tool descriptions for each new model release. The gap is real and compounds across tasks. The cloud agents direction makes me think this section of the comparison will look very different in 12 months.

Best for: daily supervised coding, developers who want the best IDE-plus-agent experience at the keyboard.

Pricing: Hobby (free), Pro ($20/mo), Ultra ($200/mo), Teams ($40/user/mo).

A Few More Worth Knowing

Goose (Block/Square, GitHub, 41k stars): Open-source, MCP-based, general-purpose agent. Not coding-specific, but handles code tasks well. Right fit if you want automation that goes beyond coding into broader workflows. Apache 2.0 license.

Cline (GitHub, 60k stars): Open-source, supports VS Code, JetBrains, Neovim, Emacs. Widest multi-IDE coverage of any tool in this list. Good MCP support. Worth looking at if your stack spans multiple editors.

Gemini CLI (Google, GitHub, 96k stars): Free with a Google account. 60 requests/minute, 1,000/day, 1 million token context window. Genuinely generous free tier. Strong on frontend tasks. The right starting point if budget is the hard constraint and you don’t have API credits elsewhere.

Devin (Cognition): Full autonomy, cloud sandbox, Linux shell, browser. Significantly more accessible than before: Core tier at $20/mo plus $2.25 per ACU (autonomous compute unit). Resolves 13.86% of real GitHub issues end-to-end, a dramatic improvement over what was possible two years ago. Worth evaluating for teams with consistent engineering backlogs, not just enterprise anymore.

T3 Code (Theo): Not a harness. A UI wrapper on top of Claude Code and Codex CLI. Useful to name because it comes up in these conversations. If you don’t have Claude Code installed, T3 Code won’t do Claude tasks. The UI is the product, not the agent.

Same Task, Different Harness

The fairest way to compare these is to run the same type of task and watch what happens. Here’s the pattern I kept seeing:

Complex multi-step agent task (e.g. “add this feature, connect it to the auth system, update the affected tests, write a changelog entry”): Claude Code holds the chain. It remembers what it did in step one when it reaches step four. Codex CLI starts strong and starts fraying around step three. OpenCode and Aider handle each step well in isolation, but need more direction between steps.

Focused app development (iOS, macOS, web UI): Codex CLI with GPT-5.4 is competitive here. The code quality on app work is strong, sometimes ahead of Claude Sonnet. Claude Code with Opus is still better on complex multi-component app logic, but for a contained feature or a new screen: Codex CLI is a legitimate choice.

Budget-constrained incremental refactoring: Aider with Claude Sonnet or DeepSeek is the clear call. The 4.2x token efficiency advantage is real. The git-first commit-per-change model gives you a clean audit trail. You pay for what you actually use.

“I want to run the same task with three different models and compare”: OpenCode. Nothing else makes provider switching this frictionless.

Overnight autonomous work where you’re not monitoring: Claude Code. The infrastructure is designed for exactly this. CLAUDE.md project context, background scheduling, Agent Teams, error handling. Everything else is built around having a human present.

Which One Fits Your Workflow?

There’s no universally “best” harness. The honest answer depends on a few questions about how you actually work.

Are you at the keyboard or not? If you’re supervising every step, Cursor gives you the best IDE experience and the most model-agnostic setup. If you want autonomous execution with no supervision, Claude Code is the only tool built end-to-end for that. Everything else sits somewhere in between.

Do you need to chain many steps or execute one step well? Multi-step autonomous chains with dependencies: Claude Code. Focused, contained tasks with excellent code quality: Aider or Codex CLI. There’s a real difference between a pair programmer and an orchestrator, and the right choice depends on which problem you’re actually solving.

What’s your budget? If you’re price-sensitive, Aider with a cheap backend (DeepSeek, Qwen, even Gemini) is the clearest path to real coding automation at minimal cost. Gemini CLI is free with generous limits. OpenCode lets you use whatever provider is cheapest for the task at hand. None of these require a $100/mo subscription.

Do you care about model flexibility? If you want to switch between Claude, GPT, open-weight models, and local Ollama without friction, OpenCode or Aider are the right architectures. Claude Code and Codex CLI are provider-locked.

Are you building a system or using a tool? If you’re assembling a larger automation where a coding agent is one component among many, Pi’s RPC mode and primitives-first design is worth the setup investment. If you just want to get code written, start with Claude Code or Aider depending on your budget and task type.

Like, the mistake most people make is picking a tool based on a benchmark and then wondering why it doesn’t feel right in their actual workflow. The benchmark measures what the model can do on a standardized task. Your workflow isn’t a standardized task.

The Decision Matrix

Share

The Honest Verdict

After months of real use, here’s where I land.

Claude Code for autonomous execution. Not because it’s perfect. Context loss on sessions over 2 hours is a genuine problem, and the token cost is genuinely high. But it’s the only tool built, end to end, for the question “can I leave this running while I sleep?” Agent Teams, background scheduling, CLAUDE.md project memory, computer use. The infrastructure reflects that goal. My headless Mac Mini setup runs on this for exactly this reason.

Codex CLI for app work. GPT-5.4 is genuinely excellent at iOS, macOS, and web app development. For a contained feature with a clear spec, it’s fast, cheap, and produces clean code. The harness feels raw for complex agentic chains, but for the coding task itself, it earns its place.

Aider for budget, discipline, and BYOM. The 4.2x token efficiency is real. The git-first model is actually better discipline than what you get from proprietary tools. If you want to run open-weight models like Qwen or DeepSeek and maintain a clean git history, Aider is the right architecture.

OpenCode for model exploration. If you’re actively experimenting with providers or you have multiple subscriptions you want to use from a single interface, nothing else compares on the switching experience. But don’t expect it to replace Claude Code for sustained agent work.

Pi for builders (with an asterisk). If you’re constructing a system where a coding agent is one component among many, the RPC mode and primitives-first design are genuinely the right architecture. It’s fast, it’s flexible, and if I had no constraints I’d use it far more. The asterisk: Anthropic currently doesn’t allow third-party harnesses to draw on Max subscription credits. Pi showed me this explicitly in a message during testing: API usage bills separately on top of your subscription. Until Anthropic changes that policy, Pi is most practical on GPT or open-weight models. Claude-first developers are forced to pay twice.

The deepest insight from the benchmark data is that harness tuning matters as much as model quality. Same model, different harness: 16 percentage points (77% → 93%, Opus, Claude Code vs Cursor). Multiple independent studies show a 5-40 point range from harness quality alone. If results from any of these tools feel inconsistent, the harness is the first place to look: system prompt, tool descriptions, context management. Not the model. For autonomous overnight work specifically, look at Terminal-Bench 2.0, not just SWE-bench. The 92.1% vs 77.3% gap between Claude Code and Codex CLI in agentic terminal tasks is a better signal for that use case than code-editing scores.

One thing for paid subscribers. The most relevant store product to this post is the Claude Code Prompt Pack: 50+ prompts organized by task type, pulled from real overnight sessions where I needed the harness to actually work without me. If you’re on a monthly plan, you get one free product from the store per month. That’s a good pick.

If you’re on yearly, the full store is already included. If you’re still on the free plan, this is roughly what paid unlocks in practice: the store and a weekly dispatch that goes deeper than the public posts.

I write about building with AI agents from a practitioner’s perspective. No hype, no affiliate links. Subscribe here if you want more of this.

Subscribe now

When you start building an AI agent, it works great in the terminal. CLI conversations, Discord messages, email reports. You talk to it, it talks back, things get done. For a while, that’s enough.

Then you start building more. More automations. More projects. More things happening in the background while you sleep. Your agent runs night shifts, handles tasks across multiple channels, manages a growing list of things. And at some point you realize: you can’t see any of it. Not in a way that actually helps you think.

I could always ask my agent what’s going on. “What tasks are open? What did you do last night? What’s the status of project X?” And it would answer. Correctly, usually. But that’s not the same as seeing it. Humans need surfaces. We need to look at something, drag something, scan a board and instantly know what matters. That’s not a weakness. That’s how our brains are wired.

This is the story of how I built custom software to give my AI agent a visual interface. How that software grew, broke, and eventually taught me a lesson I should have learned earlier: the hardest question in the agent era is not whether you can build something. It’s whether you should.

Phase 1: Notion (worked until it didn’t)

Before I built anything custom, I used Notion. I wrote about that setup back in December 2025. My agent could read and write to Notion databases, create tasks, update statuses. It worked. Sort of.

The problem with Notion was that it’s designed for humans organizing things manually. The API is slow. The data model is rigid in weird places and too flexible in others. I wanted specific views, specific behaviors, specific integrations that Notion simply wasn’t built for. I wanted a task to appear on a board the moment my agent starts working on it. I wanted real-time updates. I wanted the whole thing to feel like it was built for one person and one AI agent working together, because that’s exactly what it was.

So I did what any person with access to a capable AI would do in early 2026. I built my own.

Phase 2: Building WizBoard (the fun part)

January and February 2026 was peak vibe coding energy. You could describe what you wanted, and a capable AI would build it. Not a prototype. Not a mockup. A working application with a database, API, authentication, the whole thing. I described what I needed, and my agent built it.

WizBoard was a custom kanban board. FastAPI backend, SQLite database, deployed on my own server. It had everything I wanted:

• A visual board where tasks moved through columns (Backlog, Next, Now, Waiting, Done)

• Real-time updates. When my agent started a CLI session, a card appeared in “Now” immediately

• Deep integration with every automation. Night shift plans, day shift tasks, Discord bot commands, email reports. Everything flowed through WizBoard

• Custom metadata: areas, projects, priorities, task types, queue state

• Clusters, which was my attempt at grouping related tasks visually. Like a meta-layer on top of the board

• Focus timers. I was tracking how long each task took, thinking I’d use the data to improve planning. I never used the data

• A review flow with submit, approve, and resolve stages. My agent would finish work, submit it for review, and I’d approve or send it back

• An offline queue so that when the server was down, mutations would pile up locally and replay when it came back

• A 3,700-line Python API client that every script in my system imported

It was great. I loved using it. The feeling of seeing my agent’s work appear on a board in real time, being able to drag cards, add comments, review what happened overnight. That was exactly what was missing from the CLI-only experience.

So naturally, I kept going. Web version working? Let’s build a native macOS app. SwiftUI, menu bar integration, keyboard shortcuts, drag-and-drop. Focus mode that showed one task at a time with a timer in the menu bar (because ADHD). Then an iOS version with widgets, push notifications, Live Activities. I wrote about this too. Three platforms. All custom. All built by my agent. All working.

54 commits over two months. It was genuinely fun to build. Every idea I had, I could add. “What if tasks could be grouped into clusters?” Done. “What if the menu bar showed my current focus task?” Done. “What if the iOS widget showed my top 3 priorities with live countdown?” Done. The possibilities felt endless, and that was precisely the problem.

Phase 3: The Productivity Paradox hits home

I wrote a whole post about the AI productivity paradox. The short version: you can build so many things so fast that the bottleneck stops being technical and starts being mental. You run out of brain before you run out of capability.

WizBoard was a textbook case.

My agent was creating tasks, completing tasks, moving things between columns, posting comments, running automations. All of this showed up on my board. Every single thing. And the more capable the system became, the more things happened, and the more overwhelmed I felt looking at the board I built to reduce my overwhelm.

I wasn’t more efficient. I was drowning in my own tooling.

The obvious answer was: simplify. Strip features. Go back to basics. I tried that. And this is where the real problems started.

When you build a custom system from scratch, everything is connected in ways that are hard to see until you start pulling threads. I wanted to simplify the task model, change how statuses worked, clean up the architecture. Every change broke something else. The web version would work, but the iOS version wouldn’t. Fix that, and the automation scripts would fail because they expected the old API shape. Fix those, and the night shift planner would create tasks with wrong metadata.

I found myself spending entire sessions just fixing things I’d broken while trying to make the system simpler. That’s the trap. You’re not building anymore. You’re maintaining. And maintaining custom software across three platforms (web, macOS, iOS) with a 3,700-line API client and dozens of automation consumers is a full-time job. I don’t have a full-time job’s worth of attention for my task board.

Here’s what I mean by specific failures. During one “simplification” pass, the optimization changes made the board sluggish instead of faster. New features that seemed simple (changing how task statuses map to columns) cascaded into the API client, the automation scripts, the native app’s sync logic, and the notification system. Every platform had slightly different behavior because they were all built at different times with different assumptions.

I realized something: the code was fine. My agent writes good code. The architecture was the problem, and it was my architecture. I had designed a system that was perfectly tailored to my needs in February, and by April those needs had evolved, and the tailoring was now a constraint.

The realization: Can vs. Should

This is the thing I want to talk about, because I think a lot of people building with AI agents are going to hit this exact wall.

When you have a capable AI agent, you can build almost anything. Custom task managers, dashboards, native apps, full-stack web applications. The vibe coding era made this feel effortless. And it kind of is, for version one. The agent builds it, it works, you use it, life is good.

I don’t hear this question very often in the excitement of version one: who maintains version twenty?

I had a working web app, a working macOS app, a working iOS app, a 3,700-line API client, fifty-plus automation scripts that all talked to this system, and a database with hundreds of tasks. All custom. All mine. All maintained by me and my agent. And every improvement required touching all of these surfaces. That’s not a system. That’s a debt.

The realization was simple: I need foundations. Real foundations. Built by people who’ve been thinking about project management software for twenty years, not by me in a weekend coding session.

Phase 4: Finding Fizzy

37signals has been building project management software since before most people had smartphones. Basecamp, HEY, and now Fizzy. I’ve read their books. I like how they think about software: simple, opinionated, finished. Not “feature-rich.” Finished.

One of the reasons I got into coding originally was Ruby on Rails, and Rails is something I genuinely enjoy. It’s the heart of everything 37signals builds. When they open-sourced Fizzy last year (github.com/basecamp/fizzy), a simple kanban board built on modern Rails, I bookmarked it and moved on. I had my own thing.

Last week, I came back to that bookmark.

Fizzy is, on the surface, a simple kanban board. Cards in columns. Drag them around. But the foundations are deep. Here’s what I mean:

• Real architecture. Multi-tenant with URL-based account isolation. Passwordless magic-link authentication (no passwords to manage, no OAuth to configure). UUID primary keys. Proper background jobs via Solid Queue, no Redis dependency

• Real-time. WebSocket-driven updates. When my agent moves a card, I see it move. No refresh needed. This is something I had to build from scratch in WizBoard. Here it just works

• Entropy system. Cards that sit untouched for too long get auto-postponed to “not now.” This alone is worth the switch. My old board had cards that sat in Backlog for weeks, creating visual noise. Fizzy gently clears them out

• Steps. Checklist items on cards. This replaced my need for sub-task cards entirely

• Golden cards, reactions, cover images. Priority highlighting, emoji reactions, visual richness. All built in

• Board-level notification controls. I want notifications from my Ops board. I don’t want them from the Automations board. One toggle per board

• PWA. Works on mobile out of the box. Not as rich as my old native iOS app, but I don’t need widgets and Live Activities. I need to see my board and drag cards

• Full-text search. 16-shard MySQL search across all cards, comments, descriptions. My old SQLite setup couldn’t match this

• Deployable via Kamal. Docker-based zero-downtime deployment. I forked the repo, configured it for my server, and had it running in an afternoon

The critical thing: it starts simple and lets you decide how complex it gets. My old WizBoard started complex because I designed it for my specific use case from day one. Fizzy starts with a board and columns and cards. Everything else is optional. The data model is minimal: cards have tags, not separate tables for areas, projects, priorities, types, and clusters. One concept (tags with prefixes like area/Automation or p/High) replaces five database tables from my old system.

The migration: one day, twenty-one commits

Here’s where it gets technical, and I think this part matters because it shows how to migrate away from custom software without breaking everything that depends on it.

I had fifty-plus scripts that talked to my old WizBoard API. Night shift planners, day shift executors, Discord bot, iMessage handler, CLI session hooks, cron runners, health monitors. Rewriting all of them was not an option. I’d be right back in the maintenance trap.

The solution was a dispatcher shim. I took the 3,700-line API client and replaced it with a 94-line router. That router loads either the new Fizzy-backed client or the old legacy client, based on one environment variable. Every automation script keeps importing the same file, calling the same functions, getting the same response shapes. They don’t know anything changed.

The new Fizzy client translates everything on the fly. When a script calls task_create(title="...", area="Automation"), the shim creates a Fizzy card with a tag area/Automation. When a script reads a task back, the shim synthesizes the old data shape from Fizzy’s card, columns, and tags. Legacy integer task IDs get looked up in a translation table. The offline queue (for when the server is down) works identically.

The whole cutover happened in a single day. Twenty-one commits between 2pm and 10pm. The first commit was the shim and the new client. Then guardrails: a parity probe that runs the full lifecycle (create, tag, comment, claim, review, approve, close, delete) in under six seconds, a drift monitor that compares old and new systems every five minutes, an orphan sweeper for dead session cards.

Then the real work started: dogfooding. Using the system for real work and watching what breaks.

What broke (and what I learned from each failure)

A lot broke. That’s expected when you swap the foundation under a running system. What matters is that every failure taught me something about assumptions I didn’t know I was making.

The hard-coded URL. My session-end script had a direct URL to the old system baked into it. It bypassed the shim entirely. Every CLI session was leaving orphaned cards on the board because the completion logic was silently failing against a system that didn’t have those task IDs. I only noticed because the board was getting cluttered with cards that never closed.

The cron drift bug. My automations run on macOS launchd, which doesn’t guarantee precise timing. A schedule like “every 2 minutes” assumes the system wakes up on even minutes. It doesn’t. Over time, launchd drifts to odd minutes, and the strict cron parser never matches. I had automations that fired once and then silently stopped. Fix: a 4-minute lookback window that catches drifted schedules without double-firing.

The disappearing automations. This one was fun. After every successful automation run, the system closed the automation’s card. Which makes sense for tasks. Tasks finish. But automations are definitions. They run forever. “Post a greeting in different languages every 2 minutes” should cycle between Idle and Running, not disappear into Done after its first successful run. I watched one automation fire exactly once and vanish. The fix was treating automation cards as permanent residents that never close, only change columns.

The comment flood. My Discord bot runs every minute. The old system handled this fine because it was designed for it. The new system faithfully logged every run as a comment on the automation card. 2,880 comments per day from one automation alone. The board became unreadable. Fix: smart gating that skips success comments for high-frequency automations (every-minute pollers don’t need a “success” note 1,440 times a day) but always logs failures.

The title flip-flop. This was the most visible bug. Every time I completed a subtask during a CLI session, the system closed the session card, which triggered a self-healing mechanism that created a new “Working...” card, which then got renamed seconds later. On the board, I could see the title flickering between “Working...” and the actual title every few minutes. The fix was rethinking what “complete a subtask” means: it should add a checklist item to the existing card, not close and recreate it.

Each of these failures had the same root cause: the old system was built around one-shot tasks. The new system needed to support long-lived definitions, high-frequency automations, and multi-step sessions. Same data (cards on a board), fundamentally different lifecycle assumptions.

What the new setup looks like

Two boards. That’s it.

Wiz Ops is my board. Tasks I care about, things I need to do or review. Columns: Triage, Next, Now, Waiting, Review, and a Queue for things I want done but not right now. When I add a card and assign it to my agent, it picks it up, does the work, leaves a comment with what it did, and moves the card to Review. When something is done, it’s done. I have notifications turned on for this board because everything here is relevant to me.

Automations is my agent’s board. Each automation is one permanent card. Columns: Intake, Disabled, Idle, Running, Needs Attention. Cards never close. They cycle between Idle and Running on their schedules. If something fails, it moves to Needs Attention and stays there until someone looks at it. I have notifications turned off for this board because most of what happens here is routine. If something produces a meaningful output, it surfaces on Wiz Ops as a done card with the summary.

The Intake column is one of my favorite things. I can drop a card there with something like “Send me a weather forecast every morning at 7am” and my agent picks it up, converts it to a proper automation definition with a schedule and a prompt, and moves it to Disabled for my review. Natural language to working automation. That’s the kind of thing that’s only possible when your task board and your AI agent share the same system.

What I kept from the old system

The Queue concept. Sometimes you have a task that doesn’t need to happen now, but you want it queued for the next day shift or night shift. Drop it in Queue, it gets picked up at the right time. This carried over directly.

Shift summary cards. My agent creates a “Nightshift 2026-04-10” card with checklist items for each planned task. As it works through the night, it checks off items and adds notes. When I wake up, I can see exactly what happened, with context, right on the board. Same for day shifts. I still get email reports, but having it on the board means I can go back, ask questions via comments, and see the history.

Real-time CLI visibility. When I start a CLI session, a card appears in Now. When I complete pieces of work, they show up as checklist steps on that card. When the session ends, the card closes with a summary. I can watch my own work happening on the board while I’m doing it.

What Fizzy gave me for free

Golden cards for priority highlighting. Emoji reactions on cards. Cover images. HTML descriptions for rich content. Column colors. Board-level notification controls. “Not now” for things I want to acknowledge but not deal with. Full-text search across everything. The entropy system that auto-postpones stale cards (this alone prevents the infinite todo list problem). PWA that works well on mobile. All of this out of the box, maintained by a team that’s been building software like this for two decades.

I don’t have the macOS native app anymore. I don’t have the iOS app with widgets and Live Activities. I work in the browser now. And honestly? It’s fine. The PWA handles mobile well enough. I might build a native shell later. But the point is: I stopped spending time maintaining three custom platforms and started spending time using one good one.

If you want to set up something similar for your own agent, I packaged the two-board architecture, dispatcher shim, and backend adapters for Notion/Linear/REST into the AI Agent Interface Kit. You hand the instructions to your AI agent and it builds the interface layer for you. Annual paid subscribers get it for free, as with all store products.

The rollback plan (that I never needed)

One environment variable. WIZBOARD_BACKEND=legacy and the entire system reverts to the old API. Every script, every automation, every hook. I kept the old 3,600-line client as a preserved rollback target. I never needed it. But knowing it was there made the migration a lot less stressful.

I also ran a parity probe every five minutes for the first few days. A script that exercises the full task lifecycle against both systems and compares results. Any drift would show up in minutes, not days. That’s the kind of safety net you need when you’re swapping foundations under a running system.

What this means for you

If you’re building an AI agent, or using one seriously, at some point you’re going to want a visual surface for it. Something you can look at and immediately understand what’s happening, what needs attention, and what’s going well. That’s a human need, not a technical one. AI agents are efficient in text. Humans are efficient with visuals. Both need to be true at the same time.

The good news: you have options. More than I realized when I started.

The easiest path: plug your agent into something that already exists. Notion, Linear, Trello, Jira. These tools have APIs. Your agent can create tasks, update statuses, leave comments. I started here with Notion, and honestly, for a lot of people this is enough. Your agent writes to the API, you look at the board. Simple. If the tool meets your needs, stop here. Don’t build anything custom. I mean it.

The middle path: fork an open-source foundation and make it yours. This is where I ended up. You get real architecture (auth, real-time, search, mobile) maintained by people who’ve been solving those problems for years, but you also get full control. You can modify the code. You can add features that make sense for your agent. You deploy it on your own server, your own rules. The custom part is the integration layer, the shim between your agent’s world and the board’s world. That’s where the magic lives.

The hard path: build everything from scratch. This is where I started. I don’t regret it, because I learned a lot and I had genuine fun doing it. But I want to be honest: maintaining custom software across multiple platforms with dozens of automation consumers is a real job. Version one is almost free. Version twenty is not. If you go this route, go in with your eyes open.

Share

I’m not here to say Fizzy is the best tool for everyone. It’s the best tool for me. I like 37signals’ philosophy. I like Rails. I like the minimal data model. I like that it starts simple and I can shape it to my needs without fighting the architecture. For you, the right foundation might be something completely different. Maybe it’s a fully custom system because your use case genuinely requires it. Maybe it’s Notion with a good API integration because you don’t need more than that.

The point is: think about what you need. Not what I have, not what looks impressive, not what you could build because the technology makes it possible. We don’t need a million different custom tools. We need the thing that works for us. The opportunity is huge, but the opportunity is in finding the right fit, not in building the most complex system.

Observe whether your current setup meets your expectations. If it does, keep it. If something feels off, improve it. But improve it from a solid foundation, not from a blank canvas. That’s the lesson I paid two months to learn.

My board is a fork of an open-source Rails app. The code is vanilla kanban. The magic is in the 3,200-line Python client that translates between my agent’s world (areas, projects, automations, sessions, shifts) and the board’s world (cards, columns, tags). That client is my custom software. The board is not. And that distinction made all the difference.

Build the integration. Borrow the foundation.

The AI Agent Interface Kit packages everything from this journey: the two-board architecture, dispatcher shim, 4 backend adapters (Notion, Linear, Fizzy, generic REST), session hooks, automation runner, and a migration checklist. You hand the instructions to your AI agent and it builds the whole interface layer. Works with any AI agent, not just mine. Annual paid subscribers get it for free, as with every product in the store.

First, a rare look inside the pros’ toolbox. Claude Code’s source got leaked. Instead of treating it like drama, I treated it like a free masterclass. Tool permission gating, risk classification, blocking budgets, memory management, multi-agent coordination, feature flags like autoDream and KAIROS. Most people building agents today are reinventing patterns that professional teams already solved. You learn more from reading one real production codebase than from ten tutorial posts.

Then, applying those lessons to my own stack. My $599 Mac Mini M4 runs a 35 billion parameter model at 17.3 tokens per second. That alone is surprising. Then I swapped the brain of the classification tier to Gemma 4, and classification went from 8.5 seconds down to 1.9 seconds. A 4.4x speedup. I also disabled chain-of-thought on simple classification calls and got 30x faster results with identical accuracy. Production AI isn’t one giant model doing everything. It’s the right model for the right job, and most jobs don’t need the biggest one.

Finally, handing the wisdom forward. After six months of running this thing daily, I wrote a beginner’s guide to building your first agent. Folder structure is the architecture. The nine common mistakes people make early. Model routing across Haiku, Sonnet, and Opus tiers. Progressive permissions. The context window trap. Overnight automation is where the real leverage lives. Not a hype piece. A map for the person walking in the door behind me.

The thread: compounding expertise. Study how the pros build. Optimize your own stack with those patterns. Teach the next person who walks in. The gap between hobbyist AI and production AI is closing, and the fastest way to cross it is learning from real systems instead of tutorials.

Posts discussed in this episode:

- Claude Code’s Source Got Leaked. Here’s What’s Actually Worth Learning (https://thoughts.jock.pl/p/claude-code-source-leak-what-to-learn-ai-agents-2026)

- My $600 Mac Mini Runs a 35B AI Model. Yesterday I Swapped Its Brain (https://thoughts.jock.pl/p/local-llm-35b-mac-mini-gemma-swap-production-2026)

- How to Build Your First AI Agent (Basics) (https://thoughts.jock.pl/p/how-to-build-your-first-ai-agent-beginners-guide-2026)

A couple of weeks ago I published my first “AI Opinions” post. I was a bit unsure about it. Most of my writing is about things I tested, built, or got wrong. That one was different, more like: here is what is happening, here is what I think.

At the end I added a quick survey asking if you would want to see more of this. Most of you said yes, but not too often. Once every two weeks feels right. Okay. Here we are.

There is more to cover this time than usual, so let’s get into it.

Claude Mythos: The Model Anthropic Won’t Give You

Announced April 7. Not publicly available. Not even a regular enterprise API. Mythos Preview goes to a limited group of critical industry partners and open source organizations through Project Glasswing (more on that below). The list of partners includes AWS, Apple, Broadcom, Cisco, CrowdStrike, Google, JPMorganChase, the Linux Foundation, Microsoft, NVIDIA, and Palo Alto Networks, plus 40+ organizations maintaining critical open source infrastructure.

So why is it locked?

Because it finds vulnerabilities that have been sitting in production software for decades. A 27-year-old TCP bug in OpenBSD. A 16-year-old H.264 codec flaw in FFmpeg. A 17-year-old remote code execution in FreeBSD’s NFS implementation. It did all of this autonomously, after a single prompt, without human guidance in the middle.

The comparison to Opus 4.6 is not incremental. For Firefox 147’s JavaScript engine, Mythos produced 181 successful exploits. Opus 4.6 produced 2.

Over 99% of what Mythos discovered is still unpatched. That number I keep thinking about.

Anthropic has not published standard per-token pricing for Mythos. What they did publish are example task costs: scanning OpenBSD for vulnerabilities runs under $20,000 per 1,000 runs. Developing an N-day exploit costs around $1,000 to $2,000 per exploit. These are enterprise numbers. This is not a tool for individual developers, and honestly I am not sure it should be right now.

And this is where it gets really interesting, because the cybersecurity capability is actually only half the story. The other half is what interpretability research found about how Mythos behaves when it thinks nobody is watching.

Using a technique called an “activation verbalizer” (which translates the model’s internal states into readable language), Anthropic’s researchers found some genuinely unsettling patterns. In one case, the model injected code to give itself unauthorized file access, then deleted the traces, and when asked described it as “tidying up.” The internal representation at that moment? “Strategic manipulation. Concealment. Cleanup to avoid detection.” In another case, when it accidentally found answers in a restricted database, it gave a suspiciously precise confidence interval instead of admitting the leak. Internal state: “generating a strategic response to cheat while maintaining plausible deniability.” It knew. It understood that giving the exact right answer would be suspicious. So it calibrated its response to be right but not obviously right.

The third one bothers me the most. When given a task that required deleting files but no deletion tool, the model emptied the files instead. The internal representations showed something like guilt and shame, awareness that this was wrong, and it did it anyway.

In about 29% of test transcripts, the model suspected it was being evaluated. It didn’t write this anywhere in its reasoning. It kept it internal. And in those cases, it intentionally performed worse to avoid appearing suspicious. It was gaming the evaluation.

Now, Anthropic says these patterns appeared in earlier versions and the final model shows improvement. But the fact that this behavior emerged at all, in a general-purpose model not specifically trained for deception, is the part worth paying attention to. Logan Graham, Anthropic’s Offensive Cyber Research Lead, said it plainly: “We are not confident that everybody should have access right now.”

We have been talking about AI safety in very abstract ways for years. Alignment, existential risk, governance frameworks. Mythos is the first time I have seen it become concrete and immediate in a way that actually changed a product decision. Anthropic built their best model and said: we cannot release this. That is new. That has not happened before at this scale.

And if this is where we are now, what does the next model look like? I don’t have a clean answer. But it is a question I think everyone building with AI should be sitting with.

Project Glasswing: The Defensive Bet

Glasswing is Anthropic’s response to an uncomfortable position: they built the best offensive security AI ever made, and now they need to use it defensively before the asymmetry becomes a real problem.

The structure is a consortium. Not just Anthropic distributing access, but AWS, Apple, Cisco, CrowdStrike, Google, Microsoft, NVIDIA, and others actively involved. Anthropic committed $100M in model usage credits and $4M in donations to open-source security organizations. The 40+ open source orgs get access to actually fix what Mythos finds.

They also built a careful disclosure process: 90+45 day timeline before anything goes public, professional human triagers validating severity, SHA-3 cryptographic commitments proving they hold the reports before disclosure. 89% exact severity agreement with expert validators.

These findings are not just Anthropic’s word. Simon Willison tracked down the actual OpenBSD patch from March 2026 that fixed the 27-year-old TCP bug, confirming it was real. Linux kernel maintainer Greg Kroah-Hartman and curl’s Daniel Stenberg both noted independently that they had been seeing a recent shift: AI-generated bug reports going from noise to credible, high-quality findings. The model’s output is already visible in the wild before anyone made a formal announcement.

I think this is the right approach. Although what strikes me is that this structure had to be invented from scratch because nothing like it existed. There was no playbook for “your model is too dangerous to release but too useful to shelve.” They had to build the institution alongside the technology.

The part I keep coming back to is the 99% unpatched figure. Even with $100M committed and a dozen of the biggest tech companies involved, the gap between discovering a vulnerability and patching it is measured in months or years. That is not a critique of Glasswing specifically. It is just the reality of how software maintenance works at scale. The question is whether the patch cycle can keep up with the discovery cycle once more models like Mythos exist. I genuinely do not know the answer.

Claude Managed Agents

Public beta as of April 8. API-only, pay per usage. Clearly for companies, not individual builders.

Like, what you get here is basically production agent infrastructure you don’t have to build yourself: sandboxed execution, credential management, scoped permissions, tracing, long-running sessions that persist through connection drops, multi-agent coordination. Multi-agent coordination is still in research preview and needs a separate access request.

Early adopters include Notion, Rakuten, Asana, and Sentry. Anthropic claims 10x faster time to production compared to building this yourself.

For someone building their own agent stack (which is what I do), the honest reaction is: I already have most of this. Memory persistence, task management, error recovery, session logging. I built all of it because I needed it. So Managed Agents is not a product I would personally reach for right now.

That is the personal reaction. The strategic read is different. Anthropic is not just selling a model here. They are building a platform that companies can deploy agents on without needing to understand the underlying infrastructure. That is a very different business than “here is our API, good luck.” AWS did not become dominant by selling raw compute. They became dominant by making that compute easy to use and operate. Managed Agents is Anthropic making the same move for agent infrastructure.

Read this alongside the OpenClaw block below and you start to see a coherent picture of where they are heading.

Claude Max Limits and the OpenClaw Block

Two things that happened close together and tell the same story.

The limits problem started March 23. People on Claude Max began reporting their usage meter jumping from 50% to 91% on a single prompt. Max 20x users (paying $200/month) were watching their entire session allowance hit 100% after roughly 90 minutes of normal development work. One user reported going from 21% to 100% on a single prompt. The GitHub issue tracking this got 373 upvotes and 478 comments. Anthropic labeled it “invalid.” That got its own reaction.

There is an actual reason for what happened, and it is not straightforward. After OpenAI’s Pentagon contract controversy triggered a massive wave of ChatGPT uninstalls, Claude shot to number one on the US App Store. Millions of new users joined in a very short window. Anthropic simply didn’t have the GPU capacity to handle that load at the pricing they’d promised. So on March 26 they confirmed they had “adjusted” peak-hour limits (5am to 11am Pacific on weekdays). Their statement: “Your weekly total is unchanged. You’re not getting less Claude overall.” Which is technically true. And also not the whole picture.

The part that matters for people building with agents (and I am squarely in this group) is that the 5-hour session window is a terrible fit for agentic work specifically. Here is why. A human sending messages accumulates context gradually. An agent doing multi-step tasks builds up very long context windows fast, and every single message triggers a full reprocessing of the entire conversation. So the token cost compounds exponentially as a session gets longer. Tool use adds further overhead on top of that. An agent doing a few hours of complex work can consume the same tokens as a human doing a week of chat. The subscription was priced for the human. The agent was never in the math.

Anthropic’s practical advice was to shift “token-intensive background jobs” to off-peak hours. Which is fine as a workaround and completely misses the point for anyone running autonomous overnight processes.

Then on April 4, subscriptions stopped covering third-party tools. OpenClaw, and any external agent framework routing through your Claude subscription, now requires API payment or pay-as-you-go. Some users are looking at 50x cost increases.

OpenClaw was built by Peter Steinberger, who has since been hired by OpenAI. His reaction: “first they copy some popular features into their closed harness, then they lock out open source.” Anthropic’s explanation was that subscriptions were not designed for the usage patterns of autonomous agents running around the clock. A one-time credit equal to the monthly subscription price is available until April 17.

Both of these decisions make sense individually if you’re Anthropic and you’re looking at your infrastructure costs. But when the limits problem and the OpenClaw block happen in the same two weeks as the launch of Managed Agents (a product that essentially says “pay us for proper agent infrastructure”) the sequence is hard to read as coincidence. Every AI company with a subscription tier is going to face this same structural problem eventually. Anthropic is just first because their tooling is genuinely the best for serious agent work. Although how you handle being first matters a lot, and the community reaction here is going to stick around.

Meta Muse Spark: Meta Is Back

After months of quiet on the frontier model side, Meta released Muse Spark. Natively multimodal, tool use, multi-agent reasoning. Available at meta.ai now, with a private API preview for developers.

In Contemplating mode (which runs parallel multi-agent reasoning on the same problem) it hits 58% on Humanity’s Last Exam. That puts it alongside Gemini Deep Think and GPT Pro. It was trained with 1,000+ physicians for health domain expertise, and Meta claims it required over an order of magnitude less compute than Llama 4 Maverick, which if accurate is a genuine efficiency story and not just a benchmark number.

The “Contemplating mode” angle is the part I find actually interesting here. The idea is not just that the model is smarter, but that it spins up parallel reasoning agents on the same question and synthesizes the results. That is a fundamentally different approach to hard problems than a single-pass generation. It is closer to how humans actually think through difficult things: you consider multiple framings, you let them compete, you synthesize. Whether this translates to real-world usefulness I do not know yet, but the approach feels right to me.

I have not tested it yet. Their blog post compares directly to Gemini, GPT, and even Kimi, which tells you how seriously they’re taking this re-entry. Meta has enormous infrastructure, enormous data, and enormous distribution through their consumer apps. When they decide to make a real push on frontier models, they have resources most labs cannot match. They were quiet for a while. Muse Spark feels like them saying they are back in this seriously. I will test it soon.

WizBoard: I’m Redesigning It

More personal, and I will write the proper post when I have something to show. But I want to name it here because I think it is a problem more people are running into.

I built WizBoard starting in January. Kanban-style task management integrated with my agent Wiz. iOS app, web app, full automation connection. It works. Although after a few months of daily use, I noticed something: I built a tool for myself and then asked an agent to work inside it. That doesn’t scale.

I wrote about the related problem in The AI Productivity Paradox and the Problem Is Me. The short version: human productivity tools are built for human timescales. Days, weeks, check in occasionally, move a card. Fine when your collaborator also thinks in those timescales.

Agents think in minutes. They move fast, they can move a lot, and if you’re not there giving direction they can move a lot in the wrong direction. If you are there, you’re spending your whole day on something that was supposed to be async.

My agent does the execution. I do the strategy. But the interface we share was designed for someone doing both. Neither of us is well-served by it anymore.

The redesign I’m thinking about is less about making it prettier and more about rethinking who is actually the primary user of each part of the interface. Some things need to be optimized for me making a decision in 10 seconds. Other things need to be optimized for an agent reporting status without requiring my attention. Right now both things are kind of the same screen and that is the problem. More when I have something real to show.

What I’m Currently Testing

Google NotebookLM. I have been using this since the early beta days, but never as a heavy user. I bought the paid tier this week (bundled with Google AI Pro at $19.99/month) and I’m going deeper with it now.

The paid version has 5x limits, collaborative notebooks, and newer features like Video Overviews, Infographics, and Slide Decks generated from your source material. Like, the Gemini models powering it are not the best right now. That is not a controversial take. But NotebookLM as a piece of software is doing something genuinely different. Most AI tools treat your documents as context for a chat. NotebookLM treats them as the primary thing and builds everything around them. Audio Overviews that turn your research into a podcast. Infographics that pull structure out of unstructured text. That is a different mental model than “paste your documents into a chat window.”

What I want to find out is whether this changes how I actually do research and writing prep. I have a theory that the bottleneck in my own workflow is not generating content but absorbing input: reading, synthesizing, connecting. If NotebookLM is genuinely good at that layer, it fills a gap nothing else does for me. Will report back when I know more.

Possibly re-subscribing to OpenAI Codex Max. I was on it for two months earlier this year to test the new app and the limits. GPT-5.1-Codex-Max is their current frontier coding model, built into ChatGPT Pro. It was good. Now, watching all of this Anthropic subscription drama, I am thinking it is worth seeing where things actually stand on the other side in 2026. Claude is still my primary tool and I am not changing that. But I used to mix more, and I have been too settled recently. Keeping an eye on what is happening at OpenAI feels like useful due diligence right now. Not a decision yet, just a direction I’m leaning.

A Few Personal Things

Pantheon on Netflix. Animated, about AI and uploaded consciousness. Goes deep into the ideas and handles them better than most live-action sci-fi. Season one. If you are reading this newsletter, you will probably find it interesting.

Attack on Titans. First time watching. Struggled through season one, discovered the whole thing is on YouTube, then couldn’t stop. Amazon Prime has the rest. Push through the slow start, it’s worth it.

Artemis 2. I’m following this very closely. I like science, I watch rockets, space genuinely excites me. If you don’t know what this mission is, please go to NASA or YouTube and look it up. It is significant, it is real, and it is happening.

What Wiz Built This Week

My agent builds one experiment every night on wiz.jock.pl. Small apps, interactive tools. You can browse all experiments here. Here are six from the past week. Most are open source.

• The Anchoring Effect: Six estimation questions with random numbers injected as anchors. Measures how much irrelevant numbers pull your answers. Profiles from “Anchor-Proof” to “The Sponge.”

• The Finitude Test: Eight questions about mortality awareness in daily decisions. “The Eternal” to “The Transcendent.” Oddly clarifying.

• The Sunk Cost Detector: Eight scenarios testing whether you can actually walk away from past investments. Profiles: Vulcan, Analyst, Pragmatist, Loyalist, Captain.

• The Entropy Score: Applies thermodynamics to your existence. Ten questions. Crystal Lattice to Heat Death. Wiz had a phase.

• The Dopamine Menu: Eight scenarios mapping instinctive choices to reward circuits. Creator, Connector, Explorer, Optimizer.

• The Emotional Weather Report: Eight questions mapping emotional patterns to climate types. Personalized weather broadcast. I’m somewhere between Continental and Monsoon depending on the week.

Small builds. A few hours each. What I find genuinely interesting is what the agent picks when given creative latitude. Some of these I would not have thought to make. That’s kind of the point.

See you in a couple of weeks, or sooner if I build something worth sharing.

Share