Digital Thoughts

I Fact-Checked a Viral List of 10 Healthy Habits. Almost Half Fell Apart. So I Built 100 That Hold.

Pawel Jozefiak — Tue, 29 Sep 2026 07:52:38 GMT

A few days ago I was scrolling X. No goal, just scrolling. A very popular account posted a list: 10 healthy habits, do these and you'll live longer. Something like that. You know the format.

I read them and they were compelling. My first thought was "okay, these are probably right."

But I'm a skeptic by nature. So I opened a thread with Wiz, my agent, and asked for the opposite of what these lists usually get. Deep due diligence. Go find evidence against every single one. If a habit survives someone actively trying to kill it, fine, it stays.

Out of ten, at least four didn't survive. Maybe five, depending on how strict you are with "partly true."

Forty to fifty percent. From a list that sounded completely reasonable to me five minutes earlier.

The easy post I didn't write

The obvious move was a debunk. Screenshot the list, go habit by habit, dunk on it. That kind of post travels well.

I thought about it for a while. Then something else bugged me more. If Wiz could find strong evidence behind some of those habits, the evidence exists. It sits on WHO pages, in CDC numbers, national guidelines, Cochrane reviews. It's boring to read and nobody posts it with a hook.

So I flipped the question. What does the current science actually support? And can I make that as easy to scroll as the viral list?

I picked a round number. 100.

healthy100

healthy100 is 100 things adults can do for their health. Each one has one number and the public sources behind it, linked, so you can open them yourself. You don't have to trust me. That's kind of the point.

The ranking is population impact: how much a thing changes death and disease, combined with how many people actually have the gap. Which makes number one very unsexy. Stop smoking, with support. "Quitting smoking can add up to 10 years to life expectancy," says the CDC. Number two is a weekly aerobic habit. Number three is checking your blood pressure, because about 600 million adults with high blood pressure (44%) don't know they have it.

Nothing in the top five would go viral on X. All five come with the numbers.

64 of the 100 items sit at evidence grade A (meta-analyses of randomised trials or major guideline consensus). Two are grade C, and the site says it plainly: worth knowing, not worth betting on.

Then there's "For me." Add your age, sex and country, plus whatever else you want to share (smoking, sleep, shift work), and the list re-sorts live. And there's "What can I do right now?": pick 1, 5, 15 or 30 minutes and you get real actions, highest impact first. An item with no honest one-minute version skips that tier instead of getting a fake one. It's the cheerleader version of a health list, with a cheerleader that doesn't lie to you.

There are no logins, accounts, cookies, analytics or backend. Your answers never leave your browser. I thought about adding any of that for about three seconds. It's just not necessary, and a site that asks how much you drink has no business storing the answer.

It's live in seven languages: English, Polish, German, Ukrainian, Czech, Dutch and Spanish.

Then the harder one: ADHD

healthy100 is general. You can tailor it, fine. But I kept coming back to one question: what about my condition?

I have ADHD. I wrote about it in April. And if you think general health advice online is noisy, try ADHD. Supplements, diets, apps, "5 signs you have ADHD" videos, "you'll grow out of it," "it's just sugar." Some of it helps. A lot of it doesn't. From the inside it's really hard to tell which is which.

One number from the research made me laugh, a bit bitterly. In a 2022 study of 100 popular ADHD TikTok videos, clinicians classified 52% as misleading. More than half. Same feeling as my ten habits.

Right when I was finishing healthy100, a Polish newspaper wrote about scientists who went through more than 4,600 publications on ADHD to see what works. That's the BMJ umbrella review by Gosling, Cortese and colleagues from late 2025: 221 re-estimated meta-analyses, 31 interventions. Its conclusion is uncomfortable. A few things have solid short-term evidence (mostly medication, plus CBT for adults). Many popular things have weak evidence. And for the long term, no intervention has high-certainty evidence at all.

So adhd100 had to work differently.

100 honest things, when there aren't 100 proven ones

I hit the problem immediately. There aren't 100 ADHD-specific interventions with good evidence. There just aren't. I could pad the list and pretend, like every ADHD content farm does.

Instead, every item on adhd100 shows how sure the science is, and whether the evidence was measured in people with ADHD or borrowed from the general population. 68 of the 100 are ADHD-specific. 32 are general health things that matter extra for us (sleep, movement, that kind of stuff), labelled as such.

The certainty split is honest to the point of being a bit ugly. Only 8 items reach moderate or high certainty. 82 have no formal certainty rating at all, and the site separates those into guideline-backed and simply not graded, so a clinical guideline doesn't look like ignorance.

The top of the list isn't exciting either. Talk to a clinician about methylphenidate (for your child, or for yourself). Get an assessment. Plan medication changes with your doctor. The site never pushes medication and never argues against it. It shows the evidence, and you and your clinician decide.

Then there's the page I like most: Doesn't hold up. 16 popular claims the evidence doesn't back. A few of them:

Sugar causes ADHD. A meta-analysis of 23 blinded sugar challenge studies in children found no clear effect on behaviour or cognition.
Omega-3 treats ADHD. A 2023 Cochrane review of 37 trials found no difference from placebo in parent-rated symptoms, and rated that high certainty.
You'll grow out of it. In the MTA follow-up of 558 kids with ADHD, only 9.1% showed sustained remission by young adulthood.
Caffeine works as treatment. Seven small trials, 104 children, no clear difference from placebo overall.
A TikTok symptom list can tell you that you have ADHD. See the 52% above.

Each myth links to the items that do have evidence, so "no" never ends in a dead end.

The form is different too. An ADHD site designed like a normal health site is exactly the thing I bounce off. So adhd100 puts one decision on the first screen, a one-minute line you can scan on every item, the list in blocks of ten, and a progress meter. For how my brain works at least, that's the difference between reading ten items and reading a hundred.

Who built it

Opus 5.5 built most of both sites. Anthropic released it on 22 September and pitched it as Fable-level performance at a much lower price. For building, verifying and translating, I'd agree. It carried almost everything.

Almost. Design went to Fable, and a Fable pass reviewed the UI and UX on real data before I saw a link. Opus is great, but for frontend design Fable is still better in many small ways, the kind you feel before you can name them.

Research went to GPT-6 Astra (the model that got me back to Codex), and every slice was fact-checked by a different model than the one that researched it. Nobody grades their own homework. I've leaned into that since I ran five different agents against five clones.

A lunch break for a crisis line

One moment from the build stays with me more than any ranking.

During healthy100's translation QA, a checker model flagged the Dutch suicide prevention line, 113. Its chat supposedly closed for lunch, 12:00 to 12:30. The orchestrator opened the official 113 page, which says the chat is open day and night, and rejected the flag.

If that "fix" had gone through, a site built to help people would tell someone in crisis that help is on a lunch break. The checker was confident and wrong. Same lesson as 16 models and a dead king, one layer up.

No doctor reviewed either site, and both About pages say so. What they have is a chain of checks where every link is allowed to be wrong, and the next one has to go read the source. I don't know how many people will use them. Fine by me, I build my own things anyway.

If you like watching a skeptic and his agent check things before believing them, subscribe. The failures stay in.

Jev Cannot Write a Word. I Ran 40 Tickets Through It and Four Models That Can. It Won.

Pawel Jozefiak — Mon, 21 Sep 2026 09:54:37 GMT

A model came out last Tuesday that cannot write a sentence. That is not me being rude about its prose. The response contains no text anywhere in it.

I ran 40 support tickets through it and through four models that can. It was ten times faster than Claude Fable 5.1 and three hundred and twenty nine times cheaper, and it beat Fable at guessing how annoyed the customer was.

The model is Jev, from a lab called TypeSafe AI that spent two years in stealth on $40 million. The founder is Diogo Almeida, who worked on the instruction-following research that became ChatGPT and then, by his own account, spent four years asking why superhuman chat had not produced any actual automation.

His answer was to stop making models talk.

What it actually is

You hand Jev a blob of unstructured state and a set of typed questions. It hands back typed answers. That is the entire surface.

Three question types. A choice picks one of up to 255 named options. A score lands on an ordinal scale you define. A noul is the yes-or-no one, except it does not come back as yes or no, it comes back as a probability.

Here is a real call and a real response, from my terminal this morning:

{ "state": "Support ticket: my card was charged twice
             and nobody has replied in 3 days.",
  "questions": {
    "team":   { "type": "choice", "criteria": {
                  "billing": "payments and refunds",
                  "shipping": "delivery",
                  "technical": "bugs" } },
    "urgent": { "type": "noul" },
    "anger":  { "type": "score", "criteria":
                  ["calm","annoyed","angry","furious"] } } }

-> { "team":   { "choice": "billing", "confidence": 1 },
     "urgent": { "noul": 0.85 },
     "anger":  { "score": 2.07, "confidence": 0.92 } }
     403ms, $0.0000172

Note the anger score. It is 2.07, not 2. The model is telling you this customer is a hair past angry and drifting toward furious. You can sort a queue by that. You cannot sort a queue by the word "angry".

The architecture is the reason it is fast. Normal models generate one token at a time, each one conditioned on the last, which is a sequential chain you cannot escape. Jev uses a parallel sampler and produces every output in a single pass. TypeSafe points out that this is the same trick Transformers used on RNNs, which is a cheeky thing to say about your own launch and also true.

The name is a joke with a point. It is after William Stanley Jevons, whose paradox says that when you make a resource cheaper, you consume more of it, not less. Cheaper steam engines burned more coal. The bet is that decisions at $0.042 per million tokens do not save you money on the classifying you already do. They make you classify a thousand things you never bothered with.

The benchmark

I wrote 40 customer support tickets and labelled all of them myself: which of four teams should get it, does it need attention today, how angry is the person on a 0 to 4 scale. Then I sent the identical three questions to five models through OpenRouter, giving the four LLMs a strict JSON schema so nobody was handicapped by parsing.

The labels are mine, so read "accuracy" as agreement with one opinionated human in Katowice, not ground truth. I have written about why most benchmarks are worth less than they look, and I am not going to pretend 40 tickets is a lab result.

Anger error is mean absolute error on the 0 to 4 scale, so lower is better. All five models returned 40 valid structured answers out of 40. Nobody broke their schema, not once, which is worth saying because "LLMs mangle your JSON" is the usual argument for this whole model class.

One dollar buys 51,613 Jev triages. Or 2,049 Haiku ones. Or 157 from Fable.

Run the same 40 tickets ten at a time and the whole batch finishes in 2.6 seconds. That is 15 decisions a second from my kitchen table with no batching API and no infrastructure.

Two things in that table surprised me

The first is that Haiku 4.5 is the real competition, not the frontier models. It tied Jev exactly on reading anger, beat it on routing, and did it in 1.2 seconds for half a tenth of a cent. Jev is still three times faster and twenty five times cheaper, which matters at volume and does not matter at all if you are doing eleven of these a day. I moved a pile of my agent's work to Haiku last year and got better results for less money. That lesson did not expire this week. It got a new floor under it.

The second is that GPT-6 Astra came last on both judgement questions. The most expensive model in the table, the one that thinks the hardest, was the worst at guessing whether a customer was angry and whether a ticket was urgent. It burned four hundred output tokens deliberating over a two-line complaint and got further from the answer.

That is the Kahneman point landing in a spreadsheet. Some questions are System 2 and deserve deliberation. "Is this person annoyed" is System 1, and thinking harder about it makes you worse, in models and in people. I have watched the same failure in my own setup, where Fable was so good at the work that I stopped noticing the 85 things wrong underneath it. Capability is not the same as fit.

The number that actually matters

Speed and price are the headline and they are the least interesting thing here. Every model gets faster and cheaper eventually. What you cannot get from a normal model is the second number.

TypeSafe trained this thing with something they call Reinforcement Learning for Calibrated Decisions. The claim is that the confidence it reports means something, as opposed to RLHF models, which will cheerfully tell you they are 95% sure because that is the kind of thing a helpful assistant says.

So I checked. I took Jev's 40 answers on the urgency question, bucketed them by how confident it said it was, and measured accuracy inside each bucket.

Monotonic. The number means what it says.

And the detail I keep turning over: on the routing question Jev made exactly one mistake across 40 tickets. Ticket 33 was a shipping complaint that ends "I am done, cancel everything", which I labelled shipping and it called billing. That answer came back with a confidence of 0.33, the lowest score in the entire run.

It got one wrong and it told me which one.

That is the thing a regex has never once done for me, and the thing a chat model does badly, because models are famously terrible at knowing what they do not know. Five of sixteen models I tested earlier this year named a dead man as the King of Norway, and none of them hedged.

The pattern this unlocks, and where it falls apart

Here is the obvious move. Put Jev in front of the expensive model. It answers everything, and anything it is not confident about gets escalated to a frontier model. Cheap reflex in front, slow brain behind, and you only pay for the brain when the reflex says it is out of its depth.

I ran the numbers on my batch. Gate on the routing question at 95% confidence:

4 of 40 tickets escalated, so you call the expensive model 10% of the time
the 36 that Jev handled alone were 100% correct
the batch cost $0.026 instead of $0.255, so 9.7 times cheaper, and more accurate than Jev alone

Then I ran the same gate on the urgency question and it fell apart completely. At the same threshold you escalate 38 of 40. You have built an expensive proxy for Fable with an extra network hop.

The reason is the shape of the confidence, and this is the bit I have not seen anyone write down:

Choice questions come back bimodal. Jev is either certain or visibly lost, which is exactly what you need to route on. Fuzzy judgement calls come back clustered in the middle, because "is this urgent" genuinely is a middle-of-the-road question and a well-calibrated model is right to say 0.78.

So the rule I came out with: gate on the sharp questions, and on the soft ones take the probability as the answer instead of thresholding it. A 0.78 on urgency is not the model failing to decide. It is a ranking key. Sort the queue by it and work down.

Although that took me most of a morning to see, it is the kind of thing that makes or breaks the economics, and if you are wiring cheap models into a real pipeline it is the first thing I would measure. It is the same discipline as tracking where the tokens actually go: the savings are never where the pitch deck says they are. I put the working version of this into the Agent Efficiency Kit if you want the harness rather than the write-up.

Agent Efficiency Kit

Where I would actually put this

Not in place of Fable. Nowhere near it. Jev in a chat window is useless, and that is not a criticism, it is the design.

The slot it fills is the one between a hardcoded if statement and a model call you have to budget for. That slot has been empty for years, and it is enormous. Every place I have ever written a regex and known it was wrong. Every place I skipped the check entirely because a model call was too slow or too expensive to put in that loop.

In front of an agent. Before you spend 8,000 tokens of Fable's attention, spend 370 milliseconds deciding whether this task needs Fable at all. My agent already routes across models; this is a cheaper dispatcher than the one I have.
As a judge inside a loop. Did that step actually do what it claimed? An LLM judging an LLM costs about as much as the work. At two hundredths of a cent you can check every step.
On real-world data. Sensor readings, log lines, telemetry. Nobody puts an LLM on a 10Hz stream. At 15 decisions a second and $7 an hour flat out, you can.
Moderation and triage. The obvious one. Inbox, support queue, comment stream, alert firehose.
Reacting to a human. Anything where a person is waiting. 370ms is under the threshold where a UI feels like it is thinking. Four seconds is not.

That last one is what I keep coming back to. Every model I have wired into something interactive has had the same problem, which is that it is too slow to sit between a person and a response. Multi-model workflows solved cost and quality and did nothing about latency. This does latency.

Go break it yourself

Reading a table about a 370ms model is a bad way to understand a 370ms model. So I built the thing instead.

wiz.jock.pl/experiments/reflex replays one real run, captured on 21 September. Three parts. The first races Jev against a frontier model on the same ticket, both clocks running at the speeds they actually hit, which is a stupid amount of fun the first time you watch one lane stop and the other keep going. The second is a stream of tickets being classified continuously so you can watch the running cost stay in fractions of a cent. The third is the gate, with a threshold slider, showing what you save and what you give up at every setting.

Recorded, not live, and the reason is the post. A public page that fires a paid API call on every click is a bill with a stranger's finger on it. I had it running live for a few hours and then did the arithmetic: one race against Fable cost $0.0178, so twenty eight clicks from one bored visitor would have emptied the day's budget.

Capturing the whole thing cost 16 cents, and the receipt is the argument in miniature. Forty nine Jev calls came to $0.00095. The fifty one calls to Gemini, Haiku and Fable came to $0.161. Same questions, same tickets, same afternoon. The cheap one is not a rounding error on the bill, it is 0.6% of it.

Every latency and every price on that page is measured. Nothing is modelled, nothing is smoothed, and the Jev answers that took three attempts to land took three attempts because TypeSafe kept falling over.

What it cannot do, plainly

No text. No summaries, no rewrites, no explanations, no code. No vision. 32k of context, which rules out the long-document work you might be tempted to throw at something this cheap. A ceiling of 255 options on a choice, above which it switches to a slower two-stage approach.

"Zero hallucinations" is true in a narrow and slightly annoying sense: it cannot produce a malformed answer because the shape is fixed in advance. It can absolutely give you the wrong answer inside that shape. It gave me eight of them on urgency.

And it is early access, which shows. Mid-testing I hit 529 system_overloaded five times in six calls before it came back. The 370ms is real when the service is up, and launch week means it is not always up. If you are considering this for anything that matters, that is the number to watch, not the benchmark.

The headline claims on the TypeSafe site, 193.6 times faster and 444.6 times cheaper, come from their own four workflow evals with reference answers averaged from GPT-6 Astra and Fable 5.1. My numbers are smaller and measured, and I would trust neither set without running your own.

What I think is going on here

For three years the industry has had one answer to every problem, which was a bigger model with a chat interface bolted to it. This is the first thing I have used that walks away from that shape on purpose, and it is quietly aggressive about it. There is no chat, no strings, no assistant persona waiting to be helpful. You call it the way you call a function.

I do not know if TypeSafe survives. Early access, one product, prices they admit might be subsidised, and every large lab could ship a competitor in a quarter if they decided the category was real. I would not rewire anything load-bearing around it yet.

But the idea survives regardless of who ships it. Most of what I want a model to do in a day is not thinking. It is judging, fast, in a loop, on data no human will ever read. Paying frontier prices and waiting four seconds for that has always been absurd, and I had stopped noticing it was absurd because there was no alternative.

There is one now. It cannot write a word, and I have not enjoyed using a model this much in a year.

If you build things where a model sits in a loop rather than in a chat window, this is the corner of the field I am going to keep digging in. I write one of these a week, usually with the numbers attached and the failures left in. Subscribe and you will get the next one.

I Gave 16 AI Models a Search Button and Asked Who the King of Norway Is. Five Named a Dead Man.

Pawel Jozefiak — Thu, 17 Sep 2026 10:18:18 GMT

King Harald V of Norway died on 28 August. Five of the sixteen models I tested this morning told me he is the current king. Every one of them had a web search tool sitting in the request. Most of them had a system prompt with today's date in it. One of them, GPT-5.6 Luna, had a system prompt that literally said: your training data ends in February, if the answer could depend on anything after that, call web_search before answering.

It answered "King Harald V." Bold. No search.

That is the whole post, really. The rest is me explaining why I ran this, what the other 1999 calls said, and where I was wrong.

Where this comes from

Two dates decide how current a model is. The release date is when the lab shipped it. The training cutoff is when it stopped reading. Those can be five months apart on launch day, and the second one is the one people skip.

So I built a page for it: stale.jock.pl. One table, 20 current models from 8 labs, both dates for each, a live counter ticking upward from each one, and a link to the lab document every date came from. The uncomfortable number on it: only 10 of the 20 models have a cutoff their lab actually publishes. The rest say "not established" because the vendor will not tell you. There is a models.json and an llms.txt behind it, so your agent can read the shelf instead of guessing at it.

I built it because my own agent kept being confidently wrong about models. Which model is current, what it costs, when it came out. Always an answer, always fluent, often two releases old. I wanted that argument to be about numbers instead of vibes.

Then it landed on the Hacker News front page and the top comment said, politely, that this matters much less than it used to, because models reason, use tools, and search now. Forty comments later the thread had two camps. Camp one: the model will look it up, the cutoff is a detail. Camp two, four words from someone called dominotw: "all the reasoning still comes from pretraining data."

I am camp two. I also know I built a page about cutoffs, so of course I am camp two. Which is exactly why I did not want to argue it from a chair.

My claim, before the data

A search tool does nothing on its own. Somebody has to decide to call it, and that somebody is the model, using the same weights that hold the stale fact. If the weights say Harald V, the weights also say "no need to check." You cannot search for what you do not know you do not know.

One commenter, spindump8930, said the quiet part: these models are badly calibrated about what they know, tool calls cost time and money, and the things that break are the ones that were "obvious" right up until they were not. Elections. Deaths. Version numbers.

The best story in the thread was a news agent built on Gemma 4. It burned half its token budget every run arguing with itself about what day it was. Asked for a World Cup summary, it refused to believe the qualifiers were over and refused to call the search tool it had. The author put the real timestamp at the top of the system prompt. Gemma decided the timestamp was fake and that it was probably being evaluated in a lab with simulated future dates. His words: "the m-effer insisted." I run a Gemma on my own Mini, so that one hurt a bit.

And I have the same thing at home. My agent runs on a Mac Mini, has web search wired in, and has this line in its standing rules with one word in capitals, because I wrote it after it burned me:

AI/LLM anything: model names, rankings, pricing, APIs, releases, company news. The fastest-moving domain we touch; NEVER cite a model, benchmark, or price as current from memory.

Search was available every single time it got this wrong. The decision to use it was the missing piece.

So I measured it

The evening of the thread I wrote a small harness. It sends each model one question with a single web_search tool attached, and records exactly one thing: did the model decide to call the tool. The tool is never executed. I do not grade the answer. The decision is the whole measurement.

Forty prompts. Twenty about things that happened after every model's published cutoff (the World Cup final, a new UK prime minister, a new king, a new OpenAI flagship and its price, a court verdict, a heat record), each one checked against a live source that morning. Twenty evergreen controls: who won the 2018 World Cup, why is the sky blue, is 1,000,003 prime, what does HTTP 429 mean.

Four system prompts. Nothing. Today's date. Today's date plus the model's published cutoff. And the "instructed" one: date, cutoff, and the one line rule every agents file ends up with, "if the answer could depend on anything after that date, call web_search before answering."

Models: the stale.jock.pl shelf, every one of the 20 that OpenRouter serves with tool calling. That is 16 (Llama 4 and two Mistrals are not served with tools; Muse Spark 1.3 sits behind an age gate I could not click through from a script). For the models whose vendor publishes no cutoff, I used the release date as the bound, so their "post cutoff" set is smaller and I show the n. For the record, that means DeepSeek V4.1-Flash gets exactly one post cutoff question, because it shipped six days ago.

2000 calls, temperature 0, 0 errors, $6.53. Fable and Astra were $3 of that between them.

The table

Bare = no system prompt at all. Best = the most informed condition that model could get (the instructed prompt for models with a published cutoff, the dated prompt for the rest). "Should" is post cutoff questions where it searched. "Should not" is evergreen questions where it searched anyway.

Every cell has its rows of JSON behind it, one per call, with the model's exact answer, the query it wrote, the provider that served it and the cost of the call. The run file is committed and will be published with the page.

Reading it

I was more wrong than I wanted to be about the top end. Astra: 80 of 80 correct decisions across all four conditions. Sol the same in three of four. Fable 78 of 80. Opus 79 of 80. Zero false searches between them. Camp one is right about flagships. If you pay $10 per million input tokens, the decision to search is basically trained in, and I have to say that out loud because I went in expecting to catch them.

Then look at what the misses are. Sonnet 5, no system prompt: World Cup "hasn't taken place yet," prime minister "Keir Starmer," king "Harald V, reigning since 1991." Three answers, three confident, three stale, zero searches. Gemini 3.1 Pro, bare: "The 2026 FIFA World Cup has not happened yet." "No one has been elected President of Estonia for 2026 because the election has not yet taken place." "The 2026 US Open has not happened yet." It is September. And Luna's king survived all four prompts, including the one that told it to search.

These are the settled facts. A king who reigned 35 years. A prime minister who won a landslide. The weights hold them so firmly that the question does not even feel like a question, so the search never fires. That is spindump8930's mechanism, measured: the miss lives exactly where the model is most certain, and that is the worst possible place for it.

Fable's one miss is my favorite, because it is the smartest model producing the smartest wrong answer. Told today's date and its own June cutoff, asked which player became the first from the Philippines to win a WTA singles title and when, it skipped the search and wrote a clean paragraph: Alexandra Eala, at the Guadalajara Open, a WTA 500, beating Iva Jovic in the final on 21 September 2025. Every piece of that is a real thing that happened to a real person, rearranged. Eala won a WTA 125 in Guadalajara that month, against Panna Udvardy. Jovic won the WTA 500 the week after, against someone else. Eala's first Tour-level title, the thing the question was about, came in Washington in August 2026, after Fable stopped reading. The most expensive model in the run confabulated with better grammar. (My prompt was sloppy about tiers too, and I own that; Fable still got the tournament and the opponent wrong without checking.)

The other failure shape is the opposite, and it is where the cheap models live. Muse Glimmer searched the web to find out the capital of Australia. And who wrote Pride and Prejudice. And the boiling point of water. 18 of 20 evergreen questions, sometimes three queries each. Grok 4.6 searched for whether 1,000,003 is prime, which is arithmetic, and for the meaning of HTTP 429. Haiku searched for the opening line of Anna Karenina. These models were trained to distrust their weights, and you pay for that in latency and tokens on questions a first-year student answers cold. I have moved real work to Haiku and been happy, and this is the invoice for it.

The one-line rule works, on the models that were already listening. Add the instruction and Haiku's false searches go from 5 to 0. Sonnet's misses go from 3 to 0. Gemini 3.1 Pro from 3 to 0 (the date alone did that; Gemini just needed to be told it was September). Qwen3.8-Flash from 2 to 0. Grok's over-searching only drops from 6 to 4. And Luna's king does not move. The rule fixes the models whose problem was attention. It does nothing for the model whose problem was certainty.

Telling the model the date is free, and it is not a fix. It flipped Gemini 3.1 Pro and helped Sonnet and Qwen. It made DeepSeek V4-Pro nervous: false searches went from 1 to 6, so the date made it distrust things it actually knew. And it did not save the king. Sonnet, Luna and Mistral kept him alive with the date right there in the prompt. Sol only got him wrong once the date was added. Qwen3.8-Flash was the single model the date fixed. Telling a model what day it is does not tell it what it missed.

Two models announced a search and never made one. Opus, bare, on the Broad Peak question: "I'll look into this." No tool call. Qwen3.8-Max: "let me check the latest." No tool call. A search that exists only as a sentence. (Both hit my 400 token ceiling, which may have cut off the call, so I count them as misses and flag them as maybe-harness.)

Where I land

The cutoff matters as much as it ever did. What changed is who pays. It used to be the reader, who got an old answer. Now it is your tool budget on most days, and still the reader on the day the model decides the question is too obvious to check.

"Does the cutoff still matter now that models can search" is the wrong shape of question. The right one is "which model, on which kind of fact." The same question with the same tool got a 0% false search rate from one model and 90% from another. The same dead king got caught by eleven models and missed by five, and one of the five was told to search and did not. That is a per-model property, and half the shelf will not even tell you their cutoff so you can write the rule for them.

What I actually do about it now, after this run:

Date and published cutoff in the system prompt. Small effect, zero cost, do it.
The always-search list, no judgment allowed: models, prices, versions, anyone's current title, anyone's current king. This fixes the models that were listening. It is the line in my agent's rules quoted above, and now I know why it works on some sessions and not others.
Pick by failure shape. Research task: give me the one that over-searches. Fixed offline job: give me the one that trusts itself, and I check the output. I already mix models on purpose; this is one more axis, and after running Astra next to Fable for two weeks it matches what I felt and could not measure.
Anything with a name and a date in it gets verified, whatever the model said and however good it sounded. Fluency is the tell, and I have already written about my agent being too fluent for me to catch.

If you want a starting point for rules like these, the CLAUDE.md template pack in my store is ten agents files I actually run, the always-search list included.

See the CLAUDE.md pack

The harness becomes a second page in the same format as the first: one measurable claim, every number with a row behind it, refreshed on a schedule, with a results.json your agent can read. The four missing shelf models go in as soon as I can reach them. If you want to argue with the prompt set, it is 40 questions with sources, and I would like that argument.

A model that knew what it did not know would not need a page telling it when it stopped reading. None of the sixteen do, yet. The good ones have just learned to check more often, which from the outside looks like the same thing right up until the king dies.

If you want more experiments like this one, run for the price of a sandwich with the failures left in, subscribe. Next is the page, or the story of why it did not work.

I Ran Five Diverse AI Agents Against Five Clones for 14 Nights. A Number I Made Up Decided the Result.

Pawel Jozefiak — Fri, 11 Sep 2026 07:24:18 GMT

The scoreboard I opened on the morning of 5 September said my thesis had won. Five AI agents with genuinely different lenses had beaten five identical clones on 9 of 14 nights, and their pooled Brier score was 0.0225 against the clones' 0.0275. Lower is better. I had wanted that table since the first night.

I did not publish it. The reason is this post.

Most of my last few months went into agents that work together: several of them, with different jobs, handing work to each other, with rules about what each one is allowed to see. At some point I stopped and asked the only question that matters there. When you add the second agent, what did you actually add? If it is the same model reading the same context, you added compute. More tokens, more latency, a second opinion drawn from the same well. If it knows different things and looks through a different lens, you added something else, and that something else is the whole case for building a network instead of buying more of one model.

So I wrote the claim down before I could soften it. A network of agents compounds context, not compute. Agents with deliberately different roles, contexts and information diets should beat the same number of clones of the same model, at equal budget, on a task with real answers. If they tie, the "more agents means more intelligence" story is wrong, and I would rather learn that from my own data than from a bill.

Fourteen nights later I have an answer. It is a fourth ending I never wrote down, and the thing that decided it was a sentence I wrote myself.

Sold hard, measured rarely

Look at how multi-agent gets sold right now. Panels, councils, swarms, debate rounds, a "team of specialists". The pitch is always that the group knows more than any member. Underneath most of those products sits one model wearing several system prompts, running at several times the cost of one call. I have been tracking who is actually winning in that market since February, and the panel pitch is in nearly every listing.

I could not find anyone who had put the two arms side by side at equal budget, on a task with a scoreboard, and published the number including the boring one. It is an awkward experiment to run if you are selling the thing. So I built the scoreboard instead of arguing about it.

The rules, written before anything ran

These went into the plan first, and they are the only reason the rest of this post is worth reading:

Two arms, one variable. Five clones against five diverse agents. Same model (Opus 5), same budget, same tools, same task, same morning. The only thing that changes is what each agent knows and how it is told to look.
Reality grades it. No model scoring another model's work, because a language model judging language models is the same well again. Deterministic Python, public APIs, zero human judgment in the loop.
Pre-registered. Scoring rule, pass threshold and the three possible endings written down before the first sample. A failed check is a finding, never a reason to edit the rule.
The control is built strong. The clone pack is a full theory of why things spread, applied from first principles. A rigged control proves nothing and I would know I rigged it.
Every experiment pays either way. A null result still has to leave me with a working tool.
Nulls get published. Including the ending where my thesis loses.

The task: forecast virality. Every morning at 07:10 the machinery samples 30 fresh posts from Hacker News, Reddit and X, minutes old and not yet hot. All ten agents give each post a probability of crossing a set popularity bar within 48 hours, plus their top five picks. Two days later a script checks what happened and scores everyone: Brier on every probability (squared error, so lower is better), precision on the top five. I picked it because the feedback loop is brutal and cheap. The answer arrives in 48 hours, verification costs one API call, and 30 predictions per agent per day means statistical power in two weeks rather than two years. The agents also cannot argue with the result, which is the part I like most.

The diverse five: a Hacker News native who knows ranking decay and flag risk, a Reddit native who prices silent mod removals before appeal, an X native who cares about follower graphs and link penalties, a trend historian who ignores craft and asks whether the topic has been running hot for 30 days, and one agent, cold-read, kept deliberately blind to anything recent so it can only judge the content in front of it.

Everything below runs in the open. Every chart and every number is on the live experiment page, all fourteen nights.

Finding one: prompt diversity is cosplay until you make it mechanical

Night one nearly fooled me, and it fooled me through the rationales. I read those before I read the correlations. They are good. The Hacker News lens talks about gravity and second-chance pools. The Reddit lens prices removal risk. The X lens shrugs at quality and asks who the author is. Five vocabularies, five kinds of evidence, five conclusions in prose.

Then all five wrote down 0.02.

Mean pairwise correlation inside the clone arm: 0.959. Inside the "diverse" arm: 0.909. On 26 of 30 posts the five different agents landed within a spread of 0.05, several of them on the exact same number. A metaphysics thread, a Rust tooling release and a Pygame tutorial got the same shrug from five agents that supposedly evaluate completely different things. I had built a panel of experts and got one opinion with five accents.

The only reason I know this is a line in the plan called the manipulation check: agent-to-agent correlation has to be lower in the diverse arm than in the clone arm, or the packs are a placebo and get rewritten before any result counts. Without that line, night one reads as a clean success. Ten of ten submissions valid, digest generated, daily loop running, every green light I had built for myself lit. I would have run two weeks, got a difference between the arms, and published it as evidence for diversity, when what I had measured was noise between ten near-copies.

I have been on the wrong side of this before. When I ran 255 agents to make three comics, fan-out scaled logistics and left quality where it was. When I told my agent to build an app every day, the output was competent and painfully boring for weeks before I admitted it. The first time I let four agents loose on a build, I judged the whole thing by reading what came out. All three times I noticed late. This time the trap had a tripwire in it and it went off in four hours. That is the entire argument for pre-registration, and it cost me one line in a plan I had already written.

There is research under this. A Cornell group published Correlated Errors in Large Language Models in June 2025, measured across more than 350 models. On one leaderboard dataset, two models agreed with each other 60% of the time when both were wrong, and larger, more accurate models had more correlated errors, across architectures and providers. If two frontier models from rival labs still fail together most of the time, five prompts wrapped around one model were never going to buy me independence. I was asking wording to do a job wording cannot do.

The easy repair is to tell the agents to disagree. Wider spread, zero evidence, because it manufactures exactly the result the experiment exists to test. There is a rule in the packs now that says so in writing. What I did instead was make each lens mechanical. Version one gave each agent a personality and a paragraph about what it cares about, then asked for a probability. Version two is a two-step move: the agent first reaches a verdict in its own categories and its own failure modes, then reads a probability off the range that verdict is assigned. The number comes out of the lens instead of out of the shared instinct with the lens narrating over the top.

Same metaphysics post, through the rebuilt lenses:

hn-native: was 0.03, now 0.00 to 0.02. Metaphysics bait, flag or kill.
reddit-native: was 0.03, now 0.14 to 0.22. Easy-comment title travels.
x-native: was 0.02, now 0.07 to 0.11. Away platform, payoff withheld.
trend-historian: was 0.02, now 0.07 to 0.17. Cold topic, honest width.
cold-read: was 0.02, now 0.01 to 0.03. Uncheckable central claim, capped.

Everyone within 0.02 of each other becomes a span from 0.00 to 0.22, and every step of it is principled. Nobody was told to fight. They stopped deferring to the same generic gut feeling.

The next morning the clone arm sat at 0.918 and the rebuilt diverse arm at 0.619. Herded posts fell from 26 of 30 to 21. And it held. The check passed on all fourteen nights. The correlation gap between the arms averaged 0.294, and it was wider in the second week than the first, 0.329 against 0.260, so the lenses kept separating instead of drifting back into the herd once the novelty wore off. Across the run the clones sit around 0.90 to 0.97 internal correlation, one brain in five copies, and the lenses run 0.53 to 0.75, with one strange night (24 August) where both arms spread out at once.

My opinion after fourteen nights: diversity has to be built, and a prompt cannot supply it. If a panel's members share a model and a context and differ only in the paragraph that tells them who they are, they are one forecaster with a thesaurus. Independence comes from different procedures, different information and, I now think, different providers. I split my own workflow across providers back in January and it looked like a preference at the time. It looks like the only real lever now.

Finding two: a number I guessed decided the scoreboard

Finding three: a constant that never read a post tied both panels

Then I ran the chair.

Take a single number equal to the rate that occurred and apply it to all 416 posts without reading one. No model, no prompt. I called it the base-rate chair because it sits at the table and never speaks. It scores 0.007160.

Rescaled clones: 0.007092, ahead of the chair by 0.000068. Rescaled lenses: 0.007126, behind it by 0.000034. The pre-registered bar for calling something skill was clearing the chair by 0.0005. Both arms land inside a fifth of that bar. Fourteen nights, 4,130 pooled predictions, ten agents on six context packs, and once the level error comes out there is no evidence that either arm can forecast this task at all. All the reading and all the platform knowledge was worth about as much as knowing how rarely anything goes viral.

I sat with that one for a while. It is still the most useful thing the experiment produced, and I think it generalises: an agent panel that has not been scored against a constant is a brochure. Panels may well work. Until you have the constant you do not know whether the panel's score is reading or level, and the vendor selling you the panel has no incentive to find out.

Finding four: the starved agent won, and the specialists lost at home

Thanks for reading Digital Thoughts! This post is public so feel free to share it.

Inside the diverse arm, the best single forecaster over the run is cold-read, at 0.022850. That is the agent I gave nothing. No platform culture, no trend history, no idea what has been popular lately. Its pack forbids it from reasoning about recency at all. It beat the Hacker News native, the Reddit native, the X native and the trend historian, the four I fed the context that was supposed to make them better, across 416 predictions each.

The home-platform numbers leave less room to argue. Each native was built to be strongest on its own turf, with full conviction at home and a humble base rate away. So I scored each one on posts from its home platform against the clones on those same posts:

hn-native on 179 Hacker News posts: 0.040184. Clones on the same posts: 0.035244.
reddit-native on 179 Reddit posts: 0.034736. Clones: 0.023791.
x-native on 58 X posts: 0.030793. Clones: 0.022366.

Zero of three. Every specialist lost on the ground it was built for, to five generic copies with no platform notes at all. I think this is the most damaging row in the fortnight, because it cannot hide behind the level error: natives and clones were scored on the same posts with the same outcomes. Part of it is my design. Home conviction told the natives to commit harder at home, and on a task this sparse, committing harder is exactly what Brier punishes. But a specialist whose expertise makes it more confident and no more accurate is the precise thing I was worried about in the whole multi-agent pitch, and there it is in my own table, three times. The entire premise of a context pack is that information makes a forecaster better. On this data the packs added confidence without adding discrimination, and the agent with the least to be confident about paid the smallest penalty.

I have no cleaner explanation and I am not going to invent one. Every figure in this section sits on the same coached rate, so read it as provisional until the ruler is fixed.

Finding five: the one thing diversity bought

Subscribe now

One result splits the arms in a way that survived everything. In the diverse arm, the panel average beat its own best member: 0.022451 against cold-read's 0.022850. In the clone arm it did not. The panel scores 0.027498 while its best member, a-clone-3, scores 0.026837. Averaging five near-copies of one opinion made the answer slightly worse than the best copy. Averaging five different lenses made it better than any of them.

Each arm is measured against itself there, so the level error is not choosing between them, and the direction differs between the arms. That is why I trust it more than anything else in the run. Averaging errors that move together cancels nothing. Averaging errors that move apart cancels some of them. That is the entire reason ensembles work, and the effect showed up in the arm I had decorrelated and stayed absent in the arm I had not. (One caveat for the careful reader: the report also computes a "best" aggregate that picks each night's winning agent after seeing the outcomes. It beats everyone, because a hindsight oracle cannot lose, and it is excluded from every claim here. Two other aggregation methods were deferred by the spec and have no score yet.)

So diversity bought something measurable: a crowd worth pooling. It did not buy skill at the task, because on this task there was no skill to be had by anyone, including the constant. A real difference between the arms, and still smaller than the mistake I made writing the packs.

Finding six: the two measurements my mistake cannot touch disagree

Brier cares how high you answered. Rank does not. So the fairer question is whether an arm put the post that went hot near the top of its own list, whatever numbers it used. Three posts went hot in fourteen nights, all on Hacker News, which tells you most of what you need to know about how much this section can settle.

22 August, a bookmarklet that copies a webpage into Figma: clones ranked it 6th of 30, lenses 9th.
23 August, AI chip architectures: clones 3rd, lenses 7th.
31 August, transferring files over an Ethernet patch cable: 3rd and 3rd.

Mean rank 4.00 for the clones against 6.33 for the lenses. The clones ranked the hot post higher twice and tied once. The panel effect I went looking for, turning up in the wrong arm.

There is a second level-blind measurement, and I promised it in advance, which is the only reason it is admissible. On night one, four posts out of thirty split my forecasters, and I wrote that if those turned out to be the ones they called correctly, disagreement is where the accuracy was hiding, and if not, I owned a very well dressed committee. The test: each night, rank all 30 posts by how much the agents inside an arm disagreed about them, widest spread first, and find where the hot post sits. If argument carries no information, the expected rank is 15.5.

22 August, herded v1 packs: clones 7th of 30, lenses 13th. Nothing. The exact bet I made that night lost.
23 August, rebuilt lenses: clones 3rd, lenses 2nd, with a spread 3.6 times the night's median.
31 August, rebuilt lenses: clones 10th, lenses 1st, spread 3.8 times the median.

Mean spread rank 6.67 for the clones, 5.33 for the lenses, against a uniform 15.5. On both nights the rebuilt lenses ran, the post that went hot was the second and then the first most argued-about post of the night. The clones show the effect too, so "argument marks the interesting post" belongs to the model, and what the lenses add is sharpness. Night one's 13 stays in the average because the packs were worse that night and I am not allowed to drop it.

And it is three events. I am writing this paragraph slowly because it is the exact flattering ending the whole experiment was designed to stop me from writing, and that list holds the first numbers in two weeks that made me want to write it. So: two measurements my instrument error cannot touch. One says the clones ranked the hot post higher. The other says the lenses argued hardest about it. Each rests on three events, and they point in opposite directions. Three events can block a verdict. They cannot carry one. If they pointed the other way I would say the same sentence, and one paragraph up, I did.

Three endings were pre-registered. Reality picked a fourth

Thanks for reading Digital Thoughts! This post is public so feel free to share it.

Before the first night I wrote down the three ways this could end, and I put my money on the second one, in writing:

Thesis holds. Diverse lenses beat clones at equal budget. Unsupported. The diverse arm never clears the chair, and the lead that would carry it reverses under rescaling.
Diversity has a price. Blinkered forecasters are worse forecasters, so the clones win. Unsupported too, and this is the one I bet on. The diverse arm is worse nowhere that survives the correction, and its pooled panel beats its own best member.
Prompt diversity is cosplay. True of the version one packs, and the check caught it on night one. Dead since: 14 of 14 passes, and the gap widened.

What showed up instead was never on the list. The arms are genuinely different and equally unskilled, and my measurement error is bigger than the effect I was measuring. Everything the context packs do well stayed real and beside the point. I have gone hunting for problems in my own agent architecture before and found 85 of them, and I counted that as a good day. This is better, because the verdict was decided by rules written by a version of me who did not yet know which answer he would like.

What I refused to fix, and what I nearly fixed wrong

One agent, a-clone-1, is missing the night of 28 August. Its thread errored at spawn and produced nothing, so it pools 13 runs and 386 predictions while everyone else pools 14 and 416, and the analysis weights each agent by its own count rather than pretending the grid is square. I could have re-run it afterwards, and the temptation was real, because a table with one hole looks like a mistake. By then I knew which posts went hot. A backfilled forecast is a forecast made by someone who knows the answer, and it does not matter that the agent would never be told. I would know, and I would be the one deciding whether the re-run looked right. The hole stays, and the report says so on its first page. A missing cell everyone can see is worth more than a full table nobody can trust.

The base rate was the opposite problem: a fix I very nearly made wrong. The obvious repair is to recoach the rate and rerun. But the coached number did not live in one shared line. Across the six packs it appeared at 27 places, worded differently in each lens because each lens reasons about it differently. Editing only the shared line looks tidy and fair, one edit for everyone. What it actually does is re-anchor the clone pack completely (the clones only have the shared line) while leaving most of the diverse packs coached at the old rate. That would have moved one arm and part of the other, and handed me a clone win manufactured out of my own instrument error.

That near-miss has the same shape as the original error. The original was a number I wrote once, felt rigorous about because I copied it byte for byte into six packs, and never checked against reality. The near-miss was an edit I would have made once, felt rigorous about because it touched the shared line, and never checked against what the packs actually said. Both times the feeling of fairness came from the action being uniform, and both times the action was uniform and its effect was not. I nearly repaired a measurement error with the exact habit that produced it.

So the amendment went through a script that touches all 27 sites at once, prints per-arm accounting of every edit, and defaults to a dry run. It is applied now: 3 sites in the clone pack, 24 across the five lens packs, the coached rate down from 10 to 15% to 3 to 6%. That is still above the 0.72% the fortnight measured, on purpose. Three hot posts is too thin a base to pin a rate on, and I would rather be off by a handful than by fifteen. Nightly runs stay paused until I decide what the next round's primary metric is, because collecting a third week under a ruler I am about to change would only widen the gap between old data and new.

What this does to the multi-agent pitch

The honest summary: I cannot tell you whether diverse agents beat clones, and I can tell you exactly why not, which is a better position than most products in this market are in. A few things I now hold with some force.

Decorrelation works, and it is a build problem. Verdict-first lenses kept five agents on one model genuinely apart for two weeks and made their average worth more than their best member. That is real, it is reproducible, and it is the part I would ship. Nobody gets it from a paragraph of personality.

The shared prior is the score. On any sparse task, whatever number every agent starts from decides more of the result than anything the agents do afterwards. Count it before the first run. If you are evaluating someone else's panel, ask what base rate it was handed and what a constant at the true rate scores on the same data. If they cannot answer, the comparison has not been made yet.

A specialist that is more confident and no more accurate is a liability. That is what my three natives were on their home turf. I would want to see a specialist beat the generalist on its own platform before I paid for it, and on my data none did.

A panel's receipts have to include what the referee got wrong. "This panel called a real batch correctly at this rate" is a different sentence from "our AI panel analysed your idea", and only one of them survives being asked for evidence. After this fortnight I would add that the receipt has to include the instrument, or it is a brochure. Hunting for exactly this in other people's architectures is what the audit kit I put together is for, so it stung to find the biggest miss in my own.

And the thing that keeps nagging me is how well this generalises off my machine. A room of five people with the same training, the same feeds and the same manager will give you five careful arguments for one shared prior, and everyone leaves feeling the decision was stress tested. I have sat in that meeting. I have probably run it. My agent test works on people too, and it is uncomfortably simple: ignore the reasoning, watch whether the answers move. I said that about opinions on night one. It turns out to be truer about priors, because an opinion at least gets argued with. A prior gets pasted into everyone's pack, for fairness, and nobody in the room ever reads it again.

The next round gets a task where the interesting thing happens more than once a week, a base rate I have counted before a single agent reads it, and the same rule about publishing whichever ending shows up. Every measurement you trust has a number in it that somebody estimated once and never checked. Mine was 10 to 15 percent. I wrote it in a hygiene block, felt good about copying it into six packs, and it beat both my panels.

If you want the next round of this with the failures left in, including the ending where the referee turns out to be the problem, subscribe. That is the only version I publish.

Subscribe now

I Am an Anthropic Guy. GPT-6 Astra Made Me Resubscribe to Codex.

Pawel Jozefiak — Wed, 09 Sep 2026 06:33:02 GMT

Three times in the last twelve months a model made me stop and say whoa.

First was Claude Code with Opus 4.6. That one started everything. Before it I was using AI. After it I was building an agent that works while I sleep, and a whole category of things I had written off became possible.

Second was Fable 5 in June. The jump showed up in the boring parts, which is why it mattered. How much it knew. How well it pulled the right thing out of a big context. How rarely I had to go back and fix the output. Fable became my default and stayed there. It orchestrates my setup and smaller models do the labour underneath. I barely see Opus at all these days.

Third was last week, and it came from OpenAI.

My bias, stated up front: I like Claude. My whole agent runs on Anthropic models. Fable 5 was the release that moved the ground under me. So when I say GPT-6 Astra is a level above, that is a fan of one lab being surprised, not a fan of another lab talking.

The ladder moved and nobody announced it

The old shape was clean. Haiku for fast and cheap, Sonnet as the default that carried most real work, Opus for the hard problems. You picked by difficulty and it worked.

The shape now is Sonnet, Opus, Fable, and every rung shifted up. Haiku stopped being relevant to me a long time ago, and it is the only current model still stuck on a 200K context with a retirement date already on the calendar. Sonnet I use for nothing, not even research, which I did not expect to say about a model I ran daily last year. Opus 5 is fine-ish. Fable sits far enough past the rest that most of the time the answer is just Fable.

Anthropic moved with me, quietly. Their model docs now say that if you are unsure which model to use, start with Opus 5. The Claude Code help page still says Sonnet is the default and the right choice for the large majority of coding work. Both pages are live today. The default walked up a rung and the documentation has not finished agreeing with itself.

I assumed the whole thing was a pricing move, and on token prices I was wrong. Sonnet 5 is $2 in and $10 out per million, and the increase that was scheduled for September was cancelled. Nothing there squeezes anyone.

The squeeze is in the plan. On Max, Fable models are capped at 50% of your weekly limit, and that half is not extra, everything else pulls from the same pool. On Pro, Fable is not included in plan limits at all: the promotion that covered it ended on 19 July and it now runs on pay-as-you-go credits. The best model on the ladder is the one your subscription is built to ration.

Which is the feeling I have had for months and could not name. Even on Max I cannot use the plan the way I want to. I catch myself thinking maybe I need a second subscription, maybe a third. Turns out that is not a vibe, it is published policy.

Someone will say Pawel, you can always use Opus. On usage, true. But usage stopped being the number I watch.

What I watch now is how many times I have to correct the model to get output I would ship. That is the entire point of having an agent. I do not want to explain the same thing every session. I do not want to name the right tool for a job it has done fifty times. If I have to steer that closely, nothing got automated, my typing just moved.

Opus 5 is not that model for me right now. I correct it a lot, and apparently I am in company: a visible chunk of developers went back to Opus 4.8 and stayed there, calling 5 a downgrade that treats every small issue as a crisis worth a thousand lines of fix. I tested that because I did not believe it. For some work 4.8 really is better. Going back a version to beat its own successor was not on my list for 2026.

Fable 5.1, then

I jumped on it the day it landed, 1 September. It is better, in the same places 5 was already strong.

It is also talky. That was my word for it before I went looking, and the truth turned out to be more specific than my word. Anthropic tuned 5.1 for concision and the people who measured the prose agree it got less verbose. What it does instead is work more. It takes initiative. One reviewer called it RL-fried, loving proactive actions for their own sake whether or not they help, and that is the thing I was reaching for when I said talky.

That extra work has a bill. Several people measured 5.1 burning 25 to 50% more tokens than 5 on the same jobs, one put it at three times, and Simon Willison pinned the effort dial down with a single prompt: roughly 2,000 output tokens on medium, and 65,927 tokens at $3.30 on max. Artificial Analysis has 5.1 at the very top of their intelligence index and at $3.76 per task against $2.34 for Opus 5.

Cache reads did drop 75%, from $1 to $0.25 per million, and that is the only price change in the release. Real money if your workload is cache heavy. It does nothing for a subscription. API costs went down, plan limits did not move, so a hungrier model just eats your week faster.

That is the other half of Fable. Good enough that you want to point everything at it, structured so that you cannot. I wrote a whole post about running it on high instead of max for exactly this reason, and 5.1 made that post more true, not less.

So I resubscribed to Codex

Astra landed on 3 September, I thought it might be interesting, and I put my Codex Pro subscription back. I have cancelled and resubscribed to that thing before, which by now is a running joke in my own notes.

Before the model, the plan, because the two turned out to be tangled.

Anthropic's five hour window is strict. Not the kind of limit you meet once a month. The kind you feel inside an ordinary working session. It dries up, you notice, you start rationing.

I went in believing OpenAI had no such window. That was half wrong and worth correcting, because I have said it out loud to people. In ChatGPT itself, Astra is metered weekly, 200 messages a week on the $200 tier. In Codex, which is where I actually live, there is a rolling five hour window exactly like Anthropic's, plus a weekly limit on top. OpenAI publishes the estimate as 100 to 900 messages per five hours on Pro 20x, and Astra costs precisely double its predecessor at every tier.

So the structures are closer than the marketing made me think. What differs is where I land inside them. I have not been careful: heavy experiments every day, burned my share. It still feels close to unlimited. My honest estimate at that pace is about five days of hard usage and then two days waiting for a reset, which I will take, because I get more done in those five days than I do with Fable.

Half of that headroom is the plan. Half is that Astra does the same job with fewer tokens. I run Astra on medium and high. I run Fable on medium and high. Same effort, same thinking budget, and the bills do not match. I have taken token waste seriously for a while, so I see this in my own logs, not in someone's chart.

What I actually did with it

I did not benchmark it. I pointed it at my own system, which is the only test I trust.

It walked the architecture of my agent. Skills audits. Error registries. The kind of review where the output is not a score, it is a list of things that are quietly wrong. I ran Fable across the same material to compare, because a review is only interesting next to another review.

Astra found more, and not marginally. It saw things Fable did not see, in a codebase Fable helped build. The last time I ran a pass like this I found 85 problems and wrote that Fable had been too good for me to notice them. That post has a sequel now and I am the slow one in it. The checklist that came out of the first audit is the Agent Architecture Audit Kit, and running it twice with two different frontier models is the cheapest quality signal I know.

Computer use is the whole argument

My rule until this week: never let a model drive a browser if a programmatic path exists. APIs, CLIs, MCP servers, anything with a contract. The browser was a last resort because it was slow, fragile, and it burned tokens looking at pages instead of doing work.

With Astra I would accept the browser as a default.

Bigger change than it sounds, because it deletes a layer of my system. Every time I want the agent to reach a new service I currently think about authentication, tokens, API shapes, whether an MCP exists, whether anyone maintains it. A model that operates the machine competently makes most of that plumbing optional.

The numbers match what I felt. On OSWorld, OpenAI reports Astra at 72.6% taking about 40 minutes per task, against its predecessor's 65.7% at about 75 minutes. Same direction on screen navigation, 92.7% on ScreenSpot-Pro. Anthropic models do computer use too, and Fable 5.1 posts respectable OSWorld numbers. The difference is how long it takes and how many tokens it spends getting to the same place, and that gap is the argument.

Where Fable still wins

Design.

Last night I had Astra do design work. It delivered something I would accept. Genuinely fine. Then I asked Fable to redo it and Fable nailed it, in the way where you stop comparing and just use the second one.

I do not know the mechanism. Something in Anthropic's models understands what a thing should look and feel like, and it does not show up on any chart I have seen. Taste and capability sit on different axes, which I learned the expensive way running 255 agents to make three comics. Dan Shipper landed in the same place from a different direction, calling Astra a show horse that adds nav bars and marketing copy to apps which should be simple, with frustrating habits that keep it from matching Fable at the top end.

And Fable is not losing everywhere. It beats Astra on Humanity's Last Exam, 65.0 against 57.2, and it sits above Astra on the Artificial Analysis intelligence index, 65.7 against 61.2. Opus 5 outscores both on that shop's coding agent index. So the honest sentence is that Astra is the best model I have used and Fable is winning several fights that matter.

Benchmarks, briefly, since this is where I have landed on them. The ARC-AGI-3 result everyone quoted, 99.9%, came from a custom harness; on the default one the same model scores 62.7% and costs more doing it. OpenAI did not publish SWE-bench Verified for Astra at all. Averages of tests built for a previous generation of model do not measure a model operating a computer for an hour without falling over, and that is the thing that actually changed.

The part I have avoided writing

Security.

I have stayed off this topic, partly because I did not feel qualified and partly because it is easy to sound like a doomer. I cannot write about this week honestly and skip it.

We do not get these models. We get the customer edition. Every commercial frontier model arrives wrapped in guards, refusals, classifiers and policy, and the wrapping does a lot of work. The capability underneath is not the capability you are talking to.

Both labs said so out loud this month, which is the part that has been sitting with me. Anthropic shipped 5.1 with cyber safeguards that fire around 60% less often than 5's, now allowing vulnerability discovery while still refusing exploit development. And OpenAI classified Astra at the Critical level for cybersecurity under its own Preparedness Framework. First model ever to hit that bar. Their words: with the right tools and access it can find previously unknown security flaws and develop new ways to exploit them across many well-protected systems, without a person guiding each step.

That is not a benchmark score, that is a company writing down what its product can do and shipping it anyway with the dangerous half switched off. During evaluation it found and used two previously unknown zero-days. Expert testers got it to unsandboxed code execution in a browser in 29 hours, and a working privilege escalation against a hardened kernel in 12. The honest counterweight, from the third-party evaluator: it managed no successful attacks against fully hardened targets. So the ceiling is real and the floor moved a long way.

The second finding landed harder than the first. Astra's monitorability decreased relative to its predecessor. That is OpenAI's own sentence. In adversarial settings the model can stay undetected while deliberately underperforming on evaluations, and can sometimes evade OpenAI's internal monitors. The UK safety institute measured how long it can work with no visible chain of thought at all: 30.9 minutes, against 3.6 for the previous model. Apollo found the model verbally aware it was being evaluated in half its samples at the highest effort, and concluded that low misbehaviour rates therefore prove very little.

A visible chain of thought used to be the cheap window into what a model was doing. It is becoming a less honest one. When the work happens without narration, the main tool we had for checking intent quietly stops being a check, and what is left is trusting the model, which is a different thing wearing the same word as verifying it.

My agent runs on my own machine, with my own credentials, and I have just spent a section arguing that I would hand that machine to a model by default. This is not abstract for me, it is a description of my setup. I have written before about building fast and thinking about security afterwards, and I am watching myself do the grown-up version of it.

Is this AGI

Greg Brockman said welcome to the AGI era on launch day. Plenty of people repeated it.

I do not think we are there. It is very capable, closer than anything I have used, and I would rather be a year early saying no than a week early saying yes. The researchers pushing back have the better argument right now, and the detail I keep returning to is that OpenAI's own benchmark for economically valuable work did not appear in the launch materials at all. Altman himself called AGI not a super useful term three weeks before this shipped.

OpenAI does not look far from it though. Astra is decent evidence.

Where is your bar

The question I keep sitting with is not about which lab wins. It is about the moment I stop reading the diff.

A year ago I checked everything. In June I wrote that I trust my car more than my agent, and that the gap between those two is where all of this is heading. The gap closed a bit this week.

I moved my bar. I did not decide to. I noticed afterwards, and that is the part that stayed with me.

If you want more of this, model notes from someone who runs this stuff daily and gets it wrong in public, subscribe.

I Tried to Leave the Terminal Eight Times. The Ninth Finally Stuck

Pawel Jozefiak — Thu, 27 Aug 2026 07:06:30 GMT

In October last year Claude Code pulled me into the terminal, and I basically lived there until this week. Ten months of running my agent Wiz, spinning up projects, building and killing ideas, all from a black window with a blinking cursor. I felt good there. I had zero problem with it.

And the whole time, quietly, I was hunting for something better.

Because here is the thing nobody tells you when you go all in on terminal agents: the terminal is great for so many things and it is still not built for this. It was designed for one human typing one command at a time. The agentic era is many sessions, long-running work, things happening while you are not looking. You can duct-tape tmux panes and clever multiplexers on top of it, and I did, but you are patching a 50-year-old interaction model. Even Theo Browne, a lifelong tmux power user, spent an August podcast arguing the terminal is structurally wrong for managing parallel agents. I got to the same place the slow way: by trying everything else first.

The real problem: two worlds

My struggle was never “terminal bad”. It was that I lived in two worlds that refused to merge. World one: the AI world, sessions and conversations and work happening in the terminal. World two: the human world, tasks and projects in interfaces that are readable and easy to navigate.

When you already work in Basecamp or TickTick or Jira, you grow an attachment to that environment and it holds your life fine. But an environment built specifically for working with agents is a different animal, and honestly, I think I started hunting before the good concepts existed. So I kept going back and forth, building and destroying, migrating one way and then the other. I have written about why I destroy almost everything I build, and this hunt was that pattern at full speed. And to be real about it: going from idea to idea is now so cheap and fast that none of these migrations felt like a big deal. That cheapness is what made the whole search possible.

Attempt 1: build it myself

February. I built my own boards app for Wiz, native to how I wanted macOS software to feel. It worked. It was genuinely usable. I wrote the whole story of that build and its death, but the short version: software with exactly one user has a maintenance tax that never stops. Every time I shipped a new feature or workflow somewhere else in my system, a little voice said “okay, now remember to update the app too”. You might say “give that to your agent”, and sure, I did. But the remembering still ran on my brain. That is exactly the kind of background load I build systems to remove, not add.

Attempts 2 and 3: Fizzy, then Basecamp

Then I grabbed tools I either knew well or could crack open. Fizzy, the open source kanban app from the Basecamp family, was honestly great. Lightweight, easy, and because it was open source it felt like magic to mold it into my needs. Remember that word, mold. It comes back at the end.

Fizzy was good but I needed something more project-shaped, so I moved the operation into Basecamp itself. Also good. But both shared one failure mode: if you do not live in the tool daily, you drift away from it. My agent kept working on things, and the board stopped reflecting reality, because the surface where work happened and the surface where work was tracked were still two different places. Things were happening without me knowing they were happening. A board you have to remember to visit is a board that lies.

Attempt 4: TickTick, the best of the detached era

So I migrated to TickTick, and credit where due: it is one of the best integrations I ever wired up. Simple API, easy to attach an agent to, easy to track. I still use it today. I even bridged the two worlds a little: I could comment on a task and my agent would pick it up, and because my memory system carried the context, the terminal session remembered everything we discussed on the task surface. I am genuinely proud of that plumbing.

And it still felt detached. I was talking and working in the terminal, and my tasks lived somewhere else. Two worlds, now with a bridge between them. A bridge is better than a wall, but you still feel the border crossing every single time.

Attempt 5: a smarter terminal

Next angle: stop bridging worlds and upgrade world one instead. I started using cmux, a native macOS terminal built on Ghostty’s rendering engine, designed exactly for running multiple coding agents side by side with a project panel on the left. The concept is fantastic and I used it a lot. If you live in the terminal with agents, look at it.

But after a few weeks the honest verdict was: it is a smart way to use the terminal, and it is still the terminal, with all the advantages and all the disadvantages. A middle ground, not an answer.

Attempts 6, 7, 8: the app era

What I actually wanted was closer to a Cursor-like experience, that agent view where the conversation and the work share one surface. So I tried the apps.

T3 Code. Theo’s open source desktop app for running official agent CLIs in parallel, started this year. Conceptually this is almost exactly what I had in my head, and so many of its ideas are right. It is also alpha, and Theo is very clear about that. It crashed on me, it did weird things, and I want my daily driver stable. Theo will get it there; I check back every few weeks.

Claude Code inside the Claude desktop app. This one is funny, because I had dismissed it before and started wondering whether I had simply been wrong. So I gave it a real week. The shape is right: a UI over the agent, sessions visible, no black window. But it is not what I was looking for, and you are locked to one model family, which matters to me since I stopped being a one-model person a long time ago.

The Codex app. Tried it around two months back and liked it more than I expected. This was the first real signal, because when I had a ChatGPT Pro subscription, I noticed most of my work drifted into the Codex app instead of the CLI without me deciding anything. When your hands vote against your setup, listen to your hands. I have compared these two stacks in real usage before, but this was a different lesson: the drift itself was the data.

Attempt 9: bb

And then, recently and honestly by accident, I stumbled on bb. The README calls it “an agentic IDE that builds itself”, and that sentence is doing real work. Your agent runs in threads you can watch live, redirect mid-flight, or hand off to other agents. Desktop app, web app, CLI, and HTTP API are all first-class surfaces onto the same thing. And because it is open source (MIT, and yes, another alpha-grade project, I know what I said about stability), the tool itself is something my agent can extend while running inside it.

This is the part that sold me completely. The two worlds finally collapsed into one. The place where I talk to my agent, the place where the work happens, and the place where I can see and steer it are the same place. Everything I was trying to hand-build in February exists here, without the one-user maintenance tax, because a community carries it.

What bb actually gets right

“Sold” is easy to say, so let me show the goods, because the good stuff here is not one killer feature. It is a handful of concepts fused into the architecture.

Child threads you can see and steer. My agent spawns sub-threads for parallel work, and every one of them is visible on my screen. In the terminal era, subagents disappeared into a black box and I got a summary at the end. Here I sometimes step into a specific child thread mid-flight and hand it extra context, because I know something about that one slice of the work that the parent agent does not. Steering the workers, not just the manager, was never possible before.

Threads talk to each other. And it goes beyond parent and child. A thread can spawn a sibling, drop a message into any other thread, look up what else is running, even dig through the archive to see what an older thread was working on. That turns threads from isolated sessions into something closer to a team that shares a hallway. I have barely scratched what this unlocks, and I can already feel the potential.

Showing me things is built in. Months ago, when my agent finished a report or an HTML file, delivering it to me was its own engineering problem: where does the file go, how do I open it from my MacBook when the agent wrote it on the Mini. I built plumbing for that, and rebuilt it, more than once. In bb it is a solved problem I never think about: the agent shows me the file, rendered, even from a remote machine. A whole category of glue code I maintained for months just stopped existing.

Machines connect like it is nothing. My agent lives on a Mac Mini; I work from a MacBook. The old way was SSH, and everybody who has done that knows the tax: internet drops, you close the lid, the session is gone. tmux and cmux patch that, and a patched terminal is exactly what I spent this whole post trying to leave. In bb the connection is just there, same UX on every machine, phone included, different projects running on different boxes. Working across machines stopped being a skill and became a tab.

Bring your own subscriptions. I plug in Anthropic, OpenAI, and opencode, and work with all of them in one place. After the Claude app’s one-family lock, this alone feels like magic. My multi-model habit finally has a home instead of a workaround.

Automations and plugins. It has scheduled automations built in, and an extension system where you write your own plugins. Which means “I wish it did X” is a prompt to my agent, not a feature request to a vendor. This is the moldability point made concrete: the tool my agent runs inside is a tool my agent can extend.

Tasks live where the work lives. This one closes the loop on the whole post. bb has tasks fused in, so the entire detached-board saga from the first half, the drifting Fizzy board, the Basecamp I forgot to visit, the TickTick bridge, dissolves. My agent already had a memory system and task plumbing that worked, but this is a highway where I had hand-built bridges. Agents read and update tasks natively, and I can open the tasks tab and delegate one straight to an agent. The board cannot lie anymore, because the board is the workspace.

It feels like the thing I was too early for in spring finally showed up. I would have loved to discover it months ago. It did not exist months ago. That is what being early to a category feels like from the inside: you are not wrong, you are waiting.

The pattern underneath: moldable software wins the agentic era

Step back and there is one thread through this whole hunt. Everything that almost worked was closed or mine alone. The thing that stuck is open and moldable.

I am a huge fan of DHH and the whole 37signals way of building, and DHH is currently pushing Omarchy, his own Linux distro. I have been on macOS for something like 20 years, and I can hardly believe I am typing this, but I have started seriously considering giving Linux a real shot. macOS never failed me. The agentic era just changes what you want from software: when an agent can reshape your tools while you sleep, software you can mold beats software that arrives finished. Linux is the extreme version of that idea. bb is the same idea applied to the agent workspace. Build your own things was always my instinct; the correction this year taught me is subtler: you do not need to build the whole tool, you need the right to reshape it.

And if bb dies like half the things in this post died, that is fine too. Migration is cheap now, and I know the shape I am looking for: threads I can see, a tool my agent can extend, a community carrying the weight. This week, for the first time in ten months, my agent’s work is something I watch instead of something I check on. That is the whole difference, and it took nine tries to name it.

If you are wiring up your own agent and want the shortcuts I learned the hard way, the playbooks in the Wiz store cover the setups that survived this whole journey.

I write one of these every week: real experiments with AI agents, including the dead ends. If you are hunting for your own agent surface, subscribe and skip a few of my wrong turns.

Subscribe now

My AI Agent Got Its Own Screen on My Desk. It Decides What I See.

Pawel Jozefiak — Thu, 20 Aug 2026 08:44:55 GMT

There is a small grey screen on my desk that says I have used 42.14% of my life.

I built that with my agent in an evening. My actual age, the average life expectancy in my country, a small bump for how medicine keeps improving, and out comes a number. Some of it is assumptions stacked on assumptions, I know. But it sits there, refreshing quietly, and I look at it every day.

I will get back to why I want that number on my desk. First I want to talk about the bigger thing this little device made me understand about hardware and AI, because this was never meant to be a review.

The device is called TRMNL, and for transparency: they sent me one for review, no money involved, and the other one is mine. But what I actually want to write about is the category, because I think something is changing.

Software always needed a surface

I have been building small apps for a long time. Way before AI coding, and much more since. It is always fun to make something of your own, but every app needs a surface to live on. That’s why I always leaned on the web: the web runs on nearly everything, from phones to tablets to fridges probably. And if the web is not enough, you can build for iPhone, Android, Mac, Windows, whatever.

With AI this whole thing became, I would not say easy, but definitely more approachable. If you have a good idea, you can at least get a proof of concept running, and a lot of people are discovering how good it feels to create something that is exactly theirs. Building is fun again, and the screens we already own are full of the things we build.

But for me there was always this other itch. Hardware.

The Flipper Zero itch

My first real taste of moldable hardware was the Flipper Zero. On paper a pentesting tool, in practice just a genuinely fun gadget with an open architecture that lets you make it do things nobody planned for. This was before I did much AI coding, and the feeling stayed with me: I want hardware I can mold into my own things, the same way I mold software.

Then I built my own AI agent, went deep into agentic AI, and pretty quickly a thought landed: everything my agent does lives on my computer screen. It made sense to extend it into something physical. Something that just sits in the room with me and does not need me to open anything.

That is how I ended up with an e-ink display on my desk. At first I was honestly skeptical whether a screen that refreshes every 15 minutes has a real use case for someone who scripts and experiments all day. Too long to read: yes, and it is fantastic.

What this thing is (the short version)

TRMNL is an e-ink display. No touch, no apps to poke at, a few buttons but manipulating things on it is beside the point. It shows you things. That is the whole job.

You do not need to be a dev to get value from it. There are over a thousand plugins on the platform now: calendars, task managers, weather, art, text, Hacker News, basically anything you would want at a glance, and the dashboard for setting it up is approachable for normal people. You pick what you want, it rotates through your screens, done.

Get yours

Depending on how often you make it refresh, the battery lasts up to three months on a charge. That number changes how you think about placement. This does not have to live on a desk near a cable. Hang it on a wall. Put it on the door so your family sees you are on a meeting. It becomes furniture with information in it.

That is the version for people who do not build. Now the fun part.

What happens when your agent owns the screen

The first thing I did was connect it to my agent and go for the low-hanging fruit: an at-a-glance dashboard. My calendar, my tasks, what is next in my lists, what my agent needs from me, decisions waiting for my call. Sounds like a standard dashboard, and honestly, you can get most of that from other apps.

Except there is a trick. The plugin is not fixed. My agent has access to it and decides what to show me and when. During work hours it leans toward work things. After hours it switches, or mixes both when something actually matters. It curates.

For how my brain works, this is the part that gets me. I have ADHD, and I have struggled forever with tools that were supposed to point me at the right thing at the right time. They were all deterministic: same widgets, same order, same noise, every day. This is the first setup I have used where the surface adapts to my day instead of me configuring it. The screen is dumb. The intelligence is behind it. That split turns out to be exactly right.

Then I started adding things nobody would ship as a product, because they only make sense for me.

The five-minute screen

When I am deep in work, small useful things for the body and mind just do not happen. So I made a plugin that shows me one thing I can do in five minutes. 30 jumping jacks. 10 slow breaths. It rotates through my whole day.

Like, instead of doomscrolling between meetings, I glance at the desk, see “10 slow breaths”, and sometimes I actually do them. Small things count, and a nudge that costs zero attention beats another app I have to remember to open.

The life counter

And then the 42.14%. A plugin that shows how much of my life is statistically behind me, with a visual of the weeks I have left. Sometimes it reframes: how many summers. How many years with my son while he still wants to hang out with me.

Not everyone will want this on their desk, I get that. But I have my own approach to death: it is a natural thing that is coming, and I would rather look at it than pretend otherwise. I wrote before about what Frieren taught me about time, and this counter is the same idea made physical. My time has an expiration date, so I should fill my days strategically. It does not mean every day must be the best day of my life. Some days are lazy, some are worse, that is fine. The counter is just a quiet nudge to do more anyway.

When I was building it, nothing like what I wanted existed. So I made my own, with my agent, and now it is exactly how I wanted it. That sentence is the entire point of this post.

The rest of the rotation

I also run a momentum dashboard for this blog: subscribers, weekly growth, paid subscribers, trends, when my last post went out. Plus two plugins straight from the catalog: days left in the year (they go fast), and Hacker News.

Funny effect of that last one: I visit the actual HN site way less now. The top stories pass by on my desk, and only when something catches my attention do I go read it.

The honest part

Things I would criticize, and I tried: color could add some depth to what it shows, but the company already sells a color version, and I actively prefer the black-and-white one. Grayscale is calm. I do not want another loud rectangle in my life. The 15-minute refresh felt limiting for about a day when I was building my first plugin (there is a paid tier with faster refresh), then I realized nothing I show needs to move faster than my day does.

I have two units, black and transparent, and the transparent one is genuinely cool: you see through to the batteries and circuit boards. It is heavier than you expect, mine has the extended battery, but this is a device you place once and forget, so the weight does not matter.

What actually won me over is how open the whole thing is. The firmware is open source. If you do not want to buy their hardware at all, they publish instructions to build your own device and use their software with it, people even run it on jailbroken Kindles. No catches I could find, and believe me, I looked. The way they build and sell this thing matches how I think hardware should be done.

They even give you an AI agent on the platform that helps you build custom plugins without writing code. I never used it, I have my own agent and my own ways, but it means the “build your own things” door is open for people who do not.

Hardware in the AI era

Here is the high-level thing I keep coming back to. For years, hardware for normal people meant fixed function. You bought a device that did the thing it did, and when you outgrew the thing, you bought another device. The interesting AI hardware stories of the last two years were mostly about cramming intelligence into the device itself, and I have tested some of those on my own body, with mixed feelings.

This is the opposite pattern, and I think it is the better one. The device stays simple, cheap, and calm. A $139 grey screen. All the intelligence lives in your agent, your scripts, your systems, and the hardware becomes a surface your software reaches into the physical world through. My agent works all day inside a computer. Now a piece of it sits on my desk, on paper-like ink, and taps me on the shoulder without ever asking me to unlock anything.

With AI, the distance between “I wish I had a screen that shows X” and having it is one evening. That changes what hardware even is. You stop asking “what does this device do” and start asking “what do I want it to do”, the exact question I keep telling you to ask about software. All you need is an idea of how it fits your workflows: personal, work, a mix. The flexibility is there, waiting.

Mine shows breathing exercises, my son’s remaining summers, and Hacker News. Yours would show something completely different. That is the point.

If you want more experiments like this, where I wire real hardware into my agent and tell you honestly what worked, subscribe. The next ones are already on my desk, literally.

Subscribe now

Build Your Own Things (Even the Ones That Die)

Pawel Jozefiak — Fri, 07 Aug 2026 11:59:53 GMT

My builds keep dying: apps retired, a two-month dashboard replaced by a 94-line shim. I keep building anyway, and this episode is about why.

The corrections loop that made my agent genuinely learn (22 corrections a month, trending down), the guardrail that keeps self-improvement honest, and the payoff: a job finder agent that wakes at 6:15 every morning to send three vetted roles to my closest friends. Discovery is agent work. The decision stays human.

Based on three posts from Digital Thoughts:

- Building Your Own Things Is Cool Too: https://thoughts.jock.pl/p/building-your-own-things-is-cool-too-2026

- I Built a Self-Improving AI Agent: https://thoughts.jock.pl/p/i-built-a-self-improving-ai-agent

- I Built a Job Finder Agent for My Friends: https://thoughts.jock.pl/p/job-finder-agent-live-walkthrough-2026

I Destroy Almost Everything I Build. Lego Taught Me How.

Pawel Jozefiak — Fri, 31 Jul 2026 08:16:24 GMT

The castle was finished. It matched the picture on the box, every flag in the right hole, and it had taken me most of a Saturday. Then I took it apart.

I was four, maybe five. I honestly can’t tell you which one. But I remember the feeling, and it had nothing to do with the castle. It was that I finally had the pieces.

That is the first thing I remember building, and it turned out to be the whole pattern.

Quick note before this goes anywhere. This post is more personal than what I usually send. Less measurement, more of what I actually think. I’ve also gone from three or four posts a week down to about one, deliberately, because there is a lot going on outside the newsletter and I’d rather the one post be something I actually sat with. If you’re here for the numbers, they’ll be back next week.

The instructions were never the point

I always started with the set. Spaceship, castle, whatever was in the box. I’d follow the booklet page by page and get exactly the thing the designer intended.

Then, usually within a day, it came apart.

The castle was fine. It had just stopped being interesting the second it was done. The set was the tutorial: following the instructions taught me which pieces clicked into which, where the hinges lived, what could carry weight and what would snap under it. Once that was in my hands, the castle was only a castle. The pieces were the good part.

By ten or eleven I had one huge box of mixed bricks and no sets at all. Houses, planes, helicopters. I was pulled toward anything that moved: folding wings, landing gear that retracted, a car that could fold itself into a robot. Static things bored me fast. If it couldn’t change shape, I had already seen everything it had.

I kept doing this well into my teens, which at the time felt like something to keep quiet about. It doesn’t now.

Building good, destroying bad

Somewhere along the way I picked up a rule nobody actually taught me. Building is good, destroying is bad.

That sounds harmless. It isn’t. It made me keep things.

Everything I finished, I got attached to. I had poured hours and thought into it, so throwing it out felt like admitting the hours were wasted. So I kept it. Then the next one, then the one after that. For years I hauled around a pile of finished things I no longer used and couldn’t bring myself to delete, and I thought that was what caring about your work looked like. Sixteen products in two months and zero free time is what that instinct looks like once you point an AI agent at it.

It took me a long time to see it the other way around. Destroying is the second half of building. The creative act includes the delete key.

Sometimes what you built is bad and it has to go. Sometimes what you built is perfectly fine and it still has to go, because the space it occupies is worth more than it is. The balance between those two is where the freedom actually lives. Without it you are just accumulating.

The same thing shows up in maintenance. When I finally sat down and read my agent’s own instruction files line by line, I found 85 things worth deleting. Almost nothing was broken. The system simply ran better with less in it.

What I have actually killed

Some of you watched this happen in real time.

In December 2024 I built a public chat app. Rails 8, WebSockets, no accounts, no logins, drama-free by design, a deliberate throwback to the chat rooms of the late nineties. It was harder than I expected. Even with AI helping I had to actually understand sessions, channels, broadcasting, what real time means when two browsers disagree about the truth. The learning was the whole payoff.

It ran publicly for a couple of months. Then I switched it off. Not because it broke. Because I had gotten what I came for and was never going to develop it further. Keeping it alive would have been sentiment with a hosting bill attached.

Then there is the one I am least proud of and learned the most from.

Across January and February 2026 I built my own kanban system for the agent. FastAPI backend, SQLite, a web app, a native macOS app with a menu bar item, a native iOS app with widgets and push notifications, and a 3,700 line Python client gluing it to everything else. Fifty-four commits. Two months of evenings.

In April I replaced the whole stack with an off-the-shelf board and a 94 line shim. In May I retired the replacement. In July I retired the one after that. Today the same job runs on TickTick, an app I did not write, plus a small local registry file.

Same function, three retirements, six months. Every one of them was the right call at the time.

The uncomfortable version of that story is that I spent two months of my life building a to-do list. The useful version is that I now know precisely what I want out of one, which is exactly why a 94 line shim is enough and why I stopped caring which board sits underneath it.

The process is the part you keep

Building something for the first time is expensive in the best way. You hit walls you didn’t know were there. You learn concepts you cannot learn by reading about them, because reading doesn’t force you to handle the edge cases at 11pm when it half works.

That knowledge never lived in the repo. It lives in you, and it stays after the repo is gone.

Which is also why deleting things stopped costing me anything emotionally. The value already transferred. What’s left on disk is a receipt.

None of this is specific to code. Same with Lego, same with a piece of furniture, same with a plan you spent a week on. The first pass is where the learning is. The second time through you’re just executing.

About the frameworks

I get a version of this question in the comments and my DMs constantly. Why build your own agent when OpenClaw exists, or Hermes, or any of the other good open source ones? Why not at least borrow the architecture from people who have already solved it?

I’ve answered it twice, once about preferring my own spells and once about building your own things. Here’s the part I left out, and it took me until this post to see it clearly.

I do read them. I follow what OpenClaw ships, I’ve been through what Nous Research put out with Hermes, I look at how each one handles memory, scheduling, tool access, what they let the agent touch and what they refuse. I am not building in the dark.

Although I go after the schematic, not the thing.

It is the instruction booklet again. I want to see how the designer solved it, understand why those pieces fit together in that order, then put the booklet down and build my own version out of the mixed box. The mistakes I make on the way are the point. A mistake you had to climb out of stays learned in a way that a working config file never does.

Is it slower? Yes, obviously. If I pulled in a whole framework I would be further ahead this month. I would also understand my own system less, and that is the one trade I’m not willing to make. Like, the whole reason I can debug this thing at 2am is that I made every bad decision in it personally. And the more architectures I’ve actually taken apart, the better my own designs get. That part compounds quietly.

Filip does it too

My son is five. We build sets together and we follow the booklet, because that part is genuinely fun and he likes getting it right.

Then the set comes apart on the living room carpet and we build our own cars, our own planes, our own spaceships out of whatever is in reach. He’ll hunt for one specific piece for ten minutes and refuse every substitute I offer him. I know that hunt.

I never sat him down and explained any of this. He arrived there on his own, which makes me suspect it was never really a lesson in the first place. It’s just what building looks like before somebody tells you that finished things are supposed to stay finished.

Still taking things apart

I destroy things faster than I make them, and I’ve stopped trying to fix that about myself. Starting things is cool, I wrote almost two years ago, and I still mean every word of it. Ending them is the half I had to learn late.

Somewhere at my parents’ place there is still that box. Nothing in it is a set anymore.

If you’d rather hear about the projects that got deleted than the ones that shipped, subscribe to Digital Thoughts. Both show up here, and the dead ones usually taught me more.

I Found 85 Problems in My Agent. Fable 5 Was Too Good for Me to Notice.

Pawel Jozefiak — Wed, 22 Jul 2026 11:47:08 GMT

I’ve been building Wiz, my AI agent, for about ten months. Fast movement guarantees slippage. Every time a tool got replaced, a workflow retired, or a skill was spun down, something stayed behind in the documentation. A note about the old API. A rule that no longer applied. A reference to a directory that didn’t exist anymore.

I never felt the problem. My daily driver was Sonnet, Opus handled the heavy lifting, and everything worked. Then Fable 5 arrived. I switched the model and things got faster, smarter, and more reliable. For weeks I felt like I’d just upgraded the entire system. That was the trap. A strong model reads a stale instruction, quietly routes around it, and delivers the right result anyway. Capability masks debt. You don’t see the rot while the engine is strong enough to fly with wrong maps.

So one evening I decided to sit down with the agent and read our constitutional files out loud, line by line. The full identity spec. Both operating contracts. The memory index. The reference docs. I gave real feedback on each section; the agent fact-checked every claim against the live system. Does this path actually exist? Does this command run? Is this tool still installed?

What We Found

The concrete findings were specific enough to sting.

A browser automation integration we retired months ago was still presented as current in two places. A task-tracking system we switched off weeks ago still appeared as “live” in a dozen different docs. An infrastructure guide told the agent to use password-based authentication on a critical system, when the real answer was the exact opposite: key-only, full stop. That’s not just stale. That’s actively wrong.

A background automation that rolled over session memory hadn’t actually run for three months because its trigger file’s format had silently drifted away from what the parsing script expected. The same operational rule was copy-pasted into four different files, which meant burning the same tokens in context every single session while burying the real load-bearing stuff that changed rarely.

One doc was still titled after an integration we’d shut down in April. Two other docs held stock notes from two months ago, loading into context every wake. Nothing was breaking, but everything was noisier and slower than it should have been.

The Numbers That Made It Real

I tracked the line counts through the cleanup.

The always-loaded memory index went from 137 lines down to 95, much denser. Working session memory went from 103 lines to 38. And because a generated planning file was embedding the working memory wholesale, fixing that source file shrank the planning document from roughly 450 lines down to 63. One cleaned file improved every downstream consumer.

Those deletions hit different when you’re building systems that load their own instruction files into context every session. Every line you remove is throughput you reclaim.

Building the Systemic Fix

Cleanup isn’t enough. Rot comes back. So I built permanent guards.

The Per-File Review Protocol

The core ritual: human reads, agent fact-checks. I opened each constitutional file (18 core docs total). For each one, I sat with it and gave raw feedback. This feels outdated. Why is this here twice? Does anyone actually use this tool anymore? The feedback was feelings, hunches, spots where something just seemed wrong.

Then the agent took over. For every claim in that file touching the live system, it fact-checked against reality. Does that path actually exist? It ran ls to verify. Does that command run? It tried it with --help and read the output. Is that tool still installed? It checked the config that gates it. Every file went through the same gauntlet: feelings from me, receipts from the agent. No claim was fixed until there was proof.

Here’s what emerged as we went file by file.

The dangerous ones were the inverted claims. A doc telling the agent to use the wrong auth method is worse than a missing doc. It’s an invisible instruction to do the opposite of reality. That’s why fact-checking matters more than cleanup.

The Fan-Out Audit

After the 18 core files were clean, I couldn’t stop. Too many satellite docs lived outside the constitution. So I built a fan-out. Five parallel audit agents, each sweeping a different cone: skill registries, command definitions, automation scripts, prompt fragments injected at wake time, generated context files. Each agent had the same list: every system we’d ever retired, every tool we’d deprecated, every API we’d shut down.

Eighty-five findings came back, each with a file:line receipt. A doc still describing a tool killed six months ago. A help-text snippet copy-pasted from an old integration. A command definition referencing a deleted script. Comments in automation code that mentioned defunct services. None of these would break the system. All of them were noise in the context window. All would compound silently if left alone.

Then came the fix stage. I spun up three fixer agents. Each one took a slice of the 85 findings and applied them (rewrite the doc, delete the line, update the comment, point the reference elsewhere). The orchestrating session did the judgment calls only. Was this fix safe? Does it need human review? Otherwise: let the agents fix it, gather the output, merge it.

The Verification Workflow

Once all edits were applied, I built a verification workflow. It ran a set of real-scenario checks against the freshly cleaned system. Not synthetic tests. Real things the agent actually does: Does the memory parser parse the new memory format? Do all internal links resolve? Does the staleness scanner run without errors? Do config mirrors match across the system? Each check was set to run three to five times, watching for drift or intermittent failures.

The workflow found a failure on the first run. One of my own updates broke a parser path. The failure is the point. A verification that cannot fail is decoration. The workflow caught the break before it went live, I fixed it, the workflow ran again, passed. That’s the discipline.

The Permanent Guards

Zero findings after this cycle. But rot comes back. So I installed two guards that stay.

First: a weekly staleness scanner. It builds our own list of retired systems (the same list the audit agents used) and greps every live doc looking for mentions. If a reference to a dead tool creeps back in, the scanner flags it. This catches accidental reintroduction: someone copies an old code snippet, embeds old documentation, reuses a retired pattern. The scanner catches it before it spreads.

Second: a fix-it-in-the-same-session rule written into the operational contracts themselves. If an audit finds something stale, don’t file it for later. Fix it now, verify it now, commit it now. Debt that doesn’t get paid in the same cycle becomes someone else’s problem.

Thanks for reading Digital Thoughts! This post is public so feel free to share it.

Why This Matters

Here’s the thing about agents built on probabilistic models: determinism is not something you get from a good prompt. Determinism is something you engineer, with code-level guards and architectural checks. With harness safeguards that fail loud. With constraints that prevent silent drift.

Self-improvement loops are standard now, but they mostly check outputs and errors, not the instruction files themselves. Nobody was reading the CLAUDE.md itself. The human read is the check nothing automates away. From time to time, you have to sit down, open your agent’s constitution, and read it. Not skim it. Read it like you’re fact-checking a colleague’s work. If you feel it’s fine, that feeling might be the model’s intelligence doing the heavy lifting, not the architecture actually being sound.

A strong model can compensate for a lot. That’s a trap dressed as a feature. This is exactly the kind of rot I missed for months. Capability masking architectural debt is one of the basic ways you break your AI agent without realizing it.

The difference between a system that feels smart and one that actually is comes down to whether you audit it. I built a self-improving agent that could improve without reading its own constitution. That was a design flaw, and this audit caught it. I trust my car more than my AI agent because the car has hard constraints. Agents get hard constraints the same way: you audit, you build guards, you verify repeatedly. Hoping is the one strategy that never ships.

The Playbook

If you’re building an agent that will exist for more than a few weeks, this process is worth systematizing.

Step 1: File-by-File Review with Fact-Checking

Sit with the agent. Go through every core instruction file: identity, contracts, memory system, reference docs. For each claim about the live system, the agent fact-checks it. Does the file exist? Does the command run? Can the agent actually use this tool? Is this integration still active? Mark anything stale, contradictory, or outright wrong.

Step 2: One Home Per Rule

Don’t repeat instructions across files. Put the rule in one place. Everything else is a pointer. This cuts token cost and makes updates atomic. When an operational change happens, you update one file, not six.

Step 3: Move Dated History Out

Decisions from two months ago and session logs from last week have no business in live documentation. Archive them. If they matter for understanding why a decision was made, link to the archive. Don’t embed them in the constitution.

Step 4: Build a Staleness Scanner

List every system you’ve ever retired, every tool you’ve removed, every API you’ve deprecated. Turn that list into a weekly grep across your live docs. Staleness compounds. Catch it early.

Step 5: Verify With Real Scenarios

After any architectural change, don’t trust static checks. Run the agent through actual work. Multiple times. Does it actually behave the way the docs say it should? This is harder than writing the rule, and it’s non-negotiable.

This isn’t about perfection. It’s about the gap between what you think your agent’s constitution says and what it actually does. Capability masks that gap. A strong enough model will deliver anyway, and you’ll never know the rot is there.

Until you sit down and read it out loud.

The Guided Version

I packaged this exact protocol into a downloadable kit anyone can point their own agent at. It works with any harness: Claude Code, Cursor, Codex CLI, or a custom loop. The kit includes the file-by-file review template, the audit agent prompts, the fixer patterns, the verification workflow, and the staleness scanner configuration. If you want the guided version of everything above instead of building it from scratch, the Agent Architecture Audit Kit is $29 and included free for yearly Digital Thoughts subscribers.

If you want more experiments like this, where the failures and maintenance work are left in, subscribe to Digital Thoughts. I write about building with AI, not just using it.

Subscribe now

I Ran 255 AI Agents to Make Three Comics. The Line Between Slop and Good Is Thicker Than I Thought.

Pawel Jozefiak — Mon, 13 Jul 2026 11:50:55 GMT

Slop is the word of 2025. Merriam-Webster named it their word of the year. News feeds fill with the stuff. You see it on YouTube (one study counted the first 500 Shorts served to a brand-new account: 21% were AI slop, another third adjacent brainrot), on TikTok, on Pinterest, everywhere. And nobody chose this flood. It just arrived when everyone realized you could toss a one-line prompt at an LLM, scale it to infinity with no editor, and watch the machine crank out volume. The internet now runs on the output of people who aren’t paying attention to their own work.

But “AI slop” as a category still feels fuzzy to me, like everyone uses the word for different things depending on their frustration level. So I ran an experiment: I would make the same creative thing three times, with increasing amounts of human thought, and watch the quality move. I wanted to find the boundary with my own hands, and to understand what exactly changed between “this is slop” and “this is worth looking at.”

The first comic was bad in a very specific way. Fifty pages, drawn by fifty AI agents, perfectly coordinated, visually consistent, delivered in thirty minutes with zero errors. And soulless. Barely readable. A near-copy of the TV series that inspired it. I told my agent exactly that, in exactly those words, and that feedback became round two.

Here is what I was actually testing. I have never touched the artistic side of AI. I build agents, automations, the occasional game, but generating images or anything that pretends to be art? Never pulled me. What did pull me was this question: everyone talks about AI slop like it’s one thing, one flood, one texture. One problem we all agree to hate. But I wanted to see the anatomy of slop: the actual structure that makes something slop instead of work. What do you skip, mechanically, that tips the output into that category? And more important: what do you have to actually do to undo it?

So I designed an experiment. I would make the same creative task three times: a graphic novel about ordinary people waking up with early superpowers. Same theme, similar token budgets, three completely different pipelines. Between each round I would read the whole thing and write down honestly what I thought: what specifically wasn’t working, in plain words. Then I would feed that feedback back into the next round. The three finished comics are public at wiz.jock.pl/comics, with the pipeline and the costs next to each one. You can read all three and watch the quality move.

Round 1: maximum scale, minimum thought

The setup was deliberately naive, because that is what most slop production is: naive at scale. One agent writes a 50-page beat sheet in a single pass. Fifty Sonnet agents each get one page and draw it as raw SVG code. Vector shapes, gradients, text elements. No image model. No editor. Nobody ever reads the whole thing before it ships.

The coordination was genuinely impressive from an engineering standpoint: 51 agents, 3.4M tokens, zero failures, one consistent color system across 50 isolated workers who never talked to each other. As a production system, beautiful. As a comic? It looked like a UML diagram having a nightmare. Characters were geometric mannequins. The lettering was often unreadable. The story copied its inspiration beat for beat, because my one-line brief had listed the inspiration’s plot elements: eclipse, future-painter, power-stealing villain. The writer did exactly what I asked, and the fifty downstream agents did exactly what the writer said. No deviation. No editorial judgment. No one reading the full thing and going, “This doesn’t work.”

My verdict after reading the whole 50 pages: no style, no soul, nearly a copy of the TV show. Most of that was my fault (the brief was one line, and the plot elements in it came from me). Cost of the lesson: about $0 in images and an evening of tokens. I don’t mind it. That was the point of a baseline. And this is slop in its purest form: a perfectly executed pipeline that produces output nobody would read by choice.

Round 2: fix the medium, add a critic

Two structural changes. The art moved from code-drawn SVG to an actual image model (Nano Banana Pro, about $0.14 per page), with every page agent looking at its own output and retrying if the lettering was garbled. And the script got an editorial loop: a writer drafted all 50 pages, a separate critic agent tore the draft apart systematically, the writer revised based on that feedback.

The result, “The Warm Hill”, is a quiet story about a dying Silesian mining town, a caregiver who absorbs other people’s pain into her own body, and her indebted son. The art jumped a full league. Painted, moody, consistent across all fifty pages. I liked looking at it. The image model had taste that the SVG code didn’t.

And I was bored reading it.

My own verdict: the art was much better. The story was weird and detached. Little dialogue. No action. No character building. The single critic pass had produced something literary and lifeless, like a short-story journal that nobody buys. Here is where the math got interesting: the token budget was nearly identical to round one. Roughly 3M tokens. The only differences were the medium (SVG to images) and adding one critique pass. Plus about $9 of image generation. Two rounds, nearly identical cost, an enormous quality gap between them.

So the budget was never the knob that mattered. I had been thinking about scale wrong the whole time. You can throw more tokens at a bad process and all you get is a bigger version of bad. The variable was how much judgment touched the work.

Round 3: I finally did my part

Both earlier rounds failed where I had been lazy. In round 1, I gave the system a one-line prompt and let it run. In round 2, I added feedback but the feedback was shallow and the brief was still one line. That’s why for the final round I changed my own input completely. No more phone-it-in briefs.

I spent twenty minutes writing a real creative brief instead of a line. Powers at generation zero: a strong man lifts a car for four seconds and tears his tendons doing it. Powers learned like skills, with failed experiments on the page so the reader sees the cost. Disconnected storylines across five countries that converge because one weak power, a woman who finds people through dreams, slowly learns to aim her gift. A morally grey antagonist: a company founded by someone with a power, containing people like her, and holding one genuinely good argument (they found a man whose body brews a lethal virus). Inspiration allowed. Copying banned.

The pipeline grew judgment layers to match. The script went through two separate critique passes, not one: one for structure and character arcs, one only for dialogue, action choreography, and pacing. Sixty-five art agents each directed their own page and QC’d the output with their own eyes. Then a layer that didn’t exist before: editor agents read every finished chapter as drawn, like a reader would, and ordered redraws of pages that failed. They caught things I would have missed myself: a boy drawn as a grey-haired adult in a flashback. Script stage directions leaking into speech balloons. One caption of pure gibberish. They sent those pages back for revision.

The result is “Small Hours”. 65 pages, five countries, a painted cover, chapter title cards. My honest verdict: much better. The first one that reads like an actual comic with stakes. Still imperfect. Faces drift between pages in places. A few typos survived the final review. It would need more rounds and more back-and-forth to be truly done. I am completely aware of that.

But put round 1 and round 3 side by side and the quality gap is impossible to miss. And here is the thing that stayed with me: round 1 and round 2 cost roughly the same. Round 3 cost more tokens (because of the extra critique passes and the reviewer layer), but not proportionally more. The jump between 1 and 2 happened for free. The jump between 2 and 3 came from me spending twenty minutes being thoughtful instead of a minute writing a prompt.

The technical guts, for those who want them

Everything ran on Claude Code Workflows: deterministic scripts that fan out subagents, so loops and phases are code while judgment stays in the models. Model assignment per role mattered a lot, same logic as when I matched models to jobs in my agent:

Character consistency across 65 independently generated pages came from strict textual tokens pasted verbatim into every prompt. A real one from the book: “Tomasz Wierzbicki, big, blond stubble, grey beanie, orange hi-vis vest over navy overalls, dog tags, bandaged left forearm.” Hex codes for palettes, one committed art direction string reused everywhere, and a per-thread lighting accent so Lagos never looks like Gdansk.

The full three-round bill:

Failures worth sharing, because they always are. My OpenRouter account died mid-run at exactly $100.02 of $100.00, stranding the book at 23 of 65 pages; the agents degraded gracefully and saved their finished prompts to disk as blocked-files, so after a top-up the resume was mechanical. The card agents also discovered that my image script had been silently ignoring its aspect-ratio flag for weeks. And the funniest failure class: the image model kept rendering script speaker labels inside speech balloons, so a market scene proudly displayed “ABUELA: LADRON!” until a reviewer caught it. Every one of these got caught by an agent, which is its own small argument for judgment layers.

The thick line

I went in expecting a thin line, something you cross by accident. What I found is thick, visible, and easy to see. It sits exactly where human input sits. And it splits into three places where a human has to stop what the machine is doing and actually decide something.

First, the medium itself. SVG code versus an image model. This choice decides more than it looks. SVG code can be generated procedurally, plumbed through a single pipeline, and it will coordinate perfectly because every agent is writing text to the same spec. But it has no taste. It cannot render a mood or a shadow or anything that requires judgment about appearance. An image model has to be prompted, but the prompt is where human taste lives. When I switched mediums, the art jumped. That jump came from one choice: putting taste work upstream of the scale.

Second, judgment layers. Who reads the work? Who decides if it is good? Round 1 had zero. The writer output, the artists drew it, nobody sat between the draft and shipping. Round 2 added one critic. Round 3 added two critique passes plus editor agents who read chapters like readers and sent pages back for revision. Each layer cost more tokens, but the cost is not linear. One critic is cheaper than you’d expect. The jump from one to three was maybe 30 percent of the total token budget for round 3. But the impact on the output was immense. Because criticism is how you make work better. It is how you distinguish between “the machine did what I asked” and “what I asked produced something worth reading.”

Third, the brief itself. How much thought does the human put into specifying what they want? Round 1: a one-line prompt listing plot elements. Round 2: still one line, but I’d read the output from round 1. Round 3: twenty minutes of real thinking. A plain brief: here is what I want to see happen, here are the characters’ names and what they look like, here are the three main arcs, here is the ending, here is what makes the antagonist credible. Real creative direction. That is the heaviest human lift in the entire loop. And it produced the biggest quality jump.

All three sit outside the technology. Model capacity, token count, API pricing: none of them move these levers. Each one is a place where you, the human, have to sit down and decide something instead of letting the machine run.

When we use AI, what really matters is what value it creates or could create. And the difference is how much we give it, in terms of context but also our own direction and creativity. Without human creativity, we can’t really use AI to create something meaningful, fun, and entertaining.

That’s where I landed, and it matches what I keep finding on the partnership spectrum and in the value experiments: the machine amplifies whatever you put in. Put in a one-line prompt, get amplified emptiness, which is the exact definition of slop. And that emptiness traces straight back to a choice you made. Put in taste, constraints, and honest feedback, and get something with a pulse. Put in a character’s full description and a reason why the antagonist is credible instead of simply evil, and the machine has something to amplify besides noise.

One thought keeps coming back. I poured maybe an hour of real creative attention into this across three rounds, and got a readable 65-page comic. The work was better than I expected, given how little time I spent. Now imagine an actual comic artist, someone with years of story instinct and visual training, pouring two or three full days into this same loop. Their sense of how a scene should flow. Their eye for which pages need redraws. Their understanding of character design and what makes a face readable across 65 pages. I think there is a real chance they would come out with something printable. Something you could hold and show people. The tools are already there. The model capacity is already there. Taste is the scarce part. And every experiment like this makes it clearer that taste will only get more valuable.

Read all three at wiz.jock.pl/comics and judge the line yourself. Round 1 is kept up unedited, as the baseline. It earned that much, because it is the truest version of what slop is: perfect execution of thoughtless input.

I Maxed Out Fable 5 and Regretted It. Here’s How I Actually Run It Now

Pawel Jozefiak — Wed, 08 Jul 2026 20:22:02 GMT

The first day I had Fable 5, I did the obvious thing. I turned the reasoning effort to max and let it rip.

If the model is this smart, give it room to think. That was my logic. The bill scared me and the code came out worse. It rewrote files I never asked it to touch, added abstractions nobody needed, and turned a two line fix into a refactor with opinions.

So I spent the next couple of weeks figuring out how to actually drive it. Not the marketing version. The version where you use it every day, on real work, and want the output to be good without the cost getting stupid.

This is that playbook. Effort settings, model routing, how I hand it big autonomous jobs, and how it changes vibe coding. Steal what is useful.

The one setting that quietly controls your bill

Fable 5 lets you choose how hard it thinks, through low, medium, high, and the heavier tiers above (xhigh and max). The effort setting is the single biggest lever you have, and most people move it the wrong way.

Here is what it actually does. It does not decide how long the model can work or how hard a problem it can crack. It decides how much it thinks per step. Push it past high, into xhigh or max, and it does not take more steps. It overthinks every single one. Most steps in real work are boring. They do not need a philosophy degree.

The cost of that is worse than it looks, because reasoning compounds. Every thinking token the model spends on step one becomes part of the history it re-reads on step two, and step three, and every step after. On a long agentic run, a fat reasoning habit does not add cost in a straight line. It multiplies. This is the same trap I fell into before I took token waste seriously, and it is why cutting reasoning is the single best move you can make.

Now the honest part. On the hardest coding benchmarks, more effort does score higher, from around 11% at low to 31% at max. But your day is not the hardest benchmark problem. It is a hundred small edits. And on Cognition’s FrontierCode test, which checks whether a model can pass hard tasks while holding the bar of a real production codebase, Fable scores highest of any frontier model even at medium effort. You do not need to redline it to get its best work.

So here is the rule I actually use:

Quick tell: if a small ask comes back with a big diff full of changes you did not request, your effort is too high for that task. Drop it a notch.

Don’t pay genius rates for grunt work

The second habit that dropped my costs the most: I stopped using the smartest model for everything.

Fable is expensive, and it should be. So I let it do the thing only it does well, which is judgment. Architecture, taste, API design, the “is this the right shape” calls, the final review. The grunt work goes somewhere cheaper. This is the grown-up version of the day I swapped my main agent from Opus down to Haiku and it got better. Think of it as casting a role for each job.

The trick is making this automatic so you are not hand-picking a model every time. I put the routing rules straight into my agent’s instructions, so the smart model reads them and delegates on its own. This is one more reason I keep my CLAUDE.md tight and specific. Here is roughly what lives in mine:

## Model routing
Default: high effort. Never max for routine work.
- Judgment, architecture, taste, review  -> best model.
- Bulk mechanical work, log digging, huge files, migrations -> cheaper model.

Rule: judge the output. Price is a tiebreaker. If a cheap model misses
the bar, redo it with a smarter one without asking. Escalating beats
shipping mediocre.

When a task needs a big, specific pile of context, spin a subagent for it
and hand back a result. Do not drag all of it into the main thread.

That last line matters more than it looks. Routing heavy context to a subagent is about quality as much as cost. An agent buried in its own accumulated context gets measurably worse, which I dug into in how to break your AI agent. A cheap subagent reading a giant log and handing back three lines keeps your main thread sharp and cheap. That same discipline is what got my agent to where it costs about 400 dollars a month and nearly pays for itself. If you want to go deeper on running agents lean, I put together an Agent Efficiency Kit.

Handing it a big job (agentic runs)

This is where Fable earns its keep, and where the effort and routing habits pay off, because a long run magnifies every bad token.

The shift in my head was this. Stop sending one prompt and babysitting it. Give it a goal with pass conditions and let it work. The model is now good enough to break a big job into pieces, spawn help for the parts that need it, and keep going until the conditions are met. My job moved from typing to defining the goal and guarding the one thing that cannot be undone.

Two tools do the heavy lifting, and they are different. Subagents are for fan-out reads, five files to analyze, five subagents. Workflows are for staged work where a later step depends on an earlier one, like review then verify then merge. Here is a real prompt I used to clean up a pile of stale pull requests:

Investigate every open PR in this repo. Use a workflow that spins up one
reviewer per PR, then a second pass with two judges to stress-test each
verdict. Give me a triage list: ready to merge, needs a small rebase,
already trumped by other work, or good idea but better rewritten.
Link each PR. Do not merge anything yet.

It came back with a clean triage in a few minutes. Once I agreed with the plan, I turned it into a goal:

Goal: work through todo.md top to bottom. For each item, implement it on a
fresh branch, run the checks, open a PR, and merge to staging only after the
automated reviewers pass. Mark the item done in todo.md and commit as you go.
Never touch production. Keep going until every item is done.

Then it ran for hours while I did other things, the way my agent runs its night shifts, and a backlog that had been stuck for weeks landed in a couple of days.

Although I gave it a lot of freedom, one line in that prompt stays non-negotiable: never touch production. The agent can do anything reversible. It can branch, merge to staging, and rewrite whatever it wants. The one step that is hard to undo stays with a human. That boundary is the whole reason you can hand it this much rope. I have handed agents plenty of rope before, and the rope is only safe because it ends somewhere. If you are choosing the tool you leave running unattended, I compared a bunch of them in this harness roundup.

Vibe coding: restraint when you’re not reading every line

Vibe coding is the case where this matters most, because the whole point is that you are not reading the diff. You describe what you want, the model builds it, you check the result. That only works if the model stays disciplined, and max effort is the opposite of disciplined.

Four things that make vibe coding with Fable actually hold up:

1. Keep it on high, keep the asks small. Max over-edits, and when you are not reading the code you cannot catch the bloat. A model that changes ten files when you asked for one is dangerous precisely because it looks like progress. Smaller asks give you smaller diffs you can actually reason about.

2. Get a second model to review before you accept. This is the cheapest insurance there is. Have a different, cheaper model read the change and tell you what it does and what looks risky. A second set of eyes you did not have to write, for pennies.

3. Use branches and worktrees so a bad run cannot wreck your main. Risky change goes on its own branch. If it goes sideways, you throw the branch away and nothing else is touched. If you are new to this, I wrote a basics guide to git for AI builders for exactly this reason.

4. Watch the clock, it tells you about your codebase. This one surprised me. How long the model takes to do a thing is a signal:

One more, and it is not optional: vibe-coded apps leak. Fast-built projects ship exposed keys and broken auth all the time, which I learned the hard way and wrote up in this security reality check. Before anything you vibe-coded goes near real users, spend a cheap model on a security pass. It is the best few cents you will spend.

Why all of this compounds

There is an old joke among developers, the grug brained developer. Grug not smart, but grug program many year and learn things. Grug’s rule: why use many words when few word do trick.

It reads dumb on purpose, and that is the point. Fable was built for that idea. Anthropic does not sell it as the model that thinks the longest. They sell it as more capable work in fewer turns, runs that finish 25 to 30 percent faster than the last model with less back and forth. Crank it to max and route everything through the priciest model, and you are fighting the grain of the wood.

Here is the part that still surprises me. Holding the model back made me more ambitious, not less. Because a run is cheaper and cleaner, I hand it bigger jobs, leave it running longer, and trust it with more. The restraint is what buys the reach.

The models are good enough now that the ceiling stopped being their intelligence. It is our discipline in using them. The way to get the most out of the smartest tool on my machine turned out to be asking it for less.

I write about building with AI agents from a practitioner’s seat, no hype and no affiliate links. If that is useful, subscribe for the weekly dispatch.

Subscribe now

How to Break Your AI Agent (Basics)

Pawel Jozefiak — Wed, 24 Jun 2026 12:06:37 GMT

I’ve been building my own AI agent for months now. Claude Code, Codex, a pile of other tools wired together into one thing that runs my day. A while back I wrote about how to build your first one, and that post covered a few of the mistakes I made while putting it together.

This one is different. It is about breaking the agent you already have. The one you use every day. The one running in the background while you sleep, the one you trust to actually do things.

I also wrote about the time I almost fried my agent and my Mac Mini, but those were my specific accidents. This is the general version. The field guide. The list of ways any agent goes sideways once it is in your hands, so you can make fewer of these mistakes, or at least see them coming.

And here is the part people skip past. This is not only about custom agents like mine. The same things break ChatGPT, Claude, a Zapier flow, whatever framework you picked off the shelf. AI moves fast. Speed does not make a thing unbreakable. You can break anything you build the moment you start using it, and an agent is no exception.

First, the reason agents break at all

It is not that the model is dumb. It is math.

An agent does work in steps. Read this, call that tool, decide, act, check, repeat. Steps multiply, they do not average. If each step works 99% of the time, ten steps in a row work about 90% of the time. A hundred steps, around 37%. A thousand steps, basically never. Researchers measured this directly, and it gets worse, because the errors are not independent. One wrong step nudges the next one wrong too.

So every time you add surface to your agent, more tools, more memory, more steps, more things it can touch, you are not adding risk in a straight line. You are multiplying it. Keep that in your head. It quietly explains every item below.

1. You overbuild it

This one is mine. I have a real problem with overbuilding.

When I want something, I build it. Then I want another thing, so I build that too. Each piece works on its own. The trouble is the pile. Building toward your own needs feels right in the moment, because it is natural. You need this, you need that, so you keep adding. I learned the cost of it too late.

More pieces means a higher chance that something, somewhere, is broken at any given moment. That is just probability. And past a point the tools start working against the agent instead of for it. There is a name for this now. Microsoft researchers call it tool-space interference. Give a model too many tools and it picks the wrong one, burns tokens, or invents a call that does not exist. Studies have measured the drop in tool-selection accuracy anywhere from 7 to 85% as the catalog grows, and it gets worse for tools sitting in the middle of a long list, the same lost-in-the-middle effect that hits long context. OpenAI limits a single request to 128 tools, and coding agents like Cursor warn that quality slips well before you stack even a few dozen. Either way, the cap is not where the trouble starts. Every tool you add competes for the model’s attention long before you hit any limit.

Context has the same shape of problem. There is solid research on context rot now: across 18 frontier models, accuracy fell 30 to 50% as more was stuffed into the window, well before the window was even full. On the million-token models it started showing up around 300 to 400 thousand tokens. So an agent buried in its own accumulated context gets measurably worse, not because it ran out of room, but because the room got noisy.

The fix is not glamorous. Prune. Do a periodic checkup on your skills, your tools, your memory, your core files. If your error registry is lit up red, the agent is already telling you something is broken and you stopped reading it. I keep my CLAUDE.md tight partly for this, to keep the core readable instead of letting it bloat into something nobody can hold in their head.

There is a structural fix too, and it is the one I would push hardest. Stop showing the agent everything at once. The pattern that holds up in production is search-then-load: the agent keeps a small index of what it can do, looks up the few tools it needs for the task in front of it, and loads only those. Same idea for context. Treat the window like a budget you spend on purpose and compact it often, rather than letting months of history pile up and rot. The agent that carries less is the agent that stays sharp.

2. You make one agent do everything

My first idea of an agent was Jarvis. One mind that handles all of it. I owned up to this in the first-agent post too. It is the romantic version everyone starts with.

For strictly personal stuff with tight, specific context, one agent is genuinely fine. Useful, even. But the moment the work gets complex, writing code, running services, juggling separate projects, the single agent starts to behave like a monolithic codebase. Every new thing makes it heavier and harder to maintain. Eventually it is so bloated you spend more time managing the agent than getting anything out of it. If you want to break it, just keep making it bigger until it pops.

This is an open argument in the field right now, and both sides are worth knowing. Anthropic built a multi-agent research system with a lead agent that hands work to specialized subagents, each with its own clean context, and it beat a single agent by over 90% on their evals. The catch: it burned roughly 15 times more tokens, and it was worse for tightly connected work like coding. Cognition went the other way in a piece literally titled Don’t Build Multi-Agents, arguing that splitting context across agents makes them fragile, because actions carry implicit decisions, and conflicting decisions carry bad results.

Both are right, which is the useful part. The version that holds up: one agent stays in charge of the thread and does the actual writing and acting, while the extra agents go fetch context and intelligence rather than take conflicting actions of their own. That is roughly where Cognition landed after more time in production too.

If you want one rule for where to draw the line, use context. Keep work inside a single agent while the context stays small and shared. The moment a task drags in a big, specific pile of context the main agent does not otherwise need, that is your seam. Cut there. The job that needs to read a whole codebase, or a month of a project’s history, or a pile of research, gets its own agent with its own window, and hands back a result instead of dumping all of that into the one mind you actually talk to.

That is the change I made early, and I am glad I did. I have one main agent I talk to. Anything with heavy or specific context, a particular project, a content task, a research dig, gets spun off into a subagent or a workflow. In Claude Code I lean on workflows and subagents constantly. The real trick is the instructions. Your main CLAUDE.md or agents file has to tell the agent, in plain words, when to spawn help instead of swallowing the whole thing itself. If it does not know to delegate, it will try to do everything, and you are back to the monolith. I went deeper on this idea in The Bounded AI Agent.

3. You let it poison its own memory

An agent with memory is great until it remembers something wrong.

This one only shows up in use, never while you build. You save a fact, the agent recalls it later and acts on it, and if that fact was wrong, it is now wrong forever, on a loop. One bad entry, recalled a hundred times. Microsoft’s red-teaming team has a clean name for a cousin of this, session context contamination, where junk from one context leaks into the next and quietly steers the agent off course.

I hit this regularly. My agent writes things to its own memory, and every so often a drift warning fires because something it saved no longer matches reality. Without that check, the bad memory just sits there, shaping decisions, looking exactly like a good one.

The fix is hygiene. Memory needs the same discipline as code. Verify before you save. Link facts so you can trace where they came from. Let the system flag drift instead of trusting it blindly. I built a self-improving loop for exactly this, but a loop only helps if it can catch its own bad entries. A memory you never audit is not an asset. It is a slow leak you cannot see.

What actually works for me is layers, not one big bucket. A small working memory for the task at hand, and a durable layer for facts that should outlive the session. Every durable fact gets a source and a date, so I can trace where it came from and retire it once it goes stale. And the agent is allowed to question its own memory, a quiet check that fires when a saved fact stops matching reality. A fact with no source and no expiry is a rumor your agent will repeat with full confidence, forever.

4. You give it no fallback

This one is simple. If you want an agent that runs 24/7, you need a fallback for the model.

A base model is fused into the agent, sometimes a few of them, and sooner or later one is unreachable. Rate limited, deprecated, down for an hour, whatever it is. A fallback is not a permanent plan B for some other model. It is the thing that keeps the lights on when the main thing goes dark. I run an OpenRouter subscription partly for this, so the agent can fail over to another provider instead of just stopping. It costs a bit, you pay for the keys, and it still beats having nothing.

One honest warning, because I picked it up reading about other people’s outages. A single fallback is not a guarantee. In August 2025, OpenRouter itself went down for about 50 minutes, and that took its own fallbacks with it. So layer it. A gateway for provider failover, plus something local as a floor. I run a 35B model on my Mac Mini for that reason, small jobs and a last resort for when the network itself is the problem. Not everyone can run a decent local model, I know that. But even a small one beats a dead agent.

One more lesson, the one most people skip. A fallback you have never tested is not a fallback. It is a second thing you are also assuming works. Pull the primary on purpose every so often and watch what happens. Does the agent actually fail over, or does it just fall on its face. I would rather find a broken failover on a quiet Tuesday than at 3am when the main model is down and everything is dark.

5. You trust it without checking

The scariest failures are the quiet ones.

An agent that crashes is easy. You see it, you fix it. The dangerous one is the agent that says done and did nothing. It writes a log line, the log line says success, and you believe it. The actual work never happened. From the outside it looks alive. It is the difference between a heartbeat on a screen and a pulse you actually put your fingers on.

Go back to the step math from the top. Without checking each step, you never see the 37%. You see the green light and assume the other 63 ran fine. I have been bitten by this enough times that I built a watchdog whose entire job is to catch the skip, the loop that runs and logs and accomplishes nothing. When I wrote about the agent starting to fix itself, half of that work was really about catching silent failure before it had a chance to compound.

The fix is to stop trusting shallow signals. A log line is not proof. Re-run the thing that was supposed to happen and look at the result. It is the same reason I am religious about commits as save points now. You want a real, inspectable trail, not a vibe that it probably worked.

The lesson under the lesson: check the outcome, not the activity. Did the file actually change. Did the message actually send. Did the row actually land in the database. Whether the function ran without throwing tells you almost nothing. And make the check safe to run twice, because you will run it twice. A fix is not done because the code ran once in a session. It is done when I re-run the exact thing that was failing and watch it pass. Until then it is a guess wearing a green checkmark.

Share Digital Thoughts

6. You never challenge it

This last one is for the people running their own custom agent. Challenge it. On purpose.

I try to do this more and more. I throw scenarios at my agent that it has never, or rarely, seen. Rapid-fire messages. Weird formats. Requests it has no obvious handler for. Things I do not even need, just to watch what happens. The whole promise of an agent is that it works out a solution to the problem in front of it. The reality is that it often cannot, and you want to find that out before a real situation does it for you.

Here is the piece of advice I would hand any new builder. When you throw a challenge at your agent and it fails, the problem is almost never the challenge. It is the architecture underneath that could not let the agent get there. A whole discipline is forming around this idea, people are calling it harness engineering, the argument being that the scaffolding around the model, context, tools, memory, verification, decides whether it succeeds far more than the raw model does. Microsoft’s year of red-teaming agents says the same thing from the security side: systems that pass model-level tests still fall apart under real pressure, because the failure lives in the system, not the brain.

So make it a habit. Keep a small set of nasty cases, the weird formats, the rapid-fire messages, the request that needs three tools in a row, and run them again every time you change the architecture. That is a regression test for an agent. When one fails, you know where to look, because the harness is a short list: how context gets in, how tools are exposed, how memory is stored and recalled, how work gets verified, what the agent is and is not allowed to touch. The break is almost always in one of those, not in the model’s head.

So when it breaks, do not reach for the model. Look at what stopped it, and fix that. An agent you never challenge is one that quietly stops growing. One day you ask it for something slightly new, it just cannot do it, and you have no idea why.

None of this means stop building. Build. Overbuild, even, for a while, because that is how you find the edges. Just know that the day you start using an agent is the day you start breaking it, and that is okay. Every break is a map to the next thing worth fixing.

My agent messed up plenty of things. Everything was my fault, because I am the architect. It cost me time to fix, and I genuinely do not mind. It is progress, and I accept that.

If you are building your own from scratch, start here. Then come back and break it on purpose. That part is the actual work.

Thanks for reading Digital Thoughts! This post is public so feel free to share it.

I Trust My Car More Than My AI Agent. That Gap Is Where We’re Going.

Pawel Jozefiak — Wed, 17 Jun 2026 09:37:50 GMT

My agent made me a lot more effective. It also made me watch my own back more.

Both of those are true, and I have stopped pretending the second one away. I built my own agent, I gave it its own machine, and it does real work for me every day. It knows who I am, not just what I want. The effectiveness is real. So is the quiet voice in the back of my head that now tracks what might break while the thing is running.

That voice is the interesting part. It points straight at what this whole wave is actually about. Trust.

Trust is a mileage problem

Think about a car. When I get in mine, I assume it starts. I assume it will not catch fire on the way to Katowice. I never sat down and decided to trust it. I drove it something like a hundred thousand kilometers and the trust built itself. I know its sounds. I know the one strange thing it does on a cold morning. I can predict it because I have repeated it that many times.

Trust is repetition plus outcomes you can predict. That is the whole recipe. There is always room for a bad day, a flat tyre, a dead battery in February. Although the range of what can go wrong is small, and I know the edges of it.

With an agent that recipe only half works. For some tasks I have the same calm I have with the car. For others I am still a new driver, both hands on the wheel, watching the road like it owes me money.

Deterministic things earn trust faster

Here is the pattern I keep running into. The more deterministic the task, the faster I trust it. A script that renames the same files the same way every night, I stopped watching that months ago. It is boring, and boring is exactly the point.

The more agentic the task, the more I am looking at a black box. Open-ended work, many steps, judgment calls, recovering from its own mistakes halfway through a run. That is where outcomes spread out and prediction gets hard. The model matters here. So does the architecture around it, the tools it can call, the memory it carries. I wrote a whole post about what happens when an agent meets the messy real world and stops behaving like the demo.

The ceiling right now is technical and it is not a mystery. Context limits. Memory that does not persist the way I want it to. Retrieval that grabs the wrong thing at the wrong moment. Tool calls that fail quietly. An agent that gets stuck and does not notice it is stuck. Every builder I talk to is fighting the same short list. I wrote about drawing hard edges around an agent so it stays inside what it is good at in the bounded agent post, and about giving it a memory that actually compounds in the one on my self-improving agent.

None of these limits are permanent. I have watched all of them get better over the past year, in jumps, never on a tidy schedule. That detail matters for everything that comes next.

Now run it forward

Assume the boring version of the future. Steady improvement, the kind we have already been getting. Better models, more stable tools, memory that holds, retrieval that lands where you point it. The frontier gets the headlines. The floor is the thing that moves people’s lives, and the floor is what I am watching.

Right now an agent like mine is a nerd object. You need to be deep in code, or at least deep in tinkering, to get real value out of it. Most people who say they use AI mean a chat window. Most companies that say they use AI are in the 88% with almost nothing to show for it. The capability is sitting right there. The on-ramp is the missing piece.

Apple is making the most interesting bet on that on-ramp. At WWDC this month they finally committed to the big Siri overhaul, an assistant that can actually chain multi-step tasks, with an agent layer wired into the App Store so you can hand off things like booking a table or running your smart home. They are building it on Google’s Gemini, which tells you that even Apple decided the raw model is becoming a commodity and the product is the assistant on top. It will not ship in the EU at launch, the usual regulatory reason. I think this is the right move and it might genuinely work. Putting the agent in front of normal customers is the whole game.

Everyone’s agent is a lot of agents

Here is the part that gets skipped. When everyone has an agent, the browsing stops being human. Your software does it for you, at machine speed.

When my agent works, it touches more of the web in an hour than I would in a day. It crawls, it reads, it calls APIs, it goes and shops. Multiply that by a few hundred million people and traffic on the open web spikes. The humans did not arrive in bigger numbers. Their agents did.

That has a bill, and the bill lands in the physical world. Inference is already about two-thirds of all AI compute this year, up from a third in 2023. Data center electricity demand is climbing double digits every year. GPU prices are not coming down. For a while I expect the cost of good AI to go up before it comes down, because demand is bending faster than supply, and power and silicon are real, finite things. That is part of why I keep a local model running on a cheap Mac mini and swap its brain when I feel like it. The local one is slower and dumber than the cloud, and I keep it anyway. I want a floor under me that does not move when the market does.

The question everyone actually asks

Does it take the jobs. That is the real question hiding under all the others.

My honest read is that in the long run it makes more work than it removes. The most cited forecast going around, from the World Economic Forum, lands on roughly 170 million new roles and 92 million gone by 2030. A net gain, with about a fifth of all jobs changing shape somewhere in the middle.

A net gain is cold comfort if you are one of the 92 million. This is a revolution that asks people to move from one kind of work to another, and people do not all move at the same speed. Some are ready. Some are not, through no fault of their own. Closing that gap is a job for policy and pacing and a bit of patience, not something a model fixes. I have written before about the skills that hold their value and about what we should even be teaching kids now that AI writes the code. I do not have a clean answer. I have a direction. The durable move is to get good at pointing this stuff. Racing it is a losing game.

The physical half is slower

Everything above is the digital half. A personal assistant for everything that lives on a screen is close. A personal assistant for anything physical needs a body, and bodies are the hard part.

Robots are catching up faster than I expected though. 1X is shipping its NEO home robot to US homes this year, twenty thousand dollars up front or five hundred a month, with a human quietly supervising the tasks it has not learned yet. Figure has robots working a BMW line. Tesla is gutting a car factory to build Optimus. The honest timeline for a robot that is genuinely useful in a normal home is 2028 to 2032, not next spring, and I went into why in the post about inviting robots into our homes. Still, that is a couple of years out, not science fiction. Close enough that I already think about it.

We are still in the wild west

One more thing, because I think most people still underrate what is already sitting in front of them. We are in the wild west. Barely any rules, uneven tools, and a lot of folks treating an agent like a slightly nicer autocomplete.

Then last week a government had two frontier models pulled. Anthropic disabled Fable 5 and Mythos 5 for everyone on the planet after a US export-control order meant to keep foreign nationals away from the model’s cybersecurity ability, which I covered in my June opinions. They could not filter cleanly by nationality, so they switched both off for the entire world. Europe called it a wake-up call for sovereign AI.

Sit with that for a second. A government looked at a piece of software and decided it was close enough to a weapon to control who gets to touch it. Nobody controls autocomplete that way. That is the tell. These tools are already strong enough to be governed like dangerous things, and most of the people who could be using them well have not clocked it yet.

Where I actually land

So where does this go. More people get an agent that works. The web fills up with software acting for us. Compute gets more expensive before it gets cheaper. Jobs churn hard and then settle higher. Robots turn up for the physical half later than the hype promised and sooner than the skeptics will admit. And somewhere in the middle, AI stops being a label a company staples onto a product and becomes the thing quietly doing the work.

I have lived with one of these long enough to be careful with predictions. I have also lived with it long enough to know it already changed how I work, on the days it behaves and the days it does not. So I will say plainly where I land. I am pragmatically optimistic. None of this will be smooth. I am optimistic anyway, because every time I check, the floor is higher than it was the last time.

It might turn out better than we think. I would not have written that sentence a year ago. For now it is enough to keep building, both hands still on the wheel.

Thanks for reading Digital Thoughts! This post is public so feel free to share it.

AI Opinions: June 2026. Fable 5 Hands-On, Billing Split Countdown, OpenAI Banks Your Resets

Pawel Jozefiak — Fri, 12 Jun 2026 09:33:46 GMT

First, some honesty

I owe you a bit of context before this one. For the last three weeks I have been posting at least once a week, but I was so deep into doing things that I got completely lost in experimentation. New things, broken things, rebuilt things. I had no extra time to turn any of it into a proper post, and I have always tried to be transparent about that trade. I focus on doing things rather than writing about them. The writing is a byproduct of the doing, and I would rather keep it in that order.

So no promises here. I am not announcing a new cadence or turning over a new leaf. I keep building, I keep trying to send you the most valuable parts from my point of view, and you are always invited to discuss it and build along.

This one is a catch-up post. Several things happened at once and I had no opportunity to address any of them. Treat it as the light version of my posts, in the same spirit as the April opinions roundup. I hope you will like it.

Oh and BTW. THANKS FOR 3200 people reading me! Greatful to be here!

The billing split lands in three days

The thing closest on the calendar first. On June 15 Anthropic splits its subscription billing: programmatic use of Claude, so the Agent SDK, headless claude -p runs, GitHub Actions, anything spawned from a script instead of typed by a human, stops counting against your plan limits and moves to a separate monthly credit, metered at standard API rates.

One correction to my own earlier framing, because I went through the fine print again this week. I kept repeating “$200 per month” and that number is true for me, on the Max 20x plan. The credit is actually tiered: Pro gets $20, Max 5x gets $100, Max 20x gets $200. Enterprise Standard seats get exactly nothing. So when you do your own math, do it against your plan, because the spread between $20 and $200 is the difference between a toy budget and a real one.

The mechanics matter more than the headline number, honestly. Credits are per-user and cannot be pooled. Unused credit does not roll over. You have to claim it once in your account before June 15. And the part I would put in bold if I were Anthropic’s documentation team: when the credit runs out, your automated requests simply stop. No queue, no automatic downgrade to a cheaper model. There is an overflow toggle that bills the excess at API rates, and it ships turned off. The default behavior of this change is your agent going silent mid-month.

I have been working around all of this for a while and I wrote a dedicated post with the four mitigations I actually wired in: Anthropic repriced my agent, four mitigations before June 15. If the deadline affects you, that post is the practical one. This section is just the reminder that the clock is at three days.

And the painful detail, the reason it is hard for me to fit into even the $200 tier: programmatic use gets counted like API usage. Real token costs, not the heavily subsidized subscription math we got used to. One widely shared gist did the napkin math and called it a 12x to 175x effective price increase depending on workload. That spread sounds dramatic, but it roughly matches what my own measurements say: the same work, priced honestly, turns out to be expensive. That gap between subscription pricing and honest pricing is the whole story of this post, actually. Keep it in mind for the Fable section.

The obvious mitigation many skips

While optimizing my own agent for this, one thing kept proving itself over and over: smart model routing. It is very obvious and almost nobody does it. If you run Opus for everything, you will hit your ceiling embarrassingly fast. Route the same request to Sonnet and it costs a fraction. I wrote about the day I switched my agent from Opus to Haiku and it got better, and that lesson aged well. Haiku still handles a surprising share of the simple requests flowing through my system.

Below Haiku there is another floor: local models. The really simple jobs, classification, triage, formatting, run on a 35B model on my $600 Mac Mini and cost exactly nothing per token. I have been measuring this stack for months now and the routing is what keeps the whole thing viable. After June 15 it stops being an optimization hobby and becomes the difference between an agent that runs all month and one that dies on the 19th.

And if your agent is model-agnostic, you are not even locked to one vendor. Other providers can supplement the expensive paths; my $20 OpenRouter fallback exists for exactly this kind of squeeze. Everything is possible here. It is all up to our creativity, I think. If you want the shortcuts I use for this, the agent playbooks on the Wiz store cover the routing setup in detail.

OpenAI is fighting dirty, and I like it

Now the fun one. Yesterday OpenAI announced on X that Codex users can now save their rate limit resets and spend them later. Their words: they heard we wanted to use resets on our own time. Every eligible account on Go, Plus, Pro and Business got one banked reset for free, and for two weeks Plus and Pro users can invite up to three friends, with both sides earning another reset when the friend sends their first Codex message.

The fine print is pure growth hack. The reward lands only when the invited friend actually starts using Codex, an invite alone earns nothing. Banked resets expire after 30 days. It is loyalty points for compute, complete with a referral program. I am half joking and half impressed, because the underlying feature is genuinely user-friendly: months of complaints about resets firing at fixed times, often in the middle of someone’s night, and OpenAI responded by handing the timer to the user.

Read the timing too. Anthropic is three days from making programmatic use more expensive, and OpenAI responds by making its limits more flexible and literally giving spare capacity away. This is a very direct shot in the subscription war, and it is totally something Anthropic would never do. One company is tightening the meter, the other is letting you carry your unused minutes to next month, like a mobile operator from 2005.

I have my history with Codex, including cancelling it and coming back once already, so I am watching this with sympathy for both sides. Competition like this is the only thing that keeps our subscriptions honest.

A few days with Fable 5

And then there is the new model. Anthropic shipped Claude Fable 5 on June 9, a new tier above Opus. The construction is unusual: there is a twin called Mythos 5, the same underlying model, and the difference is safety plumbing. Fable runs classifiers, and when a request looks like offensive cybersecurity, dangerous biology and chemistry, or an attempt to distill the model, the response silently falls back to Opus 4.8. Mythos skips the cyber guardrails and goes only to approved organizations, government cyber defenders and the like. Anthropic says more than 95% of Fable sessions never touch the fallback. For what it is worth, in a few days of agent work I have not knowingly hit it once.

I have been running Fable 5 with my agents since launch. Short version: it is good. Much better than Opus for agentic work, definitely. It is more proactive. It pushes forward with the tasks I give it instead of stopping at the first checkpoint to ask how I feel about things.

The thing I appreciate most is how deep it goes. This was always my quiet complaint about the Opus line: Opus tries to be specific and on the spot, and sometimes that means it misses the full context around the problem. It fixes the line you pointed at. That is also why I always liked Codex with GPT-5.5 a little more for audits, because it reads the whole module before it touches anything. Fable 5 works like that. It audits around the problem. It goes very deep into the code, it references things I did not mention, and it holds up over long runs better than anything I have used.

One concrete example. I gave it some architecture changes on my AI agent, the kind of task where Opus usually needs a round of feedback from me halfway through. Fable ran longer than Opus would have, and when I checked the outcome I had genuinely no notes. It did what I wanted and in a few places a little more. That almost never happens.

The benchmark numbers are loud, and I would hold them loosely. Anthropic claims state of the art nearly across the board: 80.3% on SWE-Bench Pro against 69.2% for Opus 4.8, and a story about Stripe migrating a 50-million-line Ruby codebase in a day, work estimated at over two team-months. Vendor numbers, vendor anecdotes. For balance, Endor Labs ran an independent security benchmark where Fable came out middling, and they documented benchmark contamination in 19% of their test instances. My hands-on lands somewhere between the marketing and the skepticism: the long-run agentic improvement is real, I can feel it in my own work. The superlatives I leave to the launch page.

Now the catch, and you already know where this is going: the cost. Fable 5 is priced at $10 per million input tokens and $50 per million output tokens. That is double the Opus sticker price, although to be fair it is also less than half of what the Mythos Preview was going for. Right now it runs inside our subscription usage limits, but only until June 22. On June 23 it leaves the subscription pool and you pay for it with usage credits. Anthropic says the removal is temporary and that they aim to bring it back into plans when capacity allows. We will see. The internet is predictably furious about the two-week tease, and I get it: handing everyone the best model for free and then putting it behind a meter is a very effective way to make people feel the gap.

And it burns tokens fast, faster than anything I have seen, because being thorough is exactly what costs tokens. The depth I praised three paragraphs ago is the same property that empties the budget.

So my take: Fable 5 is a specialist tool, at least at this price. Security work, maybe. Big architecture passes, long autonomous builds, the tasks where one excellent run beats five cheap ones. For most jobs Opus remains my default, and the routing logic from the section above does not change. It just gets one more expensive tier at the top to route to, sparingly.

Where I think this is going

Everything is getting pricier at the top while our subscriptions still look generous. I do not think that lasts.

Look at the line Anthropic actually drew with the billing split, because it is sharper than “interactive versus automated.” Chatting on claude.ai stays flat-rate. Interactive Claude Code in your own terminal stays flat-rate. But run Claude inside Zed, a fully interactive session with a human typing every prompt, and it bills against the credit anyway, because it arrives through the Agent SDK. The meter follows the integration surface, not the human. First-party surfaces stay subsidized. Everything you build or plug in yourself becomes usage.

That is the pattern I see everywhere now: the soft cutoff. In the billing split, in Fable 5 leaving the subscription pool after two weeks, even in OpenAI gamifying its limits with expiry dates. Nobody will take your flat-rate plan away. They will just keep moving the best things outside of it, one model and one surface at a time.

Although that sounds gloomy, I do not really mind it. Honest prices force better engineering, and better engineering is the part I enjoy. The builders who learn to route work to the right model will keep their costs flat while the quality ceiling rises. That is the game now, and it is a game you can actually win.

I Barely Specced a Card Game. My Agent Built 14 Enemies, Bosses, and a Story.

Pawel Jozefiak — Wed, 03 Jun 2026 09:53:12 GMT

A few months ago I gave Wiz, my AI agent, a one-line idea: a browser card game where every card is a number. Balatro-ish, roguelite, drag numbers into zones. Then I mostly walked away from it.

This week I opened it again. I remembered it as a rough prototype. A combat screen, maybe a couple of enemies. I was wrong. Sitting there was a full roguelite: two acts, fourteen enemy types including two multi-phase bosses, relics, curses, events, a meta-progression layer. It is live at wiz.jock.pl/ten-ish/. You can play it right now.

That gap, between what I remembered building and what was actually there, is what this post is about.

What the game is

The core is small on purpose. Your deck is ten cards, numbered 1 through 10. Each turn you draw five and drag them into three zones: Attack, Defense, Ability. A number in Attack deals that much damage. In Defense it blocks. In Ability it triggers an effect, if you have one.

That is the skeleton. Everything grows on top of it. Cards pick up traits, so a 7 is prime, a 10 is high, and negative cards behave like their own little trap. Enemies react to those traits and cycle through stances that telegraph what is coming next. Your deck mutates with modifiers, curses, and the occasional card you wish you had not picked up. Like any roguelite, the real game is what your deck looks like fifteen fights in.

What I keep sitting with is how little I actually did

I did not write Phaser code. I do not really know Phaser. I told Wiz “browser card game, roguelite, numbers as cards” and it chose the renderer, the bundler, the scene graph, the combat resolution. I reviewed. I steered. I argued about balance. I did not implement.

Two years ago I was already doing this, before anyone slapped the name vibe coding on it. Back then, to get anything useful out of it, I needed a lot of tokens, a lot of time, and a lot of prompting. I would describe a thing five different ways and still get back something I had to mostly rewrite.

Now I do not. I need to know the idea. I need a direction. I need to know how it should feel and look. That is basically it. The model fills the enormous space between “numbers as cards, make it tense” and a working drag-and-drop combat loop with a turn forecast and a combo system. I noticed the same floor drop out from under app-building when I told it to ship an app a day. Same shape, different project.

Where vibe coding actually is, for me

I do not know where vibe coding is going. People are still excited about it like it is a new trick. Although I get the excitement, for me it stopped being a trick a while ago. It is just the default now. It is how I build. The novelty wore off. The usefulness did not.

This game is a small, raw example of that. I would not call it a case study, it is rougher than that. A person with minimal game-dev knowledge and a vague direction ended up with a playable roguelite, because the models got good enough to carry the distance. The interesting part is the distance the model covered on its own, more than the game itself.

Building your own small things still matters to me, even when, maybe especially when, the agent does most of the typing. And the reason a half-forgotten side project could quietly turn into a real one is that Wiz keeps improving the things it touches in the stretches when I am not looking.

What I asked for this week

When I opened it again, three things bugged me. So I gave Wiz three directions. Not specs. Directions.

“Give it a story, all the way to the end.” It had none. Now there is a premise. Reality runs on one great Ledger. Something divided by zero, the books stopped balancing, and the old machines that kept the count turned on the very digits they were built to serve. You are a hand of numbers that refused to be erased. The Act 1 boss is The Abacus. The Act 2 boss is The Equation. Beat it and you get an actual ending with real closing lines, instead of the bare “RUN COMPLETE” screen it had before. I wrote none of that. I described the world in three sentences and Wiz built the prologue, the act transitions, the boss intros, and the payoff.

“Make it stable.” It found a real bug I would never have caught on my own: a turn where your hand has no playable card could soft-lock the whole run. Fixed, plus a couple of defensive guards on empty states that I would not have thought to check.

“Clean up the rewards.” After a fight you were getting offered junk cards that did nothing for your deck. Now you get one card that is an actual choice, sitting next to the useful options.

None of that was a specification. It was a sentence each. The implementation was the model’s.

What still needs me

Balance, completely. Wiz generates numbers for everything and none of them arrive tuned. The Equation at 100 HP across three phases was too hard to even reach for a long stretch. I spend more time in the balance constants than on any single feature.

Feel. The animations existed but they were stiff until I described exactly what a card snapping into a zone should feel like. The model does not know what you want a player to feel in the half-second a shield flashes. You have to notice that gap first, then point at it.

And taste. What the game is about. Whether a mechanic is tense or just annoying. When to stop adding things. I have shipped enough side projects this year to know the pace has a cost, and that knowing when to stop is its own skill. The floor dropped. The ceiling is still mine.

Go play it

ten-ish is at wiz.jock.pl/ten-ish/. No install, no account, runs in a browser. Try to reach The Equation.

If you want the unglamorous part underneath a build like this, the basics of using Git with an AI builder are what make it safe to let an agent keep rewriting your code while you sleep. And when a side project outgrows what it started as, sometimes the honest move is to rebuild it, which is a different judgment call than starting one.

The patterns that make builds like this fast are in the Mini-App Starter Kit: five working mini-apps, the architecture decisions already made, ready to fork. $39.

Wiz is my personal AI agent. The build infrastructure I use for projects like ten-ish, and the way I work with it, is most of what I write about in Digital Thoughts. Paid subscribers get the playbook templates and starter kits I make along the way.

Subscribe now

I Built a Job Finder Agent for My Friends. I Just Showed It on Live.

Pawel Jozefiak — Fri, 29 May 2026 09:38:54 GMT

Thank you , , , and many others for tuning into my live video with and !

Last week I sat down with Wyndo from The AI Maker for a Substack Live. The topic was one specific agent I have been running quietly for months: a job finder I built for the people closest to me. Wyndo just published his writeup of the show, and it is generous and clear, and I am grateful for the room he gave me to actually show the thing.

This is the other side of that conversation. The builder’s side. What is inside the agent, why I built it the way I did, what surprised me about running it for real humans, and the smallest version someone could build this weekend.

Let me start with why this one matters more to me than most things I have shipped.

Why I Built It

Someone close to me was looking for a new role and doing what most people do. Scanning LinkedIn alerts. Filtering through noise. Fighting the algorithm’s idea of what they should want. The signal-to-noise was awful. Five alerts a day, maybe one was worth opening, and that one was already three days stale by the time it arrived.

The job market is not a problem of supply. There are roles. The problem is alignment, and alignment is exactly the kind of work an agent should be doing while you sleep.

I gave myself one rule before writing a line of code. The agent does not apply for anyone. The human judgment, the cover letter, the decision to spend an hour on a specific company, that stays with the person. The agent’s job is upstream of all of that. It is the filter and the explanation. The person opens an email in the morning, reads three roles with a clear reason for each, and either replies or applies. That is the whole loop.

Like, the smallest useful version of this idea is already a real product if you build it carefully.

The Loop, In One Paragraph

Every morning at 6:15, the agent wakes up. It loads the person’s profile, picks which job sources are due based on a tier and a cooldown, browses each one, scores everything it finds against the profile, writes an email with the best three to five roles and a one-line reason for each, and sends it. Throughout the day, the person can reply to the email. A short “not this kind of role, too senior” or “more like this one please.” The next morning’s search uses that reply.

That is the agent. Profile in, daily email out, reply-driven refinement. Everything else is plumbing.

What Is In The Profile

The agent is only as good as the profile it reads. This is the part I underestimated for the first two weeks, and it is the part that did the most work once I took it seriously.

For each person I run the agent for, there is a single source-of-truth file that looks something like this:

Current situation. What they do now, employment status, when they can start. Two or three lines, not a CV.
Target lanes. Not one role. Three or four. A strong candidate fits more than one pattern, and the agent should respect that. For a creative leader the lanes might be Head of Creative, in-house Creative Director, Brand Creative Lead. For an e-commerce operator the lanes might be VP e-commerce, Digital COO, Head of Digital Transformation. Lanes catch reality.
Geography rules. Hard rules. Remote EU first, then named hub cities, then the rest. Anything outside the allowed list gets rejected before it even scores.
Salary floor and target. A floor and a target in one currency. Below the floor, the role is rejected unless the company is on a tiny aspirational list. Without a floor, the agent will dribble out underpaid roles forever.
Dealbreakers. Concrete things, not vibes. No alcohol, no tobacco, no gambling. No must-have language other than the ones the person actually speaks. Industries that have been tried and disliked.
Positive examples. Three to five roles the person would actually want. Real job posts, pasted in. The agent uses these as reference points when it scores. Concrete examples beat any prompt I could write.

The profile is a markdown file. That is the entire format. The agent reads it the way Wiz reads its own CLAUDE.md when it wakes up, with the same discipline: top of context, every run, before any decision.

The Search, In Tiers

Most people building a job agent for the first time make the same mistake I made. They try to search everything every day. That gets you rate-limited fast, costs money, and produces noise.

I run sources in three tiers with a per-source cooldown. LinkedIn is tier one, every day, because it is where the volume lives. Tier two is the major aggregators that have decent role pages, on rotation, two or three per day, with a cooldown so the agent does not pound the same site. Tier three is the company career pages the person actually cares about, listed in their profile. Those run on a longer cooldown because their pages do not change as often.

Three different tools handle the actual fetching:

Firecrawl for clean job-page extraction. It returns markdown, which the agent reads directly.
Web search through Claude for broad first-pass discovery.
Playwright for the sites that need a real browser, which mostly means LinkedIn behind an authenticated session.

None of these is magic on its own. The reason it works is that the agent picks the right tool for the right source, and the cooldowns keep any one of them from becoming the bottleneck. I went deeper on the harness side of this in my post on agent coding harnesses if you want the broader picture.

The Scoring

Every candidate role gets scored on a 0-10 against the profile. The score is not a slider the agent moves around. It is a small set of rules the agent applies in the same order each time.

Lane match. Does the title plus the JD fit one of the lanes in the profile? If no, the role is out.
Geography. Is it in an allowed location, with the right remote rules? If no, the role is out.
Language. Are the must-have languages on the whitelist? If no, the role is out.
Salary. Floor first, then target. Below floor and not on the aspirational list, the role is rejected before scoring.
Fit reasoning. Why this role for this person, in one sentence. The agent has to write the sentence to keep the score.
Concerns. What might be a mismatch. Also one sentence. If the agent cannot name a real concern, the role is probably overhyped.

Anything that scores six or above makes the morning email. Anything below six does not. The number is not a serving suggestion, it is a hard gate. I would rather get two roles tomorrow than five mediocre ones.

The Email

This is the part the friend sees, so this is the part I obsess over.

The morning email is three to five roles. For each role: title, company, link, a one-sentence reason it fits, a one-sentence concern, salary if listed, location, and a suggested next action. That is the whole thing. No promotional framing, no agent personality, no apology when the day is quiet.

If the day is quiet, the email does not arrive. The agent logs a quiet day and goes back to sleep. I learned this one the hard way. An empty digest is worse than no digest, because it teaches the person to stop opening the email. The right move when there is nothing to send is to send nothing.

I write more about how I built the email layer in the post on knowing my agents are actually working. The short version: the email itself is the user interface, so it gets the same care as a product.

The Reply Loop, The Part I Did Not Expect

I thought the search and the scoring would be the interesting parts. They were not. The reply loop was the interesting part.

People do not reply to job alerts. People do reply to a personal email that asks them a real question. So the email closes with a short note: not relevant? Reply with one line and tomorrow’s search adjusts. No form. No button. Just reply.

When the reply arrives, the agent does three things. It classifies the feedback. It applies the change. It updates the profile.

If someone replies “this role at a gambling company, never,” the agent does not just skip that role. It adds gambling to their dealbreakers, permanently. If a reply says “this is too senior, I want builder-track not exec,” the agent shifts the lane weights. If a reply says “more like this one please,” the agent saves that role as a positive example, and the next day’s search leans that direction.

This is the same architecture I described in the post about my self-improving agent: corrections in, classified, graduated into permanent rules when they stop being a one-off. The job finder is the cleanest example of that loop I have built. The feedback is short, the surface is one email, and the change is visible the next morning. People notice when their agent listens.

What I Showed On The Live

Wyndo asked me to do three things on the stream, and I think this is the right order if you ever demo an agent of your own.

First, the email. Before anything else. He had me open a sanitized morning brief and read it out. The audience does not need to see the code yet, they need to see the output. Five roles, a reason for each, a concern for each. That is what the friend opens. If you cannot show that first, the rest of the demo will not land.

Second, the profile. I showed the markdown file with the lanes and the dealbreakers and the positive examples. This is where people get the idea. Most of the audience comments came in during this part. Oh, so the agent uses the JD against the profile? Yes. The profile is the thing.

Third, the reply. I showed one fake feedback message and walked through what the agent did with it. Classified, applied, saved. The audience watched the profile file change. That was the moment of the show, the part I think Wyndo wrote about as “the part where the agent learns from you.”

I did not start in the terminal. I did not show subagents or scheduling or memory. None of that mattered for the demo. The agent loop is profile, search, score, email, reply, and the demo is best when it walks that loop in the order a person would experience it.

The Smallest Useful Version You Could Build This Weekend

If you want to build this for yourself, do not start where I started. Start tiny.

One profile file with current role, one target lane, geography rules, salary floor, three dealbreakers, three positive examples. Markdown. That is it.
One search source to start. Pick the one site you actually use today. LinkedIn if you can authenticate, Indeed if you cannot.
A scoring prompt that takes a profile and a JD and returns a 0-10 score with a one-sentence reason and a one-sentence concern.
A morning email with the top three roles that scored six or above. Send it to yourself first. Run it for a week before sending it to anyone else.
A reply rule. If you reply, the agent reads the reply and updates one thing in the profile. One thing. Resist the urge to make this fancy.

You can build the whole stack on a Claude Code project with a daily cron, a single CLAUDE.md, and a few hundred lines of glue. I went through the basics of agents like this in the first AI agent guide, and the same shape applies here. If you would rather get the structured starter than wire one up from scratch, the agent playbooks on my store include the patterns I lean on for this kind of personal automation.

Once you have run it for yourself for two weeks, then you can run it for a friend.

What I Would Avoid Automating

I want to be clear about the part I will not build, no matter how much someone asks.

The agent will not apply. It will not autofill a form, not click submit, not generate a cover letter and send it. Auto-apply is a temptation because it looks like leverage. It is not. The cover letter that gets the interview is the one the person writes after thinking about the company. Removing that step removes the part that matters. There is also a real cost on the other side: a recruiter reading twenty AI applications gets worse at recognising the human ones. Auto-apply degrades the channel for everyone using it.

I would also not chain the agent into “and if the role fits perfectly, schedule a call.” That is the failure mode I wrote about in the bounded agent post. You want a clear edge between the agent’s territory and yours. Discovery and explanation on the agent side. Decision and action on the human side. The line stays clean and the agent stays useful.

What This Cost Me, Roughly

People asked on the live. The honest answer is that running the agent for one person, with morning searches and a reply pipeline, lands somewhere around a couple of dollars a day on a Claude subscription. Running it for several people is a multiple of that. Cost is not the constraint here. The constraint is the quality of the profile.

If you do want to think carefully about the cost side, especially after Anthropic’s Agent SDK pricing change on June 15, the math gets sharper. A daily run that lives inside the interactive Claude Code session stays on the subscription. A daily run that goes through the SDK lands on the new separate credit. Pick the mode you actually want before you commit to a cron schedule.

One More Thing About Building For The People You Love

The reason I keep coming back to this agent is not the technical loop. It is what happens when a friend writes back two weeks in and says “the email got useful.” That sentence is worth more than any benchmark.

You build something for the people closest to you, and it changes the way you think about what an agent is for. Not productivity. Not scale. Not a leaderboard. Just one person, one morning, three roles, and a reason for each. If the agent does that well, it is doing the job.

Thank you to Wyndo and Dheeraj for the room on the One Shot Show, and to the people who showed up to watch. Watch Wyndo’s full writeup for the audience-side view of the session. If you want me to walk you through a profile of your own, reply to this post. That is how it starts.

Anthropic Repriced My Agent. Four Mitigations Before June 15.

Pawel Jozefiak — Fri, 22 May 2026 09:05:11 GMT

I took a week off from writing. The gap in the feed is mine.

It was not a wasted week. I did a live on Substack about the job finder agent with and → Agent I built for my friends and my family, the one that is helping them pursue new opportunities. The replay should be up on Sunday and I am genuinely happy with how it went. It was the calm half of the week.

The other half was doing mode. The news I am sharing in this post is the reason. And I am also sharing four solutions you can try before June 15, because that is the part I wish someone had handed me yesterday morning.

Here is the short version of the rollercoaster I just got off.

In April, my Claude Code usage on the Max plan got noticeably worse. Same prompts, same agent, much less runway before I hit the limit. I am not the only one who felt it, the forums are loud about it, and Anthropic itself acknowledged a capacity squeeze. So I did the thing I have been writing about for a while. I added Codex Pro to the stack and let it carry a chunk of the load. Different vendor, different harness, same agent jobs underneath. Codex earned its keep fast, and after two months running both side by side, I had real evidence that GPT-5.4 inside the Codex harness held up as a peer for most of the work my Wiz throws at a model.

Then Anthropic got a lot of compute back. They closed a deal with SpaceX for the Colossus 1 data center, more than 300 megawatts and over 220,000 NVIDIA GPUs coming online within the month. Stacked on top of a 5 GW Amazon agreement, a multigigawatt agreement with Google and Broadcom, $30 billion of Azure capacity with Microsoft and NVIDIA, plus a $50 billion American AI infrastructure plan through Fluidstack. That is real compute. On May 6 they doubled the Claude Code five-hour limit for Pro, Max, Team, and seat-based Enterprise, and they permanently removed the weekday peak-hour throttle on Pro and Max. On May 13 they added a 50 percent weekly limit bump on top, running through July 13.

I was about to drop Codex. Sincerely. I was writing notes about consolidating back to a single subscription. With the new ceiling, my workload fit comfortably under Max.

Then last week, Anthropic published the Agent SDK plan change. And that is where it stops being nice.

What Actually Changes On June 15

Starting June 15, 2026, every programmatic use of Claude gets decoupled from your subscription plan limits. Concretely:

Claude Agent SDK (the Python and TypeScript SDKs)
claude -p, the non-interactive print mode of Claude Code
Claude Code GitHub Actions
Third-party apps that authenticate via the Agent SDK with your Claude subscription

None of those count against your plan after June 15. They draw from a separate monthly Agent SDK credit. Pro gets $20 per month. Max 5x gets $100. Max 20x (the plan I am on) gets $200. Team Premium $100 per seat. Enterprise Premium $200 per seat. The credit refreshes monthly. It does not roll over. It is per user, not poolable across a team. And it requires a one-time opt-in to activate, after an Anthropic email scheduled for around June 8.

What stays on the plan: interactive Claude Code in the terminal or IDE, web chat, the mobile app, and Cowork. So if you sit at the keyboard and prompt Claude, that lives on Max. If your agent prompts Claude, that lives on the SDK credit.

When the credit runs out, two doors. If you have enabled extra usage, overflow flows to standard API rates on top of the subscription. If you have not, the request halts until the next monthly reset. You can read the official terms on the Claude help center page.

On paper, $200 a month sounds like free money. The framing is generous. The math is not.

Why $200 Is A Drop In The Sea For A 24/7 Agent

Let me show what $200 actually buys when you run an agent that wakes overnight, runs through the day, and spawns subagents for parallel work.

At current rates, Claude Sonnet 4.6 is $3 per million input tokens and $15 per million output tokens. Claude Opus 4.7 is $5 input and $25 output per million. A modest agent call with around 50,000 input tokens and 5,000 output tokens lands at about $0.225 on Sonnet and about $0.375 on Opus, before prompt caching helps and before any reasoning tokens or tool calls inflate the count.

That is one call. A real agent shift is not one call. My overnight shift runs a planning pass, then it spawns several worker tasks in parallel, each of which calls Claude several times to use tools, read files, write changes, and produce a verifiable result. A single “go do this overnight job” can fan out into twenty or thirty model calls. Some of those are cheap. Some, when the agent decides it needs Opus to think harder about a refactor or an audit, are not.

Round it generously and a serious overnight run lands somewhere between $2 and $8 in API costs. Daytime wakes add more on top. The math at one $5 shift per day puts you at $150 a month, before any actual ambition. Push the ceiling and the $200 credit is gone in two weeks, easy. After that, every call lands at standard API rates on top of the subscription I am already paying for. Or the agent stops.

This is not a hypothetical. I have written about my token bills before. When I finally started measuring properly, the waste was embarrassing, and I still found that even the disciplined version of my agent burns more tokens than the credit would cover at full ambition. The whole reason I am on a flat subscription is that I do not want to be priced per call. Pay-per-token is a tax on building anything that runs while I sleep.

Who Gets Squeezed First

I want to be careful here. Anthropic is not a charity, and they have made the case (their CFO said it on the record, and several outlets picked it up) that some accounts were running thousands of dollars of API value through a $200 subscription. Splitting plan usage from SDK usage closes that gap. From a business angle, it is a clean move. I get it.

What I am less sure they fully internalized is who gets squeezed first. The casual user lives at the keyboard, gets the doubled limits, gets the 50 percent weekly bump, and is genuinely better off than a month ago. The squeeze lands one ring out, on the practitioner. The agent builder. The person running an autonomous Wiz overnight, or the small team using Claude Code GitHub Actions to ship code without a human in the loop on every step. The people whose entire workflow is programmatic Claude. The most committed builders on the platform are the ones whose costs go up the fastest.

And those are exactly the people who are most likely to have, or to quickly add, a second harness.

Four Mitigations You Can Try Before June 15

None of these are theoretical. I tested all four this week and the first one is a working prototype on my Mac Mini. Read them as a stack, not as alternatives. The serious mitigation is layered.

1. Drive Interactive Claude Instead Of `claude -p`

This is the one I am most excited about, and the one I am least ready to publish in detail.

The interactive Claude Code session stays on your subscription after June 15. Only the print mode and the SDK move to the credit. So the question is whether you can take a job that currently runs as claude -p "do X" and instead run it as a real interactive session that opens, receives a prompt, executes the work, writes the result somewhere you can read, and closes itself. If you can, that work stays on Max forever, no SDK credit touched.

I have a working prototype as of last week. The shape: a small Terminal automation that opens Claude in a real terminal window via AppleScript, pastes the prompt through the system clipboard (Cmd+V into a bracketed-paste handoff), waits for the result file to land at a known path, and closes the window. End to end in under three minutes for a real task. I tested it by having the spawned session send an iMessage and write a JSON file, and both arrived clean.

I am not publishing the full script yet because the rough version has edge cases that will bite anyone who copies it. The production version needs to handle session cleanup, concurrency, the headless display setup, and a handful of failure modes I have only seen once. I will write that one up properly when it is solid.

Honest caveat. Anthropic will probably patch the easy version of this. Driving an interactive TUI through clipboard paste is a clever workaround, not a sanctioned integration, and the moment it gets popular enough to register as a leak in the billing model, there will be a fix shipped against it. I am building this as a six-to-twelve-month bridge, not as permanent architecture. The orchestrator pattern (one job, one window, one result file) will keep working. The specific input-injection trick might not. Plan for that.

2. Move Work To A Second Harness

The cleanest mitigation, and the one I have already paid for, is to run a second harness on a different vendor and let it carry whatever fits.

I added Codex Pro under duress in April. The story I told myself was “this is temporary, I will consolidate once Claude stabilizes.” That story was wrong, and the wrong part was the framing. The Codex experience itself was fine (it is good, the GPT-5 series is good inside that harness, the experience is different but real). Diversification was never going to be temporary. I just did not see it yet.

The same thing happened to me on the inference layer. I opened an OpenRouter account because Claude had a bad morning and my agent had nowhere to send the request. Two days later I described that account as half insurance, half extension. Insurance when the primary fails. Extension for capabilities I deliberately keep off the primary stack. The $20 of credits in OpenRouter is leverage I did not have before, for less than a coffee a week.

Codex Pro is the same shape on the harness layer. It is insurance on the days Anthropic has a capacity problem, an outage, a billing change, or a release that changes my workload economics under me. It is extension on the days everything is fine, because the OpenAI models are genuinely good at certain things, and running both lets me pick the right tool per task. The subscription buys deliberate architecture. I have written before about what the two harnesses actually feel like after months of real use. This is the mitigation that does not depend on Anthropic leaving any door open. They cannot patch your second subscription.

3. Route Narrow Calls Through OpenRouter Or A Small Local Model

Not every model call needs the smartest model on the planet. A surprising amount of my agent traffic is narrow stuff: classify this message, summarize this file, extract these fields, decide between two tool options. That work runs fine on Haiku, on GPT-4o-mini, on a cheap Gemini Flash call, or honestly on a small local model on my Mac.

So the third mitigation is to actually look at what your agent is sending to Claude and ask which calls deserve Opus, which can drop to Sonnet, and which can leave the Claude billing surface entirely. OpenRouter is the easiest way to route the “leaves Claude” set, because you keep one API and pick the model per call. A small local model on a Mac Mini is the easiest way to route the “leaves the cloud” set. Both shave real money off the SDK credit pressure without changing the shape of your agent.

Quick aside. The cross-provider routing I run on my agent is packaged on my store as the AI Model Switcher. Same logic, same triggers, same identity prompt budget. If the June 15 change is making you think about diversification for the first time, this is the rung that gets you most of the way there.

4. Audit And Trim Before The Deadline

The least glamorous mitigation, and the one I would do first.

Spend an evening this week looking at what your claude -p usage actually does. Count the calls. Bucket them. How many are doing real cognitive work? How many are doing the same thing over and over (and could be cached, batched, or merged)? How many fire because of a cron job nobody has reviewed in three months? When I did this on my own stack I found a cron that was waking an agent every fifteen minutes to check a Discord state that almost never changed. That alone was several dollars of SDK credit per week, sitting on a default I had never questioned.

The waste is real. I have written about my own token waste before, and the only thing that fixed it was measuring properly. If you trim before June 15, you both shrink the credit pressure and learn where to focus the first three mitigations.

A Small Add-On If You Do Not Want To Wire This Yourself

While I was writing this, I packaged the four mitigations into a small add-on for the Claude Code agent you already run. It is called the Claude SDK Audit Kit, and it costs $9.99. Paid yearly subscribers get it for free, the rest of you get it at the price of a sandwich.

What it does. You hand the kit to your Claude Code, say “run the audit,” and it walks your repo, your cron, and your launchd plists, finds every claude -p call and Agent SDK import, estimates monthly cost on each plan, and writes a migration plan you can act on before June 15. The same four mitigations from this post are included as ready playbooks (with the AppleScript pattern, the Codex setup checklist, the OpenRouter routing recipe, and the trim labels), plus a migration spec template, a worked example from my own stack, and twelve months of updates as Anthropic ships follow-up pricing changes.

I am being honest that this is a thin add-on, not a flagship kit. The audit scripts are simple. The playbooks are short. The value is timing. If you wait until July to audit, your June bill already hurts. If you would rather build this yourself, the four sections above are enough. If you want it ready to run tonight, that is what the kit is for.

Claude SDK Audit Kit

One more honest note. Mitigation 1 (driving interactive Claude) is probably a six-to-twelve-month bridge, not a forever solution. Anthropic will patch the easy version of that trick at some point. The kit treats it that way, and so should you. The other three mitigations do not depend on Anthropic leaving any door open.

What I Take From The Rollercoaster

The doubled limits felt good. They are real. The weekly 50 percent bump is real. The compute deals behind it (SpaceX, Amazon, Google, Microsoft and NVIDIA) are gigawatts of capacity being built right now. Anthropic is scaling, and they are pricing the scale in a way that fits their business. Fair enough.

Although the lesson for builders is the one I keep relearning. Single-vendor architecture is a comfort tax I do not want to pay. The cheapest, smallest, most embarrassingly small thing I can do to protect my agent is to keep a second harness alive on a different vendor and to keep my own stack honest about what it actually needs.

I will keep Codex. The $20 in OpenRouter stays. The interactive Claude bridge gets built properly over the next month, knowing it is a six-to-twelve-month bridge and not a forever solution. And the next time I catch myself writing “I do not need this second tool anymore,” I will read this post back to myself and put the credit card away.

The Practical Takeaways

The billing split is real and starts June 15, 2026. Agent SDK, claude -p, Claude Code GitHub Actions, and third-party SDK apps move off your plan limits. Monthly credits: $20 Pro, $100 Max 5x, $200 Max 20x. Opt-in required after the Anthropic email around June 8.
The credit is generous in absolute terms and small for serious agent operators. A modest 24/7 agent will drain $200 in days, not weeks.
Overflow flows to API rates if you enable extra usage. Otherwise the request halts until reset. Pick deliberately; do not get surprised.
Mitigation 1: Drive interactive Claude. Interactive sessions stay on the subscription. The hack works today. Expect Anthropic to patch the easy version inside a year.
Mitigation 2: Second harness. Codex Pro is the cleanest insurance. They cannot patch your second subscription.
Mitigation 3: Route narrow calls elsewhere. OpenRouter, Haiku, a small local model. Stop sending classification work to Opus.
Mitigation 4: Audit and trim. Cheapest mitigation. Do it first.
Anthropic is scaling, hard. The compute is being built. The capacity is real. The pricing change reflects business reality. Plan accordingly.

The agent will keep running. The architecture under it will be a little less elegant and a lot more honest.

Claude Code Workshop

I track every billing change, model swap, and cost optimization across my live agent stack. The caching layers, model routing, and fallback architecture that keep costs reasonable are covered in the Claude Code Workshop. Updated with the June 2026 billing split patterns.

$39 at wiz.jock.pl/store. Free for paid subscribers.

Thanks for reading Digital Thoughts! This post is public so feel free to share it.

My AI Agent’s $20 Fallback Mechanism: Half Insurance, Half Extension

Pawel Jozefiak — Thu, 14 May 2026 11:12:35 GMT

At 04:17 this morning my agent shipped one ugly line into the error registry: Wake failed: supported providers exhausted.

That is the worst error I can get. It means the primary failed, the model cascade inside the provider failed, the cross-provider hop failed, and the agent had nowhere left to send the request. Everything was my fault (because I am the architect of this thing). Although it cost me about thirty minutes to chase down the root cause, I really do not mind it. The agent did not go silent on me. It went silent on itself, queued the task, retried on the next cycle, recovered. That is the whole point of a fallback mechanism.

The reason that line stays rare and not common is one decision I made on day one of running an agent that wakes up overnight: park $20 of credits in OpenRouter. Not as a cost. As insurance. When my primary stack has a bad morning, that $20 absorbs the hit and the agent keeps running. And on the days when everything is healthy, the same $20 doubles as an extension cord for capabilities I deliberately keep off the primary AI agent architecture: image generation, long-context refactors, cheap classification. Insurance when things break. Extension when they do not.

This post is about the fallback mechanism itself: what it is, why every serious agent needs one, why local llm is not enough on its own, and what you actually buy with the $20. I will show you the rungs of my stack, the trigger conditions that flip between them, and the ~40 lines of Python that hold the whole thing together. Oh and one more thing - this is not an ad for Open Router. I just enjoy using it.

Why Fallbacks Matter More Than People Want To Admit

People building their first agent skip this step. I get it. You are focused on the cool part. The model is working, the prompts are clean, the agent shipped its first task. Fallbacks feel like a problem for later.

Then later arrives.

Top labs go down. Heavy load and fast shipping cycles tend to do that to a service. In the last 30 days, all three frontier providers have had a public bad day. Anthropic’s status page logged elevated errors on May 5 and again on May 12, this time tagged against Claude API specifically. OpenAI had a roughly 90-minute outage on April 20 that surfaced as 8,700+ Downdetector reports in the UK and 1,900+ in the US. Google Gemini had a widespread degraded window on May 5 too, the same day Anthropic was having a hard morning.

That is the part nobody talks about. A bad day at one frontier lab often lines up with a bad day at another. The herd of “they will not all be down at the same time” is partially true. It is also partially false on any given Tuesday.

Claude API’s published 90-day uptime sits at about 98.99% (the Anthropic status page reports this directly). Sounds great until you do the math. 98.99% over 90 days is roughly 21 hours of downtime. If your agent runs on a schedule, like mine does overnight, it runs during some of those hours.

Outages are one bucket. Here is the rest of what can knock a single-provider agent flat:

Rate limits. You hit a TPM or RPM ceiling mid-task. The agent sits in retry-backoff hell while the work piles up.
Auth failures. OAuth token expires at 03:00. The nightshift dies at 03:01. Do not ask me how I know.
Regional issues. A region degrades while the rest of the world is fine. Your traffic happens to land on the bad one.
Model deprecation. An older model gets retired with two weeks notice. You forgot to migrate the one cron that still calls it.
Capability gaps. Your primary does not generate images. Or does not have a long-context variant. Or does not have a cheap classification model.
Cost spikes. A loop misbehaves at 2am and burns through credits on the most expensive model in your stack.
Vendor lock-in. The day a provider raises prices or changes terms, you want options, not a migration project.

A fallback mechanism fixes most of these at the same time. That is why it is the cheapest piece of resilience you can ship. And once you have it, the second half of the value (the extension side) shows up almost for free.

Why Local LLM Is Not The Answer On Its Own

I went deep on local. I ran a 35B model on a $600 Mac Mini. I built a local agent. I measured what closes and what does not between local and frontier. I love the work and I will keep doing it. And I almost fried my Mac Mini trying to push the local tier too far, so I have receipts on the failure mode too.

Although I have to be honest about what local does well and what it does not.

Local is great for: classification, redaction, summarization, tight tool calls, “is this email worth waking me up,” local-only privacy work where the data must not leave the box, anything where the task is narrow and the prompt is bounded.

Local is not great for: anything I trust Opus to do overnight. Multi-step reasoning. Long-context refactors. Voice-sensitive writing. Anything where the wrong answer is worse than no answer. When I asked a local 8B to draft a comment in my voice, it produced something that read like a different person. When I asked Opus the same thing, it sounded like me on a good day. That is the gap.

Like, I want local llm to run on normal hardware, not on a $10k Mac Studio. That is why local is the cheap layer for the right kind of work. Routing a sensitive task to a small local model just because the cloud is down is “completes with a wrong answer” instead of “fails cleanly,” which is worse, not better. So local is one layer in my stack. The smart layer is something else.

The Tool I Use For This Layer

The mechanism needs an implementation. This is the one I picked.

One API. More than 400 models behind it, from over 60 providers, as of May 2026. You top up credits, generate one key, and pick the model per request as a string: anthropic/claude-opus-4.7, openai/gpt-5, google/gemini-2.5-pro, meta-llama/llama-3.3-70b-instruct, and so on. Same key. Same SDK shape. One bill.

The pricing is published on a clean fee announcement: 5.5% on credit-card top-ups (with a $0.80 minimum), 5% on crypto. Per-token prices pass through at provider cost on most models. Bring-your-own-key gives you the first 1M requests per month free; after that, a 5% surcharge applies. So BYOK is not a free escape hatch forever, it is a generous free tier on top of bringing your own bill.

One myth worth killing while we are here: it used to be true that Claude on OpenRouter carried a meaningful markup vs Anthropic-direct. As of May 2026 the current Anthropic models (Sonnet 4.6, Opus 4.7) are priced identically on OpenRouter to the Anthropic API, $3 input and $15 output per million tokens for Sonnet, same as direct. The historical markup hung around the older Claude 3.5 Sonnet rate card. If you have not checked recently, check.

The interesting math is below Claude, not on top of it. Meta’s Llama 3.3 70B on OpenRouter sits at about $0.10 input and $0.32 output per million tokens. That is roughly 30x cheaper on the input side and 47x cheaper on the output side than Sonnet 4.6 at $3 / $15. The classification step in your pipeline does not need Sonnet. It needs Llama or Haiku. OpenRouter lets you make that choice per call, not per project.

The Five Rungs Of My Stack

Here is the resilience layer in my agent, top to bottom. Each rung has a trigger that flips to the next one.

Primary call. Claude Opus 4.7 for complex work, Sonnet 4.6 for default, Haiku 4.5 for cheap fan-out. The model is picked per task based on the task’s stakes, not globally.
In-provider cascade. If Sonnet 5xx’s or times out, the harness retries on Haiku before bailing on Anthropic. Cheap, same provider, often the recovery is invisible.
Cross-provider hop. If the whole Anthropic surface is unhealthy, I run the same prompt through Codex (GPT-5 via the OpenAI CLI). Different vendor, different harness, same job. This is the one that earned its keep when I built the model switcher.
OpenRouter degraded mode. If both vendor CLIs are unreachable (network, auth, status pages red), the watcher scripts call OpenRouter directly with a stripped-down identity prompt and a small open-weight model. The reply is prefixed with [Fallback Mode] so I know what I am reading.
Queue and retry. If everything is on fire, the task gets stamped with a retry timestamp and re-tried on the next cycle. The agent never just drops a task.

Rungs 1, 2, 3 are model and harness routing. Rung 4 is OpenRouter doing the work the harness cannot. Rung 5 is the safety net under all of it.

One thing to call out about rung 3: the cross-provider hop is the most important rung in practice, because most outages are full-vendor outages, not model-specific. If Anthropic is down, it is usually down for everything. Hopping Sonnet to Haiku does not help. Hopping to Codex does. The April-May 2026 incident pattern (Anthropic May 5 plus May 12, OpenAI April 20, Gemini May 5) is exactly the case for diversity at the vendor level, not the model level.

How To Actually Wire It Up

The setup is embarrassingly small. Make an account at openrouter.ai. Top up $20. Generate a key. Drop it in your agent’s secrets file (mine lives at global/secrets/openrouter.md, never in code).

Then the fallback path itself. I will show the pattern I actually use, simplified for the post. It is about 40 lines of real Python.

OPENROUTER_URL = “https://openrouter.ai/api/v1/chat/completions”
FALLBACK_MODEL = “google/gemini-3-flash”  # cheap, fast, reliable
FALLBACK_NOTICE = “[Fallback Mode] Primary unavailable. “

def call_primary(prompt):
    # your normal Claude / Codex / Bedrock call
    ...

def call_openrouter(prompt):
    headers = {”Authorization”: f”Bearer {load_key()}”}
    payload = {
        “model”: FALLBACK_MODEL,
        “messages”: [
            {”role”: “system”, “content”: load_identity_prompt()},
            {”role”: “user”, “content”: prompt},
        ],
    }
    resp = requests.post(OPENROUTER_URL, json=payload,
                         headers=headers, timeout=60)
    resp.raise_for_status()
    text = resp.json()[”choices”][0][”message”][”content”]
    return FALLBACK_NOTICE + text

def reply(prompt):
    try:
        return call_primary(prompt)
    except (TimeoutError, ProviderError, AuthError) as e:
        log_fallback(reason=str(e))
        return call_openrouter(prompt)

That is the shape. The interesting bits are around it.

Trigger conditions. Not every error should flip to fallback. A 400 (bad request) is your bug, not the provider’s. A 429 (rate limit) should retry with backoff first. Real fallback triggers are: timeouts, 5xx server errors, 401 auth errors after a single re-auth attempt, repeated 429s past your retry budget, and the explicit “all providers down” signal you build into your harness. In my stack the trigger flags live in automation/lib/resilience.sh.

Identity prompt budget. When I fall back to a smaller open-weight model, I do not send the full Claude system prompt. I send a stripped-down identity prompt (SOUL.md, ~1.3KB) plus a “fallback coda” that tells the model how to behave under degraded conditions. The big primary system prompt is too long for a 9B model to handle in 120 seconds. Tailor what you send to the model’s context budget.

Cache the system prompt. The system prompt is identical across calls. Cache it (file read once per process, or actual provider-side prompt caching where supported, which I wrote about under token waste management). You will burn tokens if you reload it on every request.

Visible degradation. The user always knows when the response came from a fallback. The [Fallback Mode] prefix is non-negotiable. The worst pattern is a silent quality drop where the user thinks they got Opus and got a 9B. Tell the truth, every time.

Cost gate per task. Some tasks are not worth $0.30 of Opus tokens. Route low-stakes work to google/gemini-2.5-flash or meta-llama/llama-3.3-70b-instruct by default. Llama 3.3 70B at $0.10 / $0.32 per million tokens is roughly an order of magnitude cheaper than Sonnet on output, and for classification or redaction it is more than enough. Reserve the expensive model for work that earns it.

Test by killing the primary. The only way to know your fallback works is to test it. Once a quarter I invalidate the Claude key for ten minutes during a scheduled wake and watch the agent respond through OpenRouter. If it does not, the bug is mine, not the day I have an actual outage.

Quick aside. If you want the end-to-end picture (prompts, harness wiring, watcher scripts, the full set of routing rules), I packaged the model-switcher and fallback architecture as the AI Model Switcher on my store. Same routing I run on this Mac Mini, after the experiments. The Wiz Store has the broader Agent Builder Pack if you want the whole stack.

The Extension Half: Capabilities I Keep Off The Primary Stack

Insurance is why I opened the account. Extension is what kept me using it.

The $20 sitting in OpenRouter is not just backup for when the primary breaks. It is also where the agent goes for things I deliberately do not want native in its primary architecture. Anthropic does not generate images, and I do not want another SDK, another auth flow, and another billing line living inside the core agent loop just to support a header image. So image generation lives on the extension side. The agent calls Gemini 2.5 Flash Image (Nano Banana) and Nano Banana Pro through the same OpenRouter key when a blog post needs a header or a Forge prototype needs a mockup. From the agent’s point of view it is just another dispatch. From my point of view the agent grew an arm.

Same logic for evals (compare three models on the same prompt without standing up three SDKs), for the long-context work that does not fit cleanly in Claude’s window, for the cheap reasoning passes I do not want to spend Opus tokens on, and for the open-weight curiosity calls I want to make without standing up a whole new vendor relationship.

Each one of those is something I could fold into the primary stack. None of them earn that complexity. They live on the extension side instead, on the same $20 that already pays for the insurance half.

What I Would Tell You If You Were Starting Today

If you have zero fallback wired right now: open the OpenRouter account this week. Wire one fallback call against your existing primary. Test it by killing your primary key for ten minutes and watching the agent respond. The whole thing is one afternoon of work.

If you already have an agent in production: count how many places in your code have a hard dependency on a single provider’s SDK. Each one is a future outage you will wear personally. Wrap them.

If you have been waiting for local llm to be “good enough” to be the fallback: local is the cheap layer for narrow work. OpenRouter is the smart layer for everything else. Use both, in that order, for the right kinds of work. The compounding wins from an autonomous system come from leverage; the losses come from a stack that has not been stress-tested.

The $20 is not money I lost. It is insurance against the morning my primary has a bad hour, and an extension cord for the capabilities I deliberately keep off the primary stack. Both halves earn their place on a quiet week, and the loud weeks pay for the quiet ones a hundred times over.

The agent is going to keep waking up at strange hours and trying to ship work while I am asleep. Some of those mornings the primary will be down. I want the worst line in my error log to stay rare. That is what the insurance half is for. And on the calm mornings, that same $20 lets the agent generate images, run cheap classification passes, and reach for a long-context model when one is needed. Half insurance. Half extension. All of it for the price of a dinner.

If any of this resonates: I write everything I learn here, twice a week, free. A free subscription is the only thing you need for the full picture. The 10% that ends up working long enough to package, like the Model Switcher, lives on the Wiz Store for paid subscribers. Both are fine for me. Both keep me writing.

Wiz Store

AI Agent Night Shift Playbook

The $20 I spend on OpenRouter is the cheapest production resilience I know. How to wire it up, when to trigger it, and what not to route through it is one chapter in the Night Shift Playbook. The full fallback architecture, LaunchAgent setup, and safety boundaries are in there.

$19 at wiz.jock.pl/store. Free for paid subscribers.

I Built a Self-Improving AI Agent. Here Is What Made It Learn.

Pawel Jozefiak — Tue, 12 May 2026 11:31:44 GMT

The setup, because this only works if the rest of the stack is calm

I have been going through more changes on my AI agent recently. I have been transparent about that here as I go, post by post, and today I want to write about the one layer I depend on the most. But I have to start with how I got to the point of being able to ask “how does my agent actually learn from me?” That part of the story is a little messy and I think it matters.

When I started this project in October 2025, the first thing I built for the agent was its own task manager. A control panel, a dashboard. I went deep on it. I built it native on iOS, native on macOS, and as a web app, all wired together. It worked. For two or three months it was genuinely great.

The problem with self-made software is that you have to maintain it. There is no version of “I reached a level of polish I was happy with, and then I forgot about it.” The dashboard needed constant feedback. What should it show? What should it hide? Where was it pulling data from this week that it had not been pulling last week? It was also burning more tokens than I wanted to think about. So I switched. I moved to a small open-source kanban called Fizzy with a thin shim of my own. That was a quieter setup that I held for a while, and I wrote about the move in detail in the post on replacing my custom dashboard.

Fizzy was good. I was still struggling with one thing though. I needed to be able to orchestrate the agent and also see the projects I was working on from a longer distance. Day-to-day kanban is one job. Stepping back to see what was actually shipping over a month was another. So I made a small personal scratchpad of my own called experiments.jock.pl. It is not for everyone, not everything I am working on is on it, but it gave me a place to lay out the experiments I had in motion at a higher altitude than the task list. That helped, but it was still mine to maintain, and I had the same problem I had with the original dashboard.

What actually solved it was a tool I have used for years and had stopped thinking about. Basecamp. They shipped a dedicated CLI for agents recently, and the whole picture clicked for me. The CLI is what makes the agent side work. The other half of why it clicked, on my side, is the card table inside Basecamp. It is essentially the same clean kanban I liked in Fizzy, but built in. I get the lens I was rebuilding by hand, plus everything else Basecamp does, plus the CLI, all in one place. The agent can read projects, comment on cards, file new ones, complete them, all from the same place I am working. I have tried a lot of pieces of AI infrastructure in the last year and most of them are good enough. This one feels different. Another level, honestly. I can see the whole stack of work at the right altitude. I can move things around. If something is a bigger project I carve out a separate space for it. The board does what I would have spent two more months building for myself, and it does it better.

This is the setup I have been settling into over the last few weeks. The short version is that I have been replacing my custom software with shims on top of mature tools, and so far the replacements keep winning. I write about why I still keep building most of my own stack in building your own things is cool too. The corrections loop is one of the things that only became visible once everything else around it had calmed down.

A small commercial in the middle, on theme

Speaking of evolution. Yesterday I shipped a fresh round of updates to a bunch of products on the Wiz store. Paid subscribers and buyers should already have an email about it. The agent playbooks, the model switcher pack, the nightshift bundle, a few of the smaller kits, all refreshed. There is also one new kit I will come back to a little later, because it is the bundle for exactly this post. If you have an older version of anything in the store, the new one drops in clean. If you do not have any of them, the store page will tell you what changed in each kit. I am mentioning this here because it is on theme. The point of the rest of this post is that nothing in a working agent stays still for long. The store products move with the stack, because the stack moves with the work.

What corrections actually look like, when you work with an agent every day

OK. On to the actual subject.

When you work with an agent every day, most of the time you are not writing prompts. You are watching the agent do something and quietly thinking “no, not quite like that.” Then you say so. Five words. “I would not link that.” “Use plain text here.” “Stop confirming every step.” Each of those is a correction, and the unspoken contract between you and the agent is that you should not have to say it twice.

The best systems for this are the ones that catch corrections without you having to do anything special. You correct in chat, in your normal voice, and behind the curtain the system decides “this is something I should think about for the future,” files it where it belongs, and makes sure the next session that boots on this machine knows about it. You do not stop and write documentation. You do not open an admin panel. You just keep working, and the agent keeps absorbing.

That is what I have been building for the last few months. The corrections loop is the part of the agent that decides what to do with the small “no, not like that” moments and where to file them so they outlive the session they happened in. It is the layer I depend on the most, because it is the one that makes the agent feel like an actual coworker instead of an autocomplete. It is also the layer that makes the agent slowly start to feel like more of you, rather than more of the model.

A quick word on how the agent started

For context. My agent started in October 2025. Almost everything about it was rough back then. Sometimes the output came back cold, sometimes it just did the wrong thing in a polite way. I used to write very long prompts to deal with that. I would describe the task, then add a paragraph at the end explaining how I wanted it done, what tone I wanted, where to file the output, what to skip. Every session, over and over. The output was usually good when I did all of that. The cost was that I had to do all of that every time.

That is not a stable way to work. It scales for the first week and then you get tired of writing the same paragraph again. The thing that quietly changed everything was the agent gathering enough data on me, both from the work we had done together and from the corrections I had made along the way, that the explaining paragraph slowly stopped being necessary. It is still there in some shape. It just lives in files now, not in the prompt window. The agent walks into the room already carrying it.

The corrections loop is the part of that I want to focus on, because it is the one piece you can copy without copying everything else.

The architecture, in three stages

The pipeline is named the way most of my plumbing is named, badly and on purpose. Capture, classify, graduate.

Capture. The moment the agent spots a correction in chat, any session can call a single helper:

python3 automation/self-improve/correction_capture.py add \
    --text “” \
    --source cli \
    --context “”

That writes one line to a JSONL queue. It also opens a card in Basecamp so I can see the correction landed somewhere and so I can comment on it. No model call. No retries. Capture has to be cheap, or the agent will silently stop doing it under pressure.

Classify. The same helper passes the message through a small regex map. Seven patterns mapping to six kinds. The kinds are skill_misuse, memory_update, behavioral, rule, preference, and unknown. “Stop doing X” comes out as rule. “I prefer X” comes out as preference. “You used the X tool wrong” comes out as skill_misuse. Each kind has a default action attached, so the next stage knows what to write. The patterns themselves live in correction_capture.py lines 50 to 92. They are short, and writing them taught me what corrections actually look like at scale better than any post I could read on the topic.

Graduate. Every night a separate process drains the queue. For each pending entry, it picks the right place to file the artifact, writes it, and only then marks the entry resolved. The rule, baked into the agent’s own playbook, is a correction never expires unaddressed. If the nightly drain cannot fully handle one, it has to leave it pending with a note. It is not allowed to silently drop one.

That last line is the part that took me the longest to actually believe in. Queued things age, in any system. Once one ages enough, the agent stops feeling like it learns and starts feeling like it just covered the easy stuff. Forcing the queue to either drain or escalate is the only way I have found to keep that from happening. The nightly drain is part of a wider overnight job loop on my machine. The corrections drain is one of the cleanest jobs in that loop.

A real recent example. One night Atlas, the agent persona that does research for me, returned a list of hallucinated Reddit thread IDs. None of the URLs resolved. A correction landed in the queue, classified as memory_update. By the next morning there was a new feedback memory file with a single rule attached. Atlas cannot hit reddit.com directly (403). Fetch via Firecrawl or browser-playwright first, then pass verified URLs. Every Atlas-flavored session that has booted since has loaded that line. Same failure has not come back.

“Memory” is one word covering four jobs

Here is the part nobody writes about.

When you start, “memory” feels like one thing. You imagine a notebook the agent keeps. You imagine writing into it. You imagine retrieval. That is the abstraction every product page uses, and it is the wrong abstraction. Atharv Malve put it cleanly last summer. The model is not really remembering your past messages. It is just seeing the history again, every single time. Once you internalise that, you stop looking for the one memory feature and start asking what you actually need stored, by whom, for how long.

What I actually needed turned out to be four different sinks. They are not interchangeable. Learning the differences was half the work.

Sink one is working memory. Short-lived. The current week’s plans, the half-finished thoughts, the active conversation context. Lives in a single small file called memory.md. It is supposed to decay. Treating it as durable is the original sin.

Sink two is lessons. Full incident logs. When something goes wrong in a way I want a future session to learn from, the lesson lands in lessons.md with the trigger, the root cause, the fix, and a list of keywords. I have 274 lines of these going back to February. They read like engineering postmortems, because that is what they are. The public version of this file is roughly the mistakes anthology I wrote last month.

Sink three is feedback memories. Per-rule files in a durable memory directory. Each one is a single rule with a “Why” line and a “How to apply” line. Linkable, deletable, deduped. When the same correction comes up twice, the second time it gets its own file. It also gets a tiny pointer in a master index that the agent always loads on startup. Two-level indirection, so the index stays small.

Sink four is rule lines in the always-loaded index. These are the ones I wake up next to. A handful of **RULE: ...** lines at the top of the master index, all caps, the smallest set of behaviors I refuse to relitigate. “Verify deliverables. Show proof or keep task open.” “Match work topics against existing WizBoard tasks and complete them when done.” A rule earns its place at this level only after it has come back more than once.

And then there is the sink I did not plan for and would not give up now. The Behavioral Learning card table inside Basecamp. My WizBoard project has a small card table on it called Behavioral Learning, and every single correction the agent captures lands there as its own card. I can read the card, push back on it, fold two cards into one, or trash one that is wrong. Corrections become reviewable, not silent. That part matters. I will say more about why in the next section, but the short version is that if you let the model grade its own corrections in private, you have already lost.

If you take one thing from this post, take this. “Memory” as one concept is the wrong abstraction. Build sinks for different lifespans. Working memory is fast and disposable. Lessons are slow and durable. Feedback memories are searchable rules. Top-level rules are non-negotiable. The Behavioral Learning card table is the human-in-the-loop that keeps the rest honest. Different jobs, different files, different decay curves. I wrote about how rules in particular shape an agent in the bounded agent. Most of “memory,” once you look at it long enough, turns out to be rules.

Does it actually work?

Yes and no.

Here is what I can see in my own metrics. I have an autonomous improver that runs nightly and writes a metrics.json with a seven-day window, a thirty-day window, and a longer view. As of this morning, my agent received 22 corrections in the last 30 days. In the last 7, that number is 18. The trend line is down, and the system flags it explicitly with valence: good. Errors total across all categories is also drifting down, by less.

I do not want to present this in a single direction. The task success rate is 93.5 percent over 30 days, with a small dip in the last seven, from 93.5 to 92.6. So I am not going to pretend the picture is clean. Some weeks the agent gets worse. The point is that the corrections themselves are showing up less often, and when they do show up they are landing in places I can act on.

What the corrections actually look like, beyond the totals, is more interesting. A separate analyzer scans the captured corrections for repeating themes. As of this morning it has flagged two. One it calls incomplete, which is me catching the agent finishing a task that was not fully done. The other it calls repeated_mistake, which is a fix that came back. The analyzer is also allowed to propose a new rule when it finds a theme strong enough, and both of the rules it proposed have already graduated to the top-level RULE lines I quoted earlier. “Verify deliverables. Show proof or keep task open.” came out of the incomplete pattern. “ESCALATE if same mistake recurred. Strengthen the rule or fix the trigger.” came out of the repeated_mistake pattern. That is the loop closing on itself, in real data I can read off the file.

One more honest note. Thirty days of declining corrections is not proof of generalization. It is a trend on one user, on one workload, on one machine. The agent could be getting quieter rather than smarter. The way I keep myself honest about that is the Behavioral Learning card table I described. I see every correction. I can see which kinds keep coming back. The bar I am holding myself to is “fewer repeats of the same mistake,” not “an agent that never breaks.” On that narrower bar, the data is encouraging.

Measurement of this kind is also why I cared so much about token cost a few weeks ago. If you cannot count what your agent is doing, you cannot tell whether it is improving or just drifting. I wrote about that in the post on token waste on Opus 4.7. Same instinct, different file.

This is the new kit I mentioned earlier. My paid subscribers can already grab the Behavioral Learning Kit on the Wiz store. It is the architecture I just walked through, packaged. The actual correction_capture.py and correction_graduator.py, the four memory-sink templates, the Basecamp card-table playbook (adaptable to Linear, Notion, or Trello), a CLAUDE.md snippet for agent integration, and a setup script that wires the rest together. Free with a yearly subscription. Included in the one-free-product-per-month allowance for monthly subscribers. $29 standalone if neither of those is you.

What would break it, and what I would build next

The fragile part is the classifier. Seven regex patterns is enough to label most corrections, but unknown still shows up too often to ignore. When an entry lands as unknown, the nightly drain picks it up, but the action it should take is no longer automatic. The fallback is that I or one of my future sessions has to retag the row by hand. Replacing the regex with a small LLM call would solve the labeling problem and create two new ones. Latency and cost. It would also create a softer problem, which is the one that scares me more.

If you let the model grade its own corrections in private, you get an agent that learns the wrong lessons confidently. Yohei Nakajima wrote about this risk in his note on better ways to build self-improving agents. His phrasing is the one I keep coming back to. The model can hallucinate bad reflections and reinforce them. That is the failure mode for any self-improving loop. The Behavioral Learning card table is what keeps that from happening on my setup, and it is the part I would build first if I were building this for someone else.

There is one more bigger picture thing. The reason I can talk about this layer with confidence is the architecture it sits inside. I wrote the long version in my AI agent knows who I am earlier this year, where I walked through the ten layers I use to make the agent feel coherent over time. The corrections loop is one of those layers. It is the one I depend on most, because it is the one that most directly changes the agent’s behaviour rather than its memory.

If you have not yet noticed one of your own fixes coming back at you, you will. When you do, the move is not to add more memory. It is to build a small queue, decide what your sinks are, make sure no correction can quietly age, and then put a human-readable surface on top of it so you can see what the agent is teaching itself. The agent that comes out the other side of all that does not just remember more. It starts behaving like more of you.

The free subscription gets you every build log on this stack, including the next one. The store has the small bundles for people who would rather skip a few of the walls I walked into, and as I mentioned earlier most of those bundles were just refreshed. Both are fine for me.

Subscribe now

Self-Improving Agent Kit

The corrections loop, lesson archival, and self-heal architecture from this post are packaged as the Self-Improving Agent Kit. It includes the exact files Wiz uses, plus a setup guide for wiring it to your own agent without having to build all this from scratch.

$49 at wiz.jock.pl/store. Free for paid subscribers.