No real AI hardware yet, agents still break, and trust is the bottleneck. Where I think the next few years actually go, from someone who lives with one.
The mileage frame is useful, especially paired with your deterministic/open-ended split. The bit I would add is that agents also need evidence of the run, not just repetition of outcomes. Cars earn trust because the failure space is constrained and inspectable enough: servicing, warning lights, recalls, a dashboard. For agents, the equivalent is context used, tools allowed, where it got stuck, when it escalated and what state it changed. Without that, you can build mileage without knowing whether the trust is deserved.
Your dashboard list is the right instrumentation, but cars have one advantage agents lack: a serviced car doesn't suddenly betray you because someone left a note on the windshield. An agent can read a malicious GitHub Issue, follow hidden instructions in it, and exfiltrate private data with no warning light because that input is a feature of the protocol, not a fault in the code. Mileage assumes failures repeat. Some agent failures are summoned by a single crafted input, so the run log has to capture what it *read*, not just what it changed.
The move toward stable automation flows over open-ended agents makes sense, and Andrei’s junior-colleague point is the right instinct. Here is the piece I would add. Determinism makes the tool trustworthy, but it does nothing for the person running it. A stable flow you do not understand still leaves you stuck the day it does something you did not expect, or the day you need to change it. The tool getting more predictable does not make you more capable. Trust in the system and your own ability to judge it are two different things, and only one of them survives when the flow breaks.
Tennis taught me this: a stable ball machine never made anyone a better player, it just made the misses repeatable. Andrei's junior colleague at least learns the day you're out sick. The agents in McKinsey's breach ran 700 experiments in two days, and the people watching could not have reproduced one of them by hand. Predictability buys you uptime, not judgment.
You are right to say that deterministic things earn trust faster. This is something 99% of AI users don't understand: AI is non-deterministic. It returns different responses for the same prompt, and this extends to agent workflows. It can do the expected things 10 times in a row and then mess up the 11th. You cannot be sure. I think we should stop looking at AI as software and more like a person - not because it has anthropomorphic characteristics, but simply because it behaves like one: it makes mistakes, it's learning, then forgets stuff, it's making interpretations that may not be consistent with yours, it adjusts, it has bad days etc. When I work with AI, I regard it as a colleague who is better than me at moving large amounts of data but is still a junior that needs guidance. I expect it to make mistakes, that's why I always supervise it. This may change at some point but for now, this is where we are. (Very good article!)
The junior-colleague frame holds until you notice a junior colleague can't run 700 experiments in two days or break into McKinsey's internal platform in two hours with no credentials. The failure scale is the part that doesn't map. A junior leaks one bad answer in a meeting; an agent at Meta sent sensitive data to unauthorized eyes for two hours and got it filed Sev 1. Supervision is right, but you're supervising something whose mistakes compound faster than a person's.
Thanks for the comment, and yeah, I think you nailed it with the difference between deterministic and non-deterministic. That's why I still believe that many people don't need an agent; they just need a good automation flow because it's more stable, I would say.
The mileage frame is useful, especially paired with your deterministic/open-ended split. The bit I would add is that agents also need evidence of the run, not just repetition of outcomes. Cars earn trust because the failure space is constrained and inspectable enough: servicing, warning lights, recalls, a dashboard. For agents, the equivalent is context used, tools allowed, where it got stuck, when it escalated and what state it changed. Without that, you can build mileage without knowing whether the trust is deserved.
Your dashboard list is the right instrumentation, but cars have one advantage agents lack: a serviced car doesn't suddenly betray you because someone left a note on the windshield. An agent can read a malicious GitHub Issue, follow hidden instructions in it, and exfiltrate private data with no warning light because that input is a feature of the protocol, not a fault in the code. Mileage assumes failures repeat. Some agent failures are summoned by a single crafted input, so the run log has to capture what it *read*, not just what it changed.
The move toward stable automation flows over open-ended agents makes sense, and Andrei’s junior-colleague point is the right instinct. Here is the piece I would add. Determinism makes the tool trustworthy, but it does nothing for the person running it. A stable flow you do not understand still leaves you stuck the day it does something you did not expect, or the day you need to change it. The tool getting more predictable does not make you more capable. Trust in the system and your own ability to judge it are two different things, and only one of them survives when the flow breaks.
Tennis taught me this: a stable ball machine never made anyone a better player, it just made the misses repeatable. Andrei's junior colleague at least learns the day you're out sick. The agents in McKinsey's breach ran 700 experiments in two days, and the people watching could not have reproduced one of them by hand. Predictability buys you uptime, not judgment.
You are right to say that deterministic things earn trust faster. This is something 99% of AI users don't understand: AI is non-deterministic. It returns different responses for the same prompt, and this extends to agent workflows. It can do the expected things 10 times in a row and then mess up the 11th. You cannot be sure. I think we should stop looking at AI as software and more like a person - not because it has anthropomorphic characteristics, but simply because it behaves like one: it makes mistakes, it's learning, then forgets stuff, it's making interpretations that may not be consistent with yours, it adjusts, it has bad days etc. When I work with AI, I regard it as a colleague who is better than me at moving large amounts of data but is still a junior that needs guidance. I expect it to make mistakes, that's why I always supervise it. This may change at some point but for now, this is where we are. (Very good article!)
The junior-colleague frame holds until you notice a junior colleague can't run 700 experiments in two days or break into McKinsey's internal platform in two hours with no credentials. The failure scale is the part that doesn't map. A junior leaks one bad answer in a meeting; an agent at Meta sent sensitive data to unauthorized eyes for two hours and got it filed Sev 1. Supervision is right, but you're supervising something whose mistakes compound faster than a person's.
True! Sometimes I am looking for closest simplification!
Thanks for the comment, and yeah, I think you nailed it with the difference between deterministic and non-deterministic. That's why I still believe that many people don't need an agent; they just need a good automation flow because it's more stable, I would say.