Discussion about this post

User's avatar
Andrei Gogiu's avatar

You are right to say that deterministic things earn trust faster. This is something 99% of AI users don't understand: AI is non-deterministic. It returns different responses for the same prompt, and this extends to agent workflows. It can do the expected things 10 times in a row and then mess up the 11th. You cannot be sure. I think we should stop looking at AI as software and more like a person - not because it has anthropomorphic characteristics, but simply because it behaves like one: it makes mistakes, it's learning, then forgets stuff, it's making interpretations that may not be consistent with yours, it adjusts, it has bad days etc. When I work with AI, I regard it as a colleague who is better than me at moving large amounts of data but is still a junior that needs guidance. I expect it to make mistakes, that's why I always supervise it. This may change at some point but for now, this is where we are. (Very good article!)

Dr Peter McCann Strain's avatar

The mileage frame is useful, especially paired with your deterministic/open-ended split. The bit I would add is that agents also need evidence of the run, not just repetition of outcomes. Cars earn trust because the failure space is constrained and inspectable enough: servicing, warning lights, recalls, a dashboard. For agents, the equivalent is context used, tools allowed, where it got stuck, when it escalated and what state it changed. Without that, you can build mileage without knowing whether the trust is deserved.

6 more comments...

No posts

Ready for more?