"You have to think about this like onboarding a new team member. If my agent needs access to my calendar, I need to share it. (I shared my iCloud calendars with the agent’s Apple ID.) If it needs email, I need to log in. (I set up Apple Mail with my Macmillan work account.) If it needs files, I need a way to exchange them."
I am thinking about something like IAM role in aws. An access control plan, so which agent can access which things. Do you have any idea of that existed?
I have a Mac Studio and can run local models. I wouldn't get super excited about them - yes, they are "Free" but you pay for that with quality. You need very specialized use cases to make the "Free" worth it. You don't have to guess though - you can test them on OpenRouter.ai and compare against frontier models. If the math works out that the local models you can run on a 96GB or 256GB Mac Studio (the 512GB is no longer available) are cheaper than frontier, easy decision.
Hah, yeah + you know what it is? It is a story of expectations and being too optimistic about the whole thing :D
And for the local LLMs, I would say that I think for now there is a very limited scope for them, at least in an agentic kind of era. I think that in the future we will get better and better smaller models that will be very good at very specific things. Because of the fine-tuning, you can get really good results for your specific needs. This is what I think will happen in the next foreseeable future.
For now, as you said, the usage for the local LMs is kind of limited. I wouldn't say it is worthless or something there is worth. I think you have privacy, at least. I wouldn't say it is free, because you always pay with something, quality or power, but yeah.
"You have to think about this like onboarding a new team member. If my agent needs access to my calendar, I need to share it. (I shared my iCloud calendars with the agent’s Apple ID.) If it needs email, I need to log in. (I set up Apple Mail with my Macmillan work account.) If it needs files, I need a way to exchange them."
I am thinking about something like IAM role in aws. An access control plan, so which agent can access which things. Do you have any idea of that existed?
Dev versus prod hell in a nutshell - nice read.
I have a Mac Studio and can run local models. I wouldn't get super excited about them - yes, they are "Free" but you pay for that with quality. You need very specialized use cases to make the "Free" worth it. You don't have to guess though - you can test them on OpenRouter.ai and compare against frontier models. If the math works out that the local models you can run on a 96GB or 256GB Mac Studio (the 512GB is no longer available) are cheaper than frontier, easy decision.
For me, it definitely wasn't.
Hah, yeah + you know what it is? It is a story of expectations and being too optimistic about the whole thing :D
And for the local LLMs, I would say that I think for now there is a very limited scope for them, at least in an agentic kind of era. I think that in the future we will get better and better smaller models that will be very good at very specific things. Because of the fine-tuning, you can get really good results for your specific needs. This is what I think will happen in the next foreseeable future.
For now, as you said, the usage for the local LMs is kind of limited. I wouldn't say it is worthless or something there is worth. I think you have privacy, at least. I wouldn't say it is free, because you always pay with something, quality or power, but yeah.
"Three days of migration. A few hundred dollars for the Mac Mini. Several late-night debugging sessions.
Absolutely worth it."
Hell yeah! this was an awesome read. Really glad to hear about your experience here.
Thanks!
As usual, I was too optimistic about whole thing xD But yeah, worth it!
the optimism is refreshing haha. i also think it leads to more discovery!