What’s the best fast and reliable local model for iCloud Calendar, Reminders, and Home Assistant on an M5 Max?
Brand new to local LLMs as I never saw the point until I got this laptop. I’m currently using GPT 5.6 as a family assistant, but it’s complete overkill for managing calendars and reminders, home automation, and stupid silly chatting with my wife and me in a…
Brand new to local LLMs as I never saw the point until I got this laptop. I’m currently using GPT 5.6 as a family assistant, but it’s complete overkill for managing calendars and reminders, home automation, and stupid silly chatting with my wife and me in a group chat with it. The model needs to be fast, reliable, and good at tool calling. I want to be able to casually say: “Bob turn off the kitchen lights and set the bedroom lamp to 20%” And have it do both correctly the first time without taking 10-30 seconds or needing repeated instructions. Ideally it should only take a couple seconds otherwise what’s even the point? The fun of it goes away and I could just do what I’m asking it myself instead of waiting. It also needs to reliably add, edit, and delete iCloud calendar events and reminders through MCP or skills. We basically treat it like a secretary. Would a good 12B model be enough for this, or is something around 30B more realistic? I’m brand new to running local models and so far all I’ve done was set up DeepSeek V4 Flash locally for fun and proof of concept, but it’s not fast enough obviously for this simple work I’d like it to do. Ideally I’d like to eventually run multiple models simultaneously given I have 128GB of memory to play with. Any help or input is appreciated, thank you!
已收录讨论
Not to hijack your thread but would you mind explaining your current use case as family assistant and the infrastructure you use? I do some super limited local LLM stuff on an RTX 3080 with 8b models and mostly use it for research without dealing with stupid model safety guardrails. But I am currently trying to come up with a system to help my wife with her calendar and scheduling (she has pretty intense adhd that makes that stuff hard). I want to simplify her interactions with both her professional calendar and our kids lives and family stuff etc. Would love any tips on what you’re using and how you have it wired up
I have a Mac mini M4 Pro I bought when it came out to play around with and keep some local servers on running 24/7 since it's so power efficient. Since I run it 24/7 when I learned about AI agents I figured it would be the best place for me to throw one. When I set it up I had the regular $20 USD ChatGPT subscription which you can log in to with agent harnesses like OpenClaw, Hermes, etc. I set up Hermes Agent on the Mac mini in a separate user account than my own, gave it it's own iCloud account, and added that iCloud account to our family iCloud so it could see our calendar, reminders, etc. Then I set up BlueBubbles with Hermes and the iCloud account so we could communicate with it over iMessage exactly the same way we txt message each other and created a group chat between me, my wife, and the Hermes agent. Then I basically just message it whenever I wanna add things to our grocery list, reminders, calendar, etc. I added Home Assistant functionality later so it could control devices in our smart home. I'm still looking to add more functionality like read-only email access so it could automatically add important things to the calendar and reminders and / or have it ask us directly on its own if it should add whatever from the emails to the calendar. Looking into other improvements for it, too but been distracted by way too many side projects lately (I also have pretty bad ADHD and am horribly disorganized). Part of why I put it all together was because I want to slowly force myself to be more organized and keep track of things. Obviously you don't need a Mac for all of this, just a computer for it to run on. I just had a Mac I was already running 24/7 and specifically wanted it to work in iMessage to feel as natural as possible for the both of us. It could all be set up with various different messaging services and even simultaneously. I got it working in Google Chat too because I cannot access iMessage when I'm at work so I wanted a way to still talk to it if needed while working. You can do this all with a cloud or local model though I was always using a cloud model to make it as smart as possible. Looking to migrate it to a local model now since I've come to realize that I don't really need a frontier model to be a silly chatting family assistant lol
Nice, I also have an m4 Mac mini that I leave on permanently and have been considering setting up Hermes on. Thank you this is very helpful!
Seconding Gemma 4 26b specifically. We run it as the brain for a household assistant with tool calling and it's been the reliable one, noticeably better than the smaller quants we tried first, which would call the right function but with wrong or missing arguments more often than their chat quality suggested. Your small-system-prompt point extends to the tool surface too. Every extra skill or MCP server you bolt on is one more thing for it to pick wrongly, and the degradation is quiet rather than obvious. OP, the specific thing to watch is in your own example. "Turn off the kitchen lights and set the bedroom lamp to 20%" is a two-part command, and dropping the second half while confidently confirming the first is the classic small-model failure here. Training_Isopod3722 is right that you have to count actual tool calls to catch it, because the reply reads fine either way.
look into gemma 4. e4b is surprisingly powerfull. try it. needs small system prompt or it gets confused, no 30k token prompt. if you can run 31b then you can through all of that at it and it will do it. im using it for exactly that and its perfect. 26b is super fast and also really good. 12b might work for you as well, but i dont have the best experience with it. the qwen models never really worked for me as good as gemma 4
I think the m4 mini could handle this on its own handily running qwen 3.6 35A3B. Check out running it through OMLX.
Intersting topic, it's neat to see where local AI is headed.
There isn't a single model that's small enough to be fast on consumer hardware but also smart enough to be reliable in all cases. And since you want to run multiple models, you should set up a model router that forwards simple queries like "turn off the lights" to Gemma 4 12b QAT with MTP or even just Qwen3.5 9b, and for tasks that require skills and additional MCP (larger context window), route it to Gemma 4 26B-A4B / 31B or Qwen3.6-35B-A3B. Keep the smaller model loaded at all times so that you can instantly get a response. You can set up the routing with https://github.com/aurelio-labs/semantic-router - this uses vector embeddings to classify prompts so no extra LLM overhead. Or if you want a simpler setup then just use a tiny model like Granite 4 Nano to classify the task and route accordingly. It'll still be pretty fast overall on your hardware.
fair warning, test the whole tool loop before picking from benchmark charts. a model can answer normal chat fine then quietly drop the second action or emit arguments your Home Assistant schema rejects. start with 20 or so real commands and count exact tool calls, not just whether the reply sounds right.
A lot of this should be done via scripts and initiated by the llm vs having the model do everything from scratch each time.
Take a look at osaurus.ai - we specialize literally for the use cases and things you mention with our priority being that Mac’s with 16gb of RAM or less can utilize real localized models to at least be able to do the kind of automation you mentioned