Hedy can now run the full AI pipeline on-device via Metal. ~15s summaries on M4 Max with Qwen 3.5 9B
I'm the founder of Hedy, an AI meeting coach with 25k+ users that's been on macOS for about a year. We just shipped Local AI Processing, which moves the entire AI pipeline (summaries, detailed notes, chat with the meeting, live coaching) off cloud servers and onto the user's Mac. Demo above shows it running with Wi-Fi turned off the entire time. Problem Most AI meeting tools depend on data centers. Audio gets transcribed somewhere, the transcript gets sent to OpenAI or Anthropic, a summary comes back. The audio itself is the most sensitive part. It contains everything anyone said in the meeting with identifiable audio patterns, and it leaves your device. This rules out AI meeting tools for the kinds of conversations that need to stay private: privileged client conversations, sensitive interviews, medical appointments, confidential coaching sessions, and any meeting where the audio shouldn't sit on a third-party server. It also rules them out for anyone working without reliable internet. The Mac engineering question we set out to answer: can we run the whole AI pipeline natively on Apple Silicon at quality that's actually useful for real meetings. As of 3.2, the answer is yes. Comparison Two closest tools in this space: Granola is the closest Mac-native competitor. Despite the "local-feeling" Mac app experience, Granola sends your meeting audio to the cloud for speech recognition, and then sends the resulting transcript to OpenAI for analysis. Otter is the dominant cloud-based tool. Same architecture: audio gets uploaded for cloud transcription, then the transcript is processed by their AI stack on the server side. Hedy works differently and always has. Speech recognition has run on-device from day one. Your audio has never left your machine. With 3.2, the AI analysis layer (summaries, notes, chat, live coaching) can now also run on-device when Local AI is turned on. That's the full pipeline running locally, end to end, with no conversation content leaving the device. This matters beyond personal preference. Sending client conversations, patient appointments, or interview audio to third-party cloud servers creates real privacy exposure and, in some professional contexts, potential legal issues. Hedy's local pipeline removes that exposure entirely. What's specifically Mac-native in Hedy's approach: Metal GPU acceleration for inference across all M-series chips Unified memory architecture means total system RAM is the constraint, not separate VRAM Native macOS app, not Electron-wrapped App surfaces a fit indicator before model download (Great fit / Tight fit / Won't fit) based on available memory Siri Shortcuts integration for hands-free session control Apple Watch companion app for session monitoring Model support: Qwen 3.6, Qwen 3.5, and Gemma 4 in the curated lineup. Range goes from 2B on the low end (works on iPhone 15 Pro+ as well) through 9B as the sweet spot for most M-series Macs, up to 27B and 35B for users with 36GB+ unified memory. Multiple quantization levels per model. Plus bring-your-own model support from Hugging Face if you don't trust our curation. Pricing Free: $0/mo (5 hours/month, all platforms, all features for trying it out) Pro: $12.99/mo or $99.99/yr (unlimited usage) Lifetime: $299 one-time (currently also available on AppSumo for a limited time) All tiers include Local AI Processing. Download: https://hedy.ai Transparency Company: Hedy AI LLC, registered in Portland, Oregon, USA https://www.linkedin.com/company/hedy-ai/ Website: https://hedy.ai Founder & CEO: https://www.linkedin.com/in/julianpscheid/ Trust Center: https://trust.hedy.ai/ Privacy Policy: https://hedy.ai/privacy-policy/ Terms of Service: https://hedy.ai/terms-of-use/ App distribution: macOS direct download from hedy.ai, iOS via App Store: https://apps.apple.com/us/app/hedy-ai-meeting-coach-notes/id6502325255 Honest tradeoffs: Cloud is still the default in Hedy because it's faster and produces higher quality output for most users Local is opt-in, off by default No silent cloud fallback. If local fails, you see an error. Deliberate design choice. Local AI is configured per device. Settings and downloaded model files don't sync between Macs. iPhone is limited to smaller models due to memory constraints Android and Web don't yet support Local AI (hardware variation and browser constraints make consistent experience hard) Staying in the thread for questions about Metal vs Vulkan tradeoffs, unified memory advantages for local inference, model selection guidance, or anything else technical.
已收集讨论
Sounds like a great app in the current AI era. Thanks for the links. How is local AI token performance on Mac Mini (say M4 32GB ram) with your app, versus your other on device and on cloud benchmarks? Esp lag & latency.
Thanks for the question. It really depends on the length of the meeting and the amount of context that Hedy is carrying. We've seen smaller local models (under 10B params) perform faster than cloud models on an M4 if it's a short meeting. Longer meetings or larger models tend to take longer than cloud inference with current consumer hardware. To give you an example on the extreme end, we use Lex Fridman podcasts for longer session tests, and the 8 hour Elon interview took about 20 minutes to summarize on an M4 with a medium sized model.
8hrs summarisation in 20mins sounds decent on device.
Amazing product. It's the best meeting, notes, insight app, or service I've ever come across. I've tried plenty, and this knocks them all out of the park. Though for my use case, I suspect I wouldn't want to trouble anything, seeing that everything just works so fine as it is. Plus, my Mac Mini, my iPhone, and my iPad are all a couple generations old. So even though I'm not running it on the latest hardware, things are well. Never had a problem, so I'll keep trucking. This product has compelled me to cancel my Otter, my Krisp, and even my Granola subscription. Don't need them.
Nice, thanks for sharing! And yes, for most users local AI doesn't make sense today. But that will likely change over the next couple of years as local models become even smaller and faster... so we are building the foundation now. Thanks for being part of our journey!
how hot does the mac get running the 9b model for long meetings?
We recommend turning off automatic suggestions (a feature that analyzes the meeting conversation automatically in the background) when running Local AI... that way Hedy doesn't actually run any inference unless you ask it something or end the session. That said, we've definitely tested sessions with inference running automatically in the background and the fan will spin up pretty quickly!
Max with what ram?
This is on my Mac M4 with 36GB. I can run the 27B+ parameter models as well, but they are noticeably slower.
Thanks!
have you run any benchmarks on latency / accuracy when you run it locally?
Yes for sure, but full disclaimer: the results vary A LOT based on the system that is running it. The great news is that the Qwen 3.5/6 and Gemma 4 models in the 30B parameter range appear to approach the accuracy of our cloud models. My M4 Mac with 36GB ram can run them pretty well, albeit results definitely come in slower than using our cloud inference. For meetings where only a summary is needed at the end, that's fine, but for real-time support during the meeting that's not ideal yet. Smaller models are way faster though and can be faster than sending it to the cloud, but then accuracy isn't as good. These are trade offs we need to make today that hopefully won't be an issue in the near future.
Why did you choose to natively handle local model inference? What are the advantages over building a bridge to existing local inference apps like Ollama and LM Studio? For Alter, it felt natural to create a bridge to let users have the choice of any models their device can handle and avoid bloating their storage with models if they already have them.
Fair question. For us it is about expanding access to local AI beyond power users that might already have an Ollama setup. Our goal is to bring Local AI into the mainstream for average users. Today that might still be gated behind model selection (and we are clear that average users might not want to try this feature yet), but we hope that soon we can make that completely transparent to users, just like we do it with local STT.
这条评论已在来源处删除。
Our model catalog includes both Qwen 3.5/6 and Gemma 4 models. Those two model families really offered the breakthrough in small model performance that gave us the confidence to launch Local AI. Of course there are stylistic differences, and Qwen models tend to work better for the response length and "personality" that we want in Hedy. That's very subjective though and varies between use cases.
这条评论已在来源处删除。