Lyra - On-device Mac writing assistant that learns how you write to each person
Problem Two shapes of writing tool exist today and neither fit how I actually write on my Mac. One is a separate chat window (ChatGPT, Claude) that needs constant copy-paste in and out. The other is a generic rewriter (Grammarly, Apple Writing Tools) that flattens everything to the same neutral voice, regardless of whether I'm messaging my wife or a client. Both interrupt whatever app I'm actually working in. Comparison vs Grammarly: Grammarly runs in the cloud and applies one house style everywhere. Lyra runs 100% on-device via llama.cpp with Metal, and builds a private local corpus of how you write to each specific person, so rewrites preserve your voice to that recipient instead of flattening it. vs Apple Writing Tools: Apple's rewrite is generic, one-shot, and doesn't learn anything about who you're writing to. Lyra is ambient (fires when you pause), recipient-aware, and includes reply-assist in Gmail and Outlook that drafts replies in your voice to that specific sender. Pricing Free: 500 rewrites per month. Pro: $9.99 per month or $99 per year. Link: heylyra.co.uk About me / transparency (per Tier 2) Solo dev based in London. LinkedIn: https://www.linkedin.com/in/pratik-garg-11537825 Contact: pratik.garg@codestackconsulting.co.uk. Privacy Policy: heylyra.co.uk/privacy. Terms: heylyra.co.uk/terms. Apple Silicon only, macOS 14+. Model runs locally: ~1GB on 12-19GB Macs (Qwen3-1.7B), ~3GB on 20GB+ Macs (Gemma 3B). Happy to answer questions.
已收集讨论
The per-recipient corpus is the interesting part and also the part I'd worry about. I've been building on-device stuff at roughly that size and the thing that kept biting me was context. A 1.7B model with a real corpus of how you write to one person eats the window fast, and once you're summarizing that corpus down to fit, you're back to a style hint rather than actual voice. So genuinely curious: are you feeding raw past messages in, or distilling them into a profile first? And does the Qwen3-1.7B tier hold voice as well as the Gemma 3B tier, or is there a noticeable drop where the small one reverts to neutral polite English? Asking because the small models were fine at local edits for me but got generic the moment I asked them to sustain a persona.
Really good question , tbh you're asking exactly the right thing. Short version: I'm feeding raw messages (few-shot exemplars), not distilling to a profile. And yes, the 1.7B tier does drop off in voice fidelity compared to Gemma 3B and not to the surprise quite noticeably. Longer version: The corpus is per-recipient SQLite of your own sent messages (harvested via AX from apps like WhatsApp/iMessage/Slack DMs/Teams 1:1s). Not the model's rewrites , your actual sent turns. On each rewrite, I retrieve N most recent exemplars for the current recipient, cap them at ~500 chars total, and inject as few-shot examples in the prompt. The rest of the window is the current input + system instruction. So window pressure is real but managed by exemplar count/length caps, not by summarising the corpus into a profile. Distilling to a profile was tempting because it scales ,but then every experiment I ran on it did exactly what you're describing: the distilled profile becomes "user writes casually, uses lowercase, prefers short sentences" which is generic advice the model applies loosely. Raw exemplars carry more signal per token because the model can pattern-match on syntax, punctuation quirks, and specific phrasings it wouldn't infer from prose about the user. On the tier drop: Gemma 3B holds voice meaningfully better than Qwen3-1.7B for anything beyond simple typo correction. Qwen3-1.7B on a small corpus (5-10 accepted rewrites for a recipient) reverts toward neutral polite English on any input with even mild ambiguity , exactly the failure mode you called out. It's genuinely a capability ceiling at 1.7B, not a prompt problem. I've been testing sampling, exemplar counts, prompt shape and all move the needle a few percent, none fix the fundamental drift. Current plan is cloud fallback via Gemini 2.5 Flash for cases where on-device output fails a quality gate. Free tier stays fully on-device. Pro users get access to the cloud model when they want it. Not shipped yet , the on-device story is the harder problem and I want to solve it properly first, not hide behind cloud. Would love your take on whether you found a way around the small-model drift, or if you also concluded the size ceiling was real and if you think users will be comfortable going to the cloud.
I see grammarly want to fix my code grammar. This should not. Some small UX problem can bring customer or churn them out.
This exact thing is why code surfaces (terminals, IDEs) are off by default in Lyra , prose correction firing inside code is worse than useless. You can turn it on per-app if you want it somewhere specific, but the default is: if you're writing code, Lyra stays out of the way. Totally agree that one wrong-context correction costs more trust than fifty good ones earn.
overlay looks a bit weird
Fair ,weird how, if you can pin it down, please? Position (where it appears vs the caret), the look of it, or the timing of when it pops in? Genuinely asking as the pill's whole job is to be unobtrusive, so "looks weird" is exactly the feedback I need specifics on. Thanks for taking time to give a feedback.
for the 1.7b tier, i'd try retrieval by message type instead of recency alone - questions, scheduling, disagreement, quick acknowledgements - then pair 2-3 raw examples with a tiny deterministic style vector like sentence length, casing, emoji rate and punctuation. keep a holdout set per recipient and test meaning plus voice drift against it. if the local result fails, i'd show 'couldn't preserve voice' instead of silently falling back. cloud should be an explicit per-rewrite choice that shows exactly which draft and exemplars will leave the Mac, since that corpus is the sensitive part.