Yet to see an OCR app like this
mistral.ai/news/mistral-ocr Mistral OCR is arguably the most advanced OCR model out there right now, and yet I can’t find a single Mac app or Raycast extension that connects to it. That seems like a huge miss. OCR on Mac is still stuck with the same standard built-in option, which is fast, very reliable but super limited. It even handles images, but an extension that skips that and just gives Markdown would also be totally fine. My current workflow is clunky: Take a screenshot Hand it to an AI model Copy the text Paste it back where I need it That’s a lot of swiping between tabs for something that should be one shortcut. A simple menu-bar app or Raycast extension that sends a screenshot to Mistral OCR and drops clean Markdown on my clipboard would be incredible. Has anyone built this, or is anyone interested in building it?
Collected discussion
I’ve been testing ZenOCR which outputs to markdown. It’s not capable of parsing diagrams, but it’s quite good at formulas. Not affiliated with the dev, just like the idea.
You know, that could be useful. I worry that since the app is drag and drop, it may be tedious compared to just hitting a shortcut: screenshotting and then getting the text on my keyboard as markdown.
Hello, i'm ZenOCR dev here, the latest version it already support use shortcut to screenshot. Feel free to try it. You can download in Mac App store and have a free trial. 😉
Hello, i'm ZenOCR dev. Recently just I added the screenshot feature to the latest ZenOCR. it now support 2 screenshot mode, fast mode to use apple vision get some text quickly, and AI model use local AI model to copy complex layout, latex and handwritings. And it's 100% Local Feel free to try it. You can download in Mac App store and have a free trail. 😉 http://zenocr.com/
Does it support Greek ocr?
i found a shortcut for that somewhere on the internet so now I just open spotlight, type ocr and hit enter to select text
That's cool. Maybe use ScreenOCR?
Does it uses Mac vision API? Looks so cool
No, unfortunately. This is Mistral OCR; you should check it on the website. It has its own sort of AI model that I'd love to see in more third-party apps.
Very cool, I remember Mistral OCR was big thing when they launched. I'll definitely check this out!
I just use cmd shft T with my cleanshotx
This is very useful when working.
Wow! I didn't know an OCR model this good was available. I'll play around with it today. Maybe, I'll build something with it. :)
Is that pulling out the contents of the tables as well? I'd be really interested to see whether that is all OCR or if there is any peeking inside the PDF as well. One of the problems with PDFs is that sometimes they contain useful info, and sometimes they're just images on every page.
I’ve built a very simple TUI app for Mistral (for myself) if you’re interested.
Check out Easydict (free), a menubar translator with Input/Screenshot/Select/OCR
Looks nice
I think the product shape here should probably be tiny, not a full document manager. For the workflow you described, the clean version is: screenshot/selected image -> OCR provider -> Markdown preview or direct clipboard -> paste wherever you were working The important distinction is that Mistral OCR is cloud, unlike Apple Vision. So I’d want the app/extension to make that boundary very explicit: BYOK API key no saved screenshot history by default optional local-only mode using Apple Vision for less sensitive stuff preview before clipboard replacement, especially for tables/math clear failure state when OCR confidence/structure is weak A Raycast extension or menu-bar app feels like the right surface. A library/archive app would probably be too heavy for this use case.
If you specifically want structured content out of research papers check out the grobid project https://grobid.readthedocs.io/en/latest/Introduction/
!remindme 2 weeks
I am the developer of FrameOCR and I think my app handles your desired use case. I am not using Mistral OCR but another custom VLM that is similarly powerful. However, my app runs completely offline and still delivers Markdown output. It can tell headers and paragraphs apart, along with math formulas and tables, packaged in the menubar app flow that we Mac users know and love. I avoided using Apple's vision framework because of how limited it is and how far it is from state-of-the-art OCR models. I welcome you to try the free trial!
This comment was deleted at the source.
The most obvious disadvantage of the Apple model is that it doesn’t handle a lot of languages in comparison