Anything better for coding than Qwen 27B Q8 at around 90GB (~42GB RAM, ~48GB VRAM)?

Can confirm Qwen3.6 27b FP8 is the most usable at that memory level other than maybe the same model in BF16. Goes to show maybe we went the wrong way with MoE. It’s also the best I can run with 192gb vram right now I find.

What type of coding? Agentic or just chatting?

I'd say qwen would be much faster on your hardware if it fits fully on vram. Offload tanks the speed a lot, I'd guess at least halving it for you. I can put either fully on vram and Qwen is faster at Q4 and similar speed at Q8. I also tried Mimo and step3.7, they're smart for sure but undercooked compared to qwen or ds4. Like they're smart but loop and stuff so I got annoyed and deleted them. Depending on your requirements and patience they might be worth trying out.

27B Q8 is definitely better than both of those other older Qwens IMO. I think it's even better than Minimax generally speaking. I wasn't impressed with it. 27B was lightning in a bottle. It's pretty incredible for the size. The only model that I think is better for coding (ignoring insanely huge ones) is Laguna S 2.1 , but OP would have to run such an extreme quant that 27B is still going to be a lot better.

Laguna is an moe so it wouldn't be too bad

I’ve tried both, I don’t find them to be as reliable though. I’ll check at Laguna S 2.1 again with the fixed chat template but I didn’t find it very impressive. With an nvl72 those companies could for sure run a 1-2T dense model for many concurrent users. Maybe its not that the model is dense, maybe its the BF16, or just better suited for my needs, but it feels like its one big coding focused expert with a bit of understanding of everything else, as opposed to activating 13B specialized params in a lower quantization.

Not as reliable though I find. I answered the same in another comment, but it’s like one big 27b coding expert with global general knowledge baked in too, running at bf16, vs activating smaller specialized dense networks for a total of 13B active q4 params in deepseek’s case.

Fire it up and let us know

I thought "not following instructions" was the main complaint about DS4? It completely ignores me at times. As for quality, I don't know about Q2, but the original weights have managed to do tasks for me that even qwen 397b failed to accomplish.

We really have no choice but to go MoE going forward. Can you imagine how slow a 1-2T dense model would be even running in an OpenAI or Anthropic datacenter? It’s also the best I can run with 192gb vram right now I find. Have you tried DeepSeek V4 Flash or Laguna S 2.1 with updated chat template?

But what if OP gets 128 GB of ram. Then they can run laguna s2.1 at q5. I currently run it on my system with 5090+ 3090 + 64GB 7200 mhz ram and I get 20 tok/s gen speed

Not at 90GB, maybe you can try Qwen3 Next or Qwen 3.5 122B but 27B Q8 is probably better. Next major step up would be Minimax M2.7 that's around 130-140GB. But then you may as well get 192GB or more and try DSv4 Flash.

Not deepseek flash? Uses a ton more but the full weights should squeeze into 192 can they not

DS4 Q2 is kinda better, but not better better strictly. It has more ideas/knowledge trained in and follows instructions better imo, so it figures things out that qwen has no idea how to deal with (and I don't know either, lol like gui stuff). However, I think most of the time qwen can do what's needed with harness and prompting pretty easily, so I choose qwen when I can. More reliable tool calls (ds inference still has bugs, qwen too lol but mostly figured out), less memory used so I can do other stuff at the same time. Never did a one to one code comparison.

ds4 at q4 should fit in 192gb, it only takes a small amount of ram for context.

no there is nothng as of now ...when i test qwen 3.6 27b (q9 inferencerlabs quant locally) against deepseek v4 flash and mimo v2.5 (from opencode go accounts api) . 7/10 times grok says qwens code ie better that deepseek and 6/10 times better than mimo 2.5 , which i can swear was not the case when deepseek and mimo were released (they were clearly ahead )...by the way my coding work involves stock market related backtests (pinescript and python )....so sorry to break it to you but the theres nothing in between to fill the void ...i have a mac 128gb by the way yet i cant find something which is "definitively or significantly" better than qwen 3.6 27b 😭 Any one saying otherwise is just speculating without actually comparing them head to head

If you look here (https://livebench.ai/) at open weight models then there's not much between Qwen 27b and DS4, it looks like DS4 is better with languages and with instruction following but basically the same on everything else. Also, DS4 at q2 is going to be degraded by say 10-20%, whereas Qwen 27b at 8bit should perform roughly the same as the benchmark, so overall I think Qwen 27b q8 will be better than DS4 at q2. Livebench is a good measurement of models real world performance in my experience.

No, running a big model (like DS or Laguna) at Q2 is generally worse for coding than running a 27B model at Q6~Q8.

Stick with 27B Q8. The quantization hit below Q4 on big models isn't worth it for code — you trade accuracy for... more parameters guessing wrong. If you can squeezeQwen 32B at Q4 that might be worth testing, but honestly the 27B Q8 is a solid daily driver at that size.

Not really. For that kind of system, 27B is still going to be the king. You'll need to upgrade your hardware for better unfortunately.

On blackwell you can do deepseek „3.9” bit https://github.com/kacper-daftcode/vLLM-Moet

Laguna S 2.1