Laguna S.21 emits </think> no matter what, on FP8 and NVFP4. How did you get around this?
So after trying Laguna for several days, I see that regardless of the quantization I try, the model emits </think> and basically stops itself from reasoning. I made sure to follow every bit of instruction, use the right template, the necessary flags, and…
So after trying Laguna for several days, I see that regardless of the quantization I try, the model emits </think> and basically stops itself from reasoning. I made sure to follow every bit of instruction, use the right template, the necessary flags, and Poolside's settings, either in vLLM or their own fork of llama.cpp. The only way to bypass this, that I can see, is to append a forced "think" string so the model doesn't just immediately follow with "</think>". This works, but it means the model is ALWAYS reasoning even for super simple prompts like "Hello". These are the models I tried: NVFP4, served via vLLM: - Model: https://huggingface.co/poolside/Laguna-S-2.1-NVFP4 - Draft: https://huggingface.co/poolside/Laguna-S-2.1-DFlash-NVFP4 Q8_0, served via Poolside's llama.cpp: - Model: Q8_0 from https://huggingface.co/poolside/Laguna-S-2.1-GGUF - Draft: laguna-s-2.1-DFlash-BF16.gguf Using OpenCode with these models, even with the reasoning hack, is wonky. They sometimes just stop mid-generation despite having plenty of context left. What am I missing in serving these?
Collected discussion
I was completely unimpressed after 2 attempts with the model I think it's just not very good
Yes, I redid everything from scratch today, so I believe so.
I’m pretty sure this model is broken in some environments.
I was completely unimpressed after ~9 attempts with the model, including with the UD-Q6_K_XL Unsloth version, which is much higher quality than NVFP4. Now trying DeepSeek-V4-Flash-0731 @ UD-IQ3_S and it’s a proper model, even though it likes to think A Lot, but I am fine with that.
Was one of the few recent models that was supported by vllm on sm120.
But whatever harness or agent using this model still needs to work it somehow, either Pi or Pool or what have you. So if they can make it work, it should be configurable in OpenCode.
Yup. Another comment validated. Poolside just updated every quantization they made, again. And proceeded to add 10GB+ while doing it. Can't really say I'd trust their quants at all now.
So apparently Laguna is made to not have a global thinking parameter. Thinking is supposed to be set per conversation, as needed, by the harness its using. Poolside recommends using their own Pool harness with this model. Apparently Pi can also do it. If you're using OpenCode you'll get nowhere (I struggled for hours and am giving up)
Just making sure you’re using their updated template? I think they had a template patch just after release
I would get around this by avoiding this model entirely
I'll wait for Chinese models honestly Laguna is a chore to set up
You give up and move on.
Don't enable thinking. When enabled it always outputs </think> at the start, but when it's not it doesn't, gives a clean output. I experimented with it, trying to make it think because I wasn't happy with the quality of the output, but regardless of any parameters it decides on its own whether to think or not. And it's obviously not very good in that estimate...
Laguna is bad, overhyped garbage.
Try another quant. There's an AWQ and an Autoround. I think it's a nice gesture that poolside provided some quants, but fact is others in the community make way better quants. Way more experienced. Like I could never get gwen's fp8 to work for me but had no issues with 3rd party quants
I am running AtomicChat/Laguna-S-2.1-MLX-8bit on M3 Ultra with Pi Cli. Loaded from LM Studio, it was just like OP has described and useless. When I serves it from oMLX and it has been solid on every task I prompted with on a large codebase. Hope this info is useful to someone.
Been using it with llama.cpp and it works fine, but the unsloth quant, not the models makers, it seems to be problematic. That aside, the model isn't very good, I've used it for a day or two and just switched back to qwen3.6 again
Maybe fine tune it? Usually models do think even before simple requests like Hi