REDDIT 原始帖子
Did anyone notice odd reasoning loops with DeepSeek v4 flash 0731?
I'm running the original model through vLLM. The model can accomplish its tasks and overall seems good enough. However, if I inspect the output traces, it sometimes goes in loops like the one in the image. I noticed that at some point it manages to escape…
I'm running the original model through vLLM. The model can accomplish its tasks and overall seems good enough. However, if I inspect the output traces, it sometimes goes in loops like the one in the image. I noticed that at some point it manages to escape them though. I was wondering if this is a misconfiguration on my end, or did someone else experience this?
已收录讨论
It looks like it is trying to call a tool in its reasoning. There was a fix allowing that in llama.cpp, so maybe there's that? Also, you're not using KV cache quants I hope, this might lower its quality. Anyway, I don't have much exp with this model yet, but I have been playing with full and Q2 variants a bit over last few days and didn't have such issues. Altho, on Max reasoning mode, it really can go all in on thinking tokens, lol, but I din't notice any loops yet or failed tool calls.
I had a good laugh ahahah
Ok, I guess that if this is a model issue, the quant version would show it even more. So this could mean more that it's a misconfiguration on my end, ty :)
I quant an experiment 2bpw class and it behaves like that. 80%ish argmax - thats the problem.
I think q8 is recommended for deepseek kv
Yes, that's what I'm using
It does, but I find that what comes out is generally worth it most of the time. I would use something else for smaller stuff that doesn't require a lot of reasoning.
Try XSS, I feel like there is something wrong with IQ3_S quant.
me whenever i open my fridge
Honestly I'm struggling with dwarfstar, it could produce 9k tokens in thinking before answering ...
I use DwarfStar and the q2-q4 0731 version and it does not behave like that. I reasons a lot and sometimes it goes back over the same topic but it looks as it actually improves the outcome. Never like that.
Yes I do. I experience the same weird loop from Unsloth IQ3 S model, XSS size rarely does it. Both models are loaded completed into vram. The looping is unusable.
It happened to me with unsloth IQ3_S with KV cache at Q8 and High reasoning in llama cpp. It run out of context after 80k tokens. The last 40k of reasoning was just repeating around 15 lines over and over again. I let it run hoping it would recover, but it never did. It was super detailed and was doing amazingly well until it started looping.
Bro is stuck in an existential crisis.
Got the same issue it starts breaking down after 30 to 40K with CC, I run a modified version of oMLX so implementation over 2 Mac’s with distributed inference …
What quant? I suspect Q4 is not enough, it will hold up to 40K context but beyond that that … seems Q8 is the one reliable in big context, would be nice if someone could confirm this. (KV cache needs to be BF16 so quality degrades over long context)