Many tool calls in one go causing kv cache checkpoint misses
I have found that whatever software you are using: open web ui, openclaw, codex; if a model does many tools calls in one turn, something happens that causes checkpoints that are created in and around those tool calls to not be valid when checked the following…
I have found that whatever software you are using: open web ui, openclaw, codex; if a model does many tools calls in one turn, something happens that causes checkpoints that are created in and around those tool calls to not be valid when checked the following turn. They get discarded and the whole session is re-processed from either the last valid checkpoint before the tool calls, or from zero if there are none. However, a single tool call, maybe even two, does not cause this behaviour. I have observed this in llama.cpp and in ds4. Does anyone have any idea why this happens and a way to fix it?
已收录讨论
Is this on a resumed chat after multiple tool calls on a single prompt or noticed after fresh chat
It’s in a chat as it happens. I don’t leave the chat. I’m sitting there, chatting with openclaw or whatever, watching the llama.cpp/ds4 logs. I’ll ask the agent to do something, it does loads of tool calls, comes back to me finishing its turn. When I then message back, all those checkpoints created around the tool calls get checked and discarded, and it goes back to processing from zero (or the most recent pre-tool call checkpoint).
not sure if accurate but this is a breadcrumb from opus5 This is a known llama.cpp issue, not something you're doing wrong. Two separate mechanisms, and both get triggered by multi-tool-call turns. Cause 1 — checkpoint restore is broken on SWA/hybrid-recurrent models. Hybrid/recurrent models (Gated DeltaNet architecture, e.g. Qwen3.5-27B and Qwen3.6-27B) force full prompt re-processing on every conversation turn, because the checkpoint search uses cur.pos_min < pos_min_thold, and for recurrent models pos_min always equals the full sequence length, so the check always fails and no checkpoint is ever restored. The log signature is exactly what you're describing: "forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory)" followed by a run of "erased invalidated context checkpoint" lines. This is your 27B/35B MoE stack. GitHubMedium Cause 2 — checkpoint budget/spacing. Defaults are 32 checkpoints per slot (-ctxck) with a minimum spacing of 8192 tokens between them (-cms). A single tool call adds a few hundred tokens — under the spacing threshold, so no checkpoint is written where you need one. A burst of 10 tool calls crosses boundaries and evicts. Also, a recent change moved checkpoint creation to message boundaries only, and users report the server now creating only two checkpoints which are always discarded. GitHubGitHub Fastest fix — try in this order: Add to your llama-server args: -cms 0 -ctxck 64 (spacing off, more checkpoints). Restart, watch logs. If still full-reprocessing and the log mentions SWA: add --swa-full. Costs VRAM, kills the problem outright. If the model is hybrid/recurrent (Qwen3.6-27B is), 1 and 2 won't save you on older builds — you need the fix from issue #22384. Check whether your build has it; you were on b9860, the issue landed around Apr–Jun 2026, so pull latest and rebuild with your pinned command. Confirm --cache-ram isn't 0 and that cache_reuse isn't being disabled at load (log line: cache_reuse is not supported by this context, it will be disabled). The reason a single tool call survives is that it doesn't move you past a checkpoint boundary — the raw prefix match still holds. Nothing needs restoring. Worth noting separately: even with checkpoints fixed, any harness that re-serializes tool_call JSON differently on the next turn (key order, whitespace, escaping) or strips prior reasoning blocks will diverge the token prefix at the first tool call and invalidate everything after. If the flags above don't fix it, dump the rendered prompt on turn N and turn N+1 and diff them.
are you using session state or freeze? i just got that sorted last week on my custom harness.. ask your model about them 😉
I'm having this issue with session state as well. I mainly use state for accidental page refreshes or after breaks but returning to the chat if I don't clear it the tools won't call until I clear that chat completely. Tried many things yet still unresolved. What did you do?
The usual culprit is a parse -> re-render round trip that isn't byte-identical. During the turn, the tool-call tokens sitting in the cache are whatever the model actually sampled. Next turn, the client re-renders the whole conversation through the chat template from the parsed representation of those calls, and the template output rarely matches what the model emitted: JSON key order, whitespace, separators injected between multiple calls, or regenerated call ids (some harnesses mint a new id on every render, which guarantees a mismatch). A single call often survives because the simple path happens to serialize identically; multiple calls hit the array-formatting/separator logic and diverge. Way to confirm: dump the prompt tokens on turn N and N+1 and diff them (llama.cpp with --verbose-prompt, or compare the tokenized prefix in the slot logs). The first diverging token is where your checkpoint dies, and I'd bet it lands right at the first tool-call boundary. Fixes that have worked for me: keep the model's raw emitted text verbatim in the transcript instead of parse-and-re-render, or patch the chat template so re-rendered tool calls serialize exactly like generation time, and check whether your harness regenerates tool_call ids per turn.