gh#158: which sequences a same-prefix batch may touch, purely. More...

#include <cstddef>
#include <vector>

Include dependency graph for batch_kv_util.h:

This graph shows which files directly or indirectly include this file:

Classes
struct	entropic::BatchKvPlan
	Which sequences a batch will write, and what it may clear. More...

Namespaces
namespace	entropic
	Activate model on GPU (WARM → ACTIVE).

Functions
BatchKvPlan	entropic::plan_batch_kv (std::size_t n, int session_slots, const std::vector< int > &temp_ids)
	Plan the sequence ids a same-prefix batch may use.

bool	entropic::plan_disturbs_sessions (const BatchKvPlan &plan, int session_slots)
	Whether a plan can disturb any session's resident KV.

Detailed Description

gh#158: which sequences a same-prefix batch may touch, purely.

The bug this encodes the fix for: run_batched_decode opened with llama_memory_clear(mem, true) — the WHOLE KV cache, every sequence, not only the ones the batch is about to write — and followed it with invalidate_all_resident_kv(). With one run per handle that was merely wasteful: nobody else held cells. Once runs are keyed per session (gh#158), a batch issued by session A destroys session B's resident prefix and B's next turn silently pays a full cold prefill having been told nothing.

It also took sequence 0 for its own first arm, which IS a session slot — so the damage did not even need the whole-cache clear to happen.

Why a pure header: The same reason session_pool_util.h and warm_keep_util.h are: the rule is integer bookkeeping that decides whether one caller can corrupt another's state, and it must be assertable by a CPU unit test with no model, no GPU and no llama_context. run_batched_decode APPLIES this plan; it does not decide it. A test that asserted against a helper the production path did not consult would prove nothing.

Definition in file batch_kv_util.h.

Classes