entropic::BatchKvPlan Struct Reference

Which sequences a batch will write, and what it may clear. More...

#include </home/runner/work/entropic/entropic/src/inference/batch_kv_util.h>

Public Attributes
std::vector< int >	seq_ids
	Sequence ids the batch will write, one per request.
bool	clears_whole_cache = false
	Whether the batch may clear the WHOLE KV cache.

Detailed Description

Which sequences a batch will write, and what it may clear.

Version

2.13.0

Definition at line 40 of file batch_kv_util.h.

Member Data Documentation

clears_whole_cache

bool entropic::BatchKvPlan::clears_whole_cache = false

Whether the batch may clear the WHOLE KV cache.

Always false from v2.13.0. Kept as a named field rather than deleted so the property is asserted rather than assumed — a future change that reintroduces the whole-cache clear has to set it, and the test that forbids it fails.

Definition at line 50 of file batch_kv_util.h.

seq_ids

std::vector<int> entropic::BatchKvPlan::seq_ids

Sequence ids the batch will write, one per request.

Definition at line 42 of file batch_kv_util.h.

---

The documentation for this struct was generated from the following file:

• inference/batch_kv_util.h