Prompt caching configuration. More...

#include <entropic/types/config.h>

Public Attributes
size_t	max_bytes = 536870912
	Maximum cache RAM (512 MB default)

bool	enabled = true
	Master switch (false = no caching)

bool	log_hits = true
	Log cache hit/miss at INFO level.

bool	warm_keep = true
	gh#96 (v2.7.5): keep the prior turn's KV resident and re-decode only the appended delta (warm-keep / incremental prefill) instead of re-decoding the whole post-system history every turn.

Detailed Description

Prompt caching configuration.

Controls host-memory KV cache prefix storage for system prompts. Setting enabled=false or max_bytes=0 both disable caching.

YAML key: inference.prompt_cache

Version: 1.8.3

Definition at line 266 of file config.h.

Member Data Documentation

◆ enabled

bool entropic::PromptCacheConfig::enabled = true

Master switch (false = no caching)

Definition at line 268 of file config.h.

◆ log_hits

bool entropic::PromptCacheConfig::log_hits = true

Log cache hit/miss at INFO level.

Definition at line 269 of file config.h.

◆ max_bytes

size_t entropic::PromptCacheConfig::max_bytes = 536870912

Maximum cache RAM (512 MB default)

Definition at line 267 of file config.h.

◆ warm_keep

bool entropic::PromptCacheConfig::warm_keep = true

gh#96 (v2.7.5): keep the prior turn's KV resident and re-decode only the appended delta (warm-keep / incremental prefill) instead of re-decoding the whole post-system history every turn.

Default on; the oracle-equality tests toggle it off as the ground-truth comparison.

Definition at line 274 of file config.h.

The documentation for this struct was generated from the following file: