Per-session llama sequence slots and their resident tokens. More...

#include </home/runner/work/entropic/entropic/src/inference/session_residency.h>

Public Member Functions
	SessionResidency (int slots=1)
	Size the pool.

int	slot_for (const std::string &key) const
	Slot currently holding this session, if any.

int	acquire (const std::string &key, std::string *evicted)
	Assign a slot to this session, evicting the least-recently used one when the pool is full.

const std::vector< Tok > &	resident (int slot) const
	Tokens recorded as resident in a slot.

void	set_resident (int slot, std::vector< Tok > tokens)
	Record what a slot now holds after a decode.

void	invalidate (int slot)
	Forget a slot's residency (its KV was cleared or invalidated).

void	invalidate_all ()
	Forget every slot's residency.

int	forget_session (const std::string &key)
	Drop one SESSION's residency and release its slot (gh#165).

int	slots () const
	Number of sequence slots in the pool.

std::size_t	assigned_count () const
	How many sessions currently hold a slot.

Detailed Description

template<typename Tok>
class entropic::SessionResidency< Tok >

Per-session llama sequence slots and their resident tokens.

Template Parameters

Tok	Token type (llama_token in production; int in tests).

Version: 2.12.0

Definition at line 54 of file session_residency.h.

Constructor & Destructor Documentation

◆ SessionResidency()

template<typename Tok >

entropic::SessionResidency< Tok >::SessionResidency ( int slots = 1 )

inlineexplicit

Size the pool.

Parameters

slots Number of llama sequences available (>= 1).

Version: 2.12.0

Definition at line 61 of file session_residency.h.

Member Function Documentation

◆ acquire()

template<typename Tok >

int entropic::SessionResidency< Tok >::acquire	(	const std::string &	key,
		std::string *	evicted
	)

inline

Assign a slot to this session, evicting the least-recently used one when the pool is full.

Parameters

	key	Session key.
[out]	evicted	Receives the key evicted to make room, empty when nothing was evicted. The caller must clear that slot's KV.

Returns: The slot now owned by key. @req REQ-INFER-019

Version: 2.12.0

Definition at line 88 of file session_residency.h.

◆ assigned_count()

template<typename Tok >

std::size_t entropic::SessionResidency< Tok >::assigned_count ( ) const

inline

How many sessions currently hold a slot.

Returns: Assigned session count. @utility

Version: 2.12.0

Definition at line 203 of file session_residency.h.

◆ forget_session()

template<typename Tok >

int entropic::SessionResidency< Tok >::forget_session ( const std::string & key )

inline

Drop one SESSION's residency and release its slot (gh#165).

invalidate(slot) needs a slot; a restore only knows a key, and after a restore the session should not hold a slot at all — it has no resident prefix worth keeping and another session may want the sequence.

Warm-keep is prefix-correct on its own (it reuses only the common token prefix and seq_rm's the divergent tail), so this is belt AND braces — deliberately. The reuse gate is token equality, not conversation identity, so a future change that coarsened it would silently decode a restored session against the prefix of the one it replaced. Dropping the residency makes the restore correct by construction rather than by an argument about the gate.

Parameters

key	Session key.

Returns: The slot that was released, or kNoSessionSlot when the key held none. The caller must clear that slot's KV cells. @req REQ-INFER-019 @req REQ-LOOP-010

Version: 2.13.0

Definition at line 180 of file session_residency.h.

◆ invalidate()

template<typename Tok >

void entropic::SessionResidency< Tok >::invalidate ( int slot )

inline

Forget a slot's residency (its KV was cleared or invalidated).

Parameters

slot	Slot index. @req REQ-INFER-019

Version: 2.12.0

Definition at line 139 of file session_residency.h.

◆ invalidate_all()

template<typename Tok >

void entropic::SessionResidency< Tok >::invalidate_all ( )

inline

Forget every slot's residency.

For the unconditional whole-context clears that still exist on some paths — after one of those, no slot holds what we think it does. @req REQ-INFER-019

Version: 2.12.0

Definition at line 153 of file session_residency.h.

◆ resident()

template<typename Tok >

const std::vector< Tok > & entropic::SessionResidency< Tok >::resident ( int slot ) const

inline

Tokens recorded as resident in a slot.

Parameters

slot	Slot index.

Returns: The slot's resident tokens; empty for an invalid slot. @req REQ-INFER-019

Version: 2.12.0

Definition at line 114 of file session_residency.h.

◆ set_resident()

template<typename Tok >

void entropic::SessionResidency< Tok >::set_resident	(	int	slot,
		std::vector< Tok >	tokens
	)

inline

Record what a slot now holds after a decode.

Parameters

slot	Slot index.
tokens	Tokens now resident. @req REQ-INFER-019

Version: 2.12.0

Definition at line 127 of file session_residency.h.

◆ slot_for()

template<typename Tok >

int entropic::SessionResidency< Tok >::slot_for ( const std::string & key ) const

inline

Slot currently holding this session, if any.

Parameters

key	Session key ("" is the default session).

Returns: Slot index, or kNoSessionSlot when not resident. @req REQ-INFER-019

Version: 2.12.0

Definition at line 72 of file session_residency.h.

◆ slots()

template<typename Tok >

int entropic::SessionResidency< Tok >::slots ( ) const

inline

Number of sequence slots in the pool.

Returns: Slot count (always >= 1). @utility

Version: 2.12.0

Definition at line 195 of file session_residency.h.

The documentation for this class was generated from the following file:

inference/session_residency.h

Public Member Functions

Detailed Description

Constructor & Destructor Documentation

◆ SessionResidency()

Member Function Documentation

◆ acquire()

◆ assigned_count()

◆ forget_session()

◆ invalidate()

◆ invalidate_all()

◆ resident()

◆ set_resident()

◆ slot_for()

◆ slots()