entropic::TokenCounter Class Reference

Track token usage across conversation. More...

#include <entropic/core/compaction.h>

Public Member Functions
	TokenCounter (int max_tokens)
	Construct a token counter.
int	count_message (const Message &msg) const
	Count tokens in a single message.
int	count_messages (const std::vector< Message > &messages) const
	Count total tokens in a message list.
float	usage_percent (const std::vector< Message > &messages) const
	Get usage as fraction of context window (0.0–1.0).
void	clear_cache ()
	Clear the token count cache.

Public Attributes
int	max_tokens
	Maximum context window size.

Detailed Description

Track token usage across conversation.

Uses simple heuristic (~4 chars per token) for estimation. More accurate counting requires the model's tokenizer.

Version

1.8.4

Definition at line 32 of file compaction.h.

Constructor & Destructor Documentation

TokenCounter()

entropic::TokenCounter::TokenCounter ( int  max_tokens )

explicit

Construct a token counter.

Parameters

max_tokens

Maximum context window size.

Version

1.8.4

Parameters

max_tokens

Maximum context window size.

Definition at line 26 of file compaction.cpp.

Member Function Documentation

clear_cache()

void entropic::TokenCounter::clear_cache

(

)

Clear the token count cache.

Version

1.8.4

Definition at line 98 of file compaction.cpp.

count_message()

int entropic::TokenCounter::count_message

(

const Message &

msg

)

const

Count tokens in a single message.

Parameters

msg	Message to count.

Returns

Estimated token count.

Version

1.8.4

Parameters

msg	Message to count.

Returns

Estimated token count (content + role overhead).

Definition at line 50 of file compaction.cpp.

count_messages()

int entropic::TokenCounter::count_messages

(

const std::vector< Message > &

messages

)

const

Count total tokens in a message list.

Parameters

messages

Messages to count.

Returns

Total estimated token count.

Version

1.8.4

Parameters

messages

Messages to count.

Returns

Total estimated token count.

Definition at line 68 of file compaction.cpp.

usage_percent()

float entropic::TokenCounter::usage_percent

(

const std::vector< Message > &

messages

)

const

Get usage as fraction of context window (0.0–1.0).

Get usage as fraction of context window.

Parameters

messages

Messages to measure.

Returns

Usage fraction.

Version

1.8.4

Parameters

messages

Messages to measure.

Returns

Usage fraction (0.0 if max_tokens is 0).

Definition at line 84 of file compaction.cpp.

Member Data Documentation

max_tokens

int entropic::TokenCounter::max_tokens

Maximum context window size.

Definition at line 71 of file compaction.h.

---

The documentation for this class was generated from the following files:

• entropic/core/compaction.h
• core/compaction.cpp