Generation parameters for a single inference call. More...

#include <entropic/types/config.h>

Public Attributes
float	temperature = 0.7f
	Sampling temperature.

float	top_p = 0.9f
	Nucleus sampling threshold.

int	top_k = 40
	Top-K sampling.

float	repeat_penalty = 1.1f
	Repetition penalty.

float	min_p = 0.0f
	Min-p nucleus sampling threshold (gh#23 MVP item 1).

float	presence_penalty = 0.0f
	Presence-penalty term in llama.cpp's penalties sampler (gh#23 MVP item 2).

std::unordered_map< int32_t, float >	logit_bias
	Per-token logit bias map (gh#23 MVP item 4).

float	frequency_penalty = 0.0f
	Frequency-penalty term in llama.cpp's penalties sampler (gh#23 MVP item 3).

int	max_tokens = 4096
	Maximum tokens to generate.

int	seed = -1
	RNG seed for reproducible sampling.

int	reasoning_budget = -1
	Per-call think budget override (-1 = unlimited)

bool	enable_thinking = true
	Enable <think> blocks (false if reasoning_budget == 0)

std::string	grammar
	GBNF grammar string (empty = unconstrained)

std::string	grammar_key
	Grammar registry key.

std::vector< std::string >	stop
	Stop sequences.

std::string	tool_call_mode
	Per-call tool-call generation mode (gh#103).

int	logprobs = 0
	Top log-probs per token (0 = disabled)

int	time_limit_ms = 0
	Wall-clock time cap in milliseconds.

std::string	profile
	GPU resource profile name.

bool	auto_adapt = true
	Enable throughput-based max_tokens auto-adaptation.

float	adapt_headroom = 0.9f
	Target time usage fraction for auto-adaptation.

std::string	tools
	Active tool definitions for this turn, as an MCP tool-list JSON array (`[{name, description, inputSchema}, ...]`), already filtered per-tier by the caller.

Detailed Description

Generation parameters for a single inference call.

Version: 2.3.16 — added logit_bias (gh#23 MVP item 4)

Definition at line 302 of file config.h.

Member Data Documentation

◆ adapt_headroom

float entropic::GenerationParams::adapt_headroom = 0.9f

Target time usage fraction for auto-adaptation.

0.9 means "use at most 90% of time_limit_ms for generation".

Version: 1.9.7

Definition at line 398 of file config.h.

◆ auto_adapt

bool entropic::GenerationParams::auto_adapt = true

Enable throughput-based max_tokens auto-adaptation.

When true, the orchestrator may reduce max_tokens to fit within time_limit_ms based on recent throughput measurements. Ignored if time_limit_ms == 0.

Version: 1.9.7

Definition at line 393 of file config.h.

◆ enable_thinking

bool entropic::GenerationParams::enable_thinking = true

Enable <think> blocks (false if reasoning_budget == 0)

Definition at line 358 of file config.h.

◆ frequency_penalty

float entropic::GenerationParams::frequency_penalty = 0.0f

Frequency-penalty term in llama.cpp's penalties sampler (gh#23 MVP item 3).

Subtracts a per-occurrence linear amount from any token that has appeared in the recent window — penalizes by COUNT rather than presence. 0.0 (default) disables — preserves pre-v2.3.15 chain bit-for-bit. Typical range: 0.0–2.0. Pairs with repeat_penalty (multiplicative) and presence_penalty (per-presence constant); all three run in one llama_sampler_init_penalties call. Any non-default value here activates the penalties stage even when repeat_penalty == 1.0 and presence_penalty == 0.0.

Version: 2.3.15

Definition at line 349 of file config.h.

◆ grammar

std::string entropic::GenerationParams::grammar

GBNF grammar string (empty = unconstrained)

Definition at line 359 of file config.h.

◆ grammar_key

std::string entropic::GenerationParams::grammar_key

Grammar registry key.

Resolved to GBNF content by orchestrator before passing to the backend. If both grammar and grammar_key are set, grammar (raw string) takes precedence.

Version: 1.9.3

Definition at line 364 of file config.h.

◆ logit_bias

std::unordered_map<int32_t, float> entropic::GenerationParams::logit_bias

Per-token logit bias map (gh#23 MVP item 4).

Maps token id → additive bias (in logit-space). Applied at the start of the sampler chain (before penalties), so it shapes the post-softmax distribution that every downstream filter (top-k, top-p, min-p, etc.) sees. Common uses:

Suppress a token: bias = -INFINITY (or a large negative value like -100 if -INFINITY doesn't survive JSON).
Force a token: bias = +INFINITY (or large positive).
Subtle nudges: bias = ±1.0..±5.0. Empty map (default) disables the stage entirely — preserves pre-v2.3.16 chain shape bit-for-bit.
Version
2.3.16

Definition at line 336 of file config.h.

◆ logprobs

int entropic::GenerationParams::logprobs = 0

Top log-probs per token (0 = disabled)

Definition at line 373 of file config.h.

◆ max_tokens

int entropic::GenerationParams::max_tokens = 4096

Maximum tokens to generate.

Definition at line 351 of file config.h.

◆ min_p

float entropic::GenerationParams::min_p = 0.0f

Min-p nucleus sampling threshold (gh#23 MVP item 1).

Filters tokens whose probability is below min_p * P(top_token). 0.0 (default) disables — preserves pre-v2.3.10 behavior bit-for-bit. Typical productive range: 0.01–0.2. Values >= 1.0 filter every non-top token (degenerate but not crash-inducing; the dist sampler still selects the single survivor).

Version: 2.3.10

Definition at line 315 of file config.h.

◆ presence_penalty

float entropic::GenerationParams::presence_penalty = 0.0f

Presence-penalty term in llama.cpp's penalties sampler (gh#23 MVP item 2).

Subtracts a constant from any token that has appeared at least once in the recent window. 0.0 (default) disables — preserves pre-v2.3.14 chain bit-for-bit. Typical range: 0.0–2.0.

Version: 2.3.14

Definition at line 322 of file config.h.

◆ profile

std::string entropic::GenerationParams::profile

GPU resource profile name.

Resolved to GPUResourceProfile by the orchestrator before passing to the backend. Empty string means use the "balanced" profile.

Version: 1.9.7

Definition at line 386 of file config.h.

◆ reasoning_budget

int entropic::GenerationParams::reasoning_budget = -1

Per-call think budget override (-1 = unlimited)

Definition at line 357 of file config.h.

◆ repeat_penalty

float entropic::GenerationParams::repeat_penalty = 1.1f

Repetition penalty.

Definition at line 306 of file config.h.

◆ seed

int entropic::GenerationParams::seed = -1

RNG seed for reproducible sampling.

-1 = random (default). Maps to LLAMA_DEFAULT_SEED when negative. (P2-14)

Version: 2.0.6-rc16

Definition at line 356 of file config.h.

◆ stop

std::vector<std::string> entropic::GenerationParams::stop

Stop sequences.

Definition at line 365 of file config.h.

◆ temperature

float entropic::GenerationParams::temperature = 0.7f

Sampling temperature.

Definition at line 303 of file config.h.

◆ time_limit_ms

int entropic::GenerationParams::time_limit_ms = 0

Wall-clock time cap in milliseconds.

Generation is cancelled if this limit is reached. 0 = no time limit (default).

Version: 1.9.7

Definition at line 380 of file config.h.

◆ tool_call_mode

std::string entropic::GenerationParams::tool_call_mode

Per-call tool-call generation mode (gh#103).

Empty = defer to tier/default ("batch"). "sequential" → the orchestrator appends the resolved family tool-call close marker to stop at stage time, so the decode loop halts at the FIRST closed tool call (one tool/turn, observe-between-actions). "batch" = no injection (parallel calls).

Version: 2.8.2

Definition at line 372 of file config.h.

◆ tools

std::string entropic::GenerationParams::tools

Active tool definitions for this turn, as an MCP tool-list JSON array ([{name, description, inputSchema}, ...]), already filtered per-tier by the caller.

gh#87 (v2.7.0): structured tool defs flow here instead of being string-injected into the system message. The orchestrator stages them on the backend (LlamaCppBackend::set_active_tools) so the common_chat render emits — and parse extracts — the model's native tool-call wire format. Empty (default) = no tools this turn (the backend renders without a tool grammar).

Version: 2.7.0

Definition at line 411 of file config.h.

◆ top_k

int entropic::GenerationParams::top_k = 40

Top-K sampling.

Definition at line 305 of file config.h.

◆ top_p

float entropic::GenerationParams::top_p = 0.9f

Nucleus sampling threshold.

Definition at line 304 of file config.h.

The documentation for this struct was generated from the following file:

entropic/types/config.h

Public Attributes