- Generated by
1.9.8
|
Entropic 2.14.3
Local-first agentic inference engine
|
Generation parameters for a single inference call. More...
#include <entropic/types/config.h>
Public Attributes | |
| std::string | session_key |
| Caller-scoped session this generation belongs to (gh#144). | |
| float | temperature = 0.7f |
| Sampling temperature. | |
| float | top_p = 0.9f |
| Nucleus sampling threshold. | |
| int | top_k = 40 |
| Top-K sampling. | |
| float | repeat_penalty = 1.1f |
| Repetition penalty. | |
| float | min_p = 0.0f |
| Min-p nucleus sampling threshold (gh#23 MVP item 1). | |
| float | presence_penalty = 0.0f |
| Presence-penalty term in llama.cpp's penalties sampler (gh#23 MVP item 2). | |
| std::unordered_map< int32_t, float > | logit_bias |
| Per-token logit bias map (gh#23 MVP item 4). | |
| float | frequency_penalty = 0.0f |
| Frequency-penalty term in llama.cpp's penalties sampler (gh#23 MVP item 3). | |
| int | max_tokens = 4096 |
| Maximum tokens to generate. | |
| int | seed = -1 |
| RNG seed for reproducible sampling. | |
| int | reasoning_budget = -1 |
| Per-call think budget override (-1 = unlimited) | |
| bool | enable_thinking = true |
| Enable <think> blocks (false if reasoning_budget == 0) | |
| std::string | grammar |
| GBNF grammar string (empty = unconstrained) | |
| std::string | grammar_key |
| Grammar registry key. | |
| std::string | resolved_grammar_key |
| The key or frontmatter stem that was NAMED for this call, set by ModelOrchestrator::resolve_grammar_key (gh#154). | |
| bool | grammar_from_tier = false |
true when resolved_grammar_key came from the TIER's frontmatter grammar: rather than the caller (gh#154). | |
| std::vector< std::string > | stop |
| Stop sequences. | |
| std::string | tool_call_mode |
| Per-call tool-call generation mode (gh#103). | |
| int | logprobs = 0 |
| Top log-probs per token (0 = disabled) | |
| int | time_limit_ms = 0 |
| Wall-clock time cap in milliseconds. | |
| std::string | profile |
| GPU resource profile name. | |
| bool | auto_adapt = true |
| Enable throughput-based max_tokens auto-adaptation. | |
| float | adapt_headroom = 0.9f |
| Target time usage fraction for auto-adaptation. | |
| std::string | tools |
Active tool definitions for this turn, as an MCP tool-list JSON array ([{name, description, inputSchema}, ...]), already filtered per-tier by the caller. | |
Generation parameters for a single inference call.
Member initializers are the authoritative sampler defaults; a default-constructed GenerationParams is already a valid request.
@req REQ-TYPE-005
| float entropic::GenerationParams::frequency_penalty = 0.0f |
Frequency-penalty term in llama.cpp's penalties sampler (gh#23 MVP item 3).
Subtracts a per-occurrence linear amount from any token that has appeared in the recent window — penalizes by COUNT rather than presence. 0.0 (default) disables — preserves pre-v2.3.15 chain bit-for-bit. Typical range: 0.0–2.0. Pairs with repeat_penalty (multiplicative) and presence_penalty (per-presence constant); all three run in one llama_sampler_init_penalties call. Any non-default value here activates the penalties stage even when repeat_penalty == 1.0 and presence_penalty == 0.0.
| std::string entropic::GenerationParams::grammar |
true when resolved_grammar_key came from the TIER's frontmatter grammar: rather than the caller (gh#154).
All three request-side sources arrive as grammar, so without this a consumer cannot tell a tier-configured grammar from one they passed themselves.
| std::string entropic::GenerationParams::grammar_key |
Per-token logit bias map (gh#23 MVP item 4).
Maps token id → additive bias (in logit-space). Applied at the start of the sampler chain (before penalties), so it shapes the post-softmax distribution that every downstream filter (top-k, top-p, min-p, etc.) sees. Common uses:
bias = -INFINITY (or a large negative value like -100 if -INFINITY doesn't survive JSON).bias = +INFINITY (or large positive).bias = ±1.0..±5.0. Empty map (default) disables the stage entirely — preserves pre-v2.3.16 chain shape bit-for-bit. | int entropic::GenerationParams::logprobs = 0 |
| int entropic::GenerationParams::max_tokens = 4096 |
| float entropic::GenerationParams::min_p = 0.0f |
Min-p nucleus sampling threshold (gh#23 MVP item 1).
Filters tokens whose probability is below min_p * P(top_token). 0.0 (default) disables — preserves pre-v2.3.10 behavior bit-for-bit. Typical productive range: 0.01–0.2. Values >= 1.0 filter every non-top token (degenerate but not crash-inducing; the dist sampler still selects the single survivor).
| float entropic::GenerationParams::presence_penalty = 0.0f |
Presence-penalty term in llama.cpp's penalties sampler (gh#23 MVP item 2).
Subtracts a constant from any token that has appeared at least once in the recent window. 0.0 (default) disables — preserves pre-v2.3.14 chain bit-for-bit. Typical range: 0.0–2.0.
| std::string entropic::GenerationParams::profile |
GPU resource profile name.
Resolved to GPUResourceProfile by the orchestrator before passing to the backend. Empty string means use the "balanced" profile.
| int entropic::GenerationParams::reasoning_budget = -1 |
| std::string entropic::GenerationParams::resolved_grammar_key |
The key or frontmatter stem that was NAMED for this call, set by ModelOrchestrator::resolve_grammar_key (gh#154).
Carries the name even when the lookup MISSED, because a named key that resolved to nothing is the state gh#154 exists to make visible — GrammarProvenance::resolved is what says whether a grammar text actually reached the sampler.
| int entropic::GenerationParams::seed = -1 |
| std::string entropic::GenerationParams::session_key |
Caller-scoped session this generation belongs to (gh#144).
"" (default) is the single legacy sequence — bit-identical to pre-v2.12.0. The BACKEND owns the key-to-llama-sequence mapping and its eviction, because it owns the slots and the residency; the engine deliberately knows nothing about sequence ids.
| std::vector<std::string> entropic::GenerationParams::stop |
| int entropic::GenerationParams::time_limit_ms = 0 |
| std::string entropic::GenerationParams::tool_call_mode |
Per-call tool-call generation mode (gh#103).
Empty = defer to tier/default ("batch"). "sequential" → the orchestrator appends the resolved family tool-call close marker to stop at stage time, so the decode loop halts at the FIRST closed tool call (one tool/turn, observe-between-actions). "batch" = no injection (parallel calls).
| std::string entropic::GenerationParams::tools |
Active tool definitions for this turn, as an MCP tool-list JSON array ([{name, description, inputSchema}, ...]), already filtered per-tier by the caller.
gh#87 (v2.7.0): structured tool defs flow here instead of being string-injected into the system message. The orchestrator stages them on the backend (LlamaCppBackend::set_active_tools) so the common_chat render emits — and parse extracts — the model's native tool-call wire format. Empty (default) = no tools this turn (the backend renders without a tool grammar).
1.9.8