|
| std::filesystem::path | path |
| | Resolved model file path.
|
| |
| std::string | adapter = "qwen35" |
| | Chat adapter name.
|
| |
| int | context_length = 16384 |
| | Context window size (512–131072)
|
| |
| int | gpu_layers = -1 |
| | GPU offload layers (-1 = all)
|
| |
| bool | keep_warm = false |
| | Pre-warm model at startup.
|
| |
| bool | use_mlock = true |
| | Lock model in system RAM.
|
| |
| int | reasoning_budget = -1 |
| | Think token budget (-1 = unlimited)
|
| |
| std::string | cache_type_k = "f16" |
| | KV cache key quantization type.
|
| |
| std::string | cache_type_v = "f16" |
| | KV cache value quantization type.
|
| |
| int | n_batch = 512 |
| | Batch size for prompt processing.
|
| |
| int | n_ubatch = 0 |
| | Physical micro-batch size for prompt processing (gh#23 MVP item 5).
|
| |
| int | n_threads = 0 |
| | CPU threads (0 = auto-detect)
|
| |
| std::string | tensor_split |
| | Multi-GPU tensor split ratios (empty = single GPU)
|
| |
| std::string | split_mode |
| | Multi-GPU split mode for model load (gh#23 MVP item 6).
|
| |
| int | main_gpu = 0 |
| | Primary GPU index for model load (gh#23 MVP item 7).
|
| |
| bool | offload_kqv = true |
| | Offload KQV ops (incl.
|
| |
| float | rope_freq_base = 0.0f |
| | RoPE base frequency override (gh#23 MVP item 9).
|
| |
| float | rope_freq_scale = 0.0f |
| | RoPE frequency scaling factor (gh#23 MVP item 10).
|
| |
| int | n_parallel = 1 |
| | Max parallel sequences per context (gh#23 MVP item 11).
|
| |
| bool | flash_attn = true |
| | Enable flash attention.
|
| |
| std::optional< std::vector< std::string > > | allowed_tools |
| | Tool whitelist (nullopt = all)
|
| |
| std::filesystem::path | mmproj_path |
| | Vision projector GGUF path.
|
| |
| std::string | model_format = "gguf" |
| | Expected model format.
|
| |
Model configuration for a single tier.
Contains all parameters needed to load and configure a model, including llama.cpp pass-through fields for KV cache, batching, threading, and attention.
Every scalar field carries its default as a member initializer here — this struct IS the default spec, not the YAML loader. std::optional fields (e.g. allowed_tools) mean "absent", resolved elsewhere, and are never collapsed into a concrete default.
@req REQ-TYPE-005
- Version
- 1.8.0
Definition at line 154 of file config.h.
| int entropic::ModelConfig::n_parallel = 1 |
Max parallel sequences per context (gh#23 MVP item 11).
llama.cpp's cparams.n_seq_max. 1 (default) matches llama.cpp's default — single-sequence context, bit-identical pre-v2.3.23 behavior. Raising this enables KV-cache slot reuse across multiple concurrent generations (e.g. speculative rejection batches, batched-server scenarios). Effective max is LLAMA_MAX_SEQ; consult llama.cpp for the current ceiling.
- Version
- 2.3.23
Definition at line 238 of file config.h.
| int entropic::ModelConfig::n_ubatch = 0 |
Physical micro-batch size for prompt processing (gh#23 MVP item 5).
llama.cpp's cparams.n_ubatch. Decoupled from n_batch since llama.cpp v0.4 — n_batch is the LOGICAL batch (max tokens queued per llama_decode call) and n_ubatch is the PHYSICAL chunk the kernels actually process. Smaller n_ubatch reduces peak GPU memory for the same n_batch. 0 (default) means "match `n_batch`" — preserves pre-v2.3.17 behavior bit-for-bit since llama.cpp's default in that case is min(n_batch, default). Typical productive values: 128, 256, 512 (== n_batch).
- Version
- 2.3.17
Definition at line 178 of file config.h.
| float entropic::ModelConfig::rope_freq_scale = 0.0f |
RoPE frequency scaling factor (gh#23 MVP item 10).
llama.cpp's cparams.rope_freq_scale. 0.0 (default) takes the model's trained value — preserves pre-v2.3.22 behavior bit-for-bit. Values in (0, 1) shrink the effective context (denser RoPE positions); values > 1 stretch it. Typical YaRN-style extension uses values like 0.5 (2× context). Pairs with rope_freq_base.
- Version
- 2.3.22
Definition at line 228 of file config.h.
