Model configuration for a single tier. More...

#include <entropic/types/config.h>

Inheritance diagram for entropic::ModelConfig:

Public Attributes
std::filesystem::path	path
	Resolved model file path.

std::string	adapter = "qwen35"
	Chat adapter name.

int	context_length = 16384
	Context window size (512–131072)

int	gpu_layers = -1
	GPU offload layers (-1 = all)

bool	keep_warm = false
	Pre-warm model at startup.

bool	use_mlock = true
	Lock model in system RAM.

int	reasoning_budget = -1
	Think token budget (-1 = unlimited)

std::string	cache_type_k = "f16"
	KV cache key quantization type.

std::string	cache_type_v = "f16"
	KV cache value quantization type.

int	n_batch = 512
	Batch size for prompt processing.

int	n_threads = 0
	CPU threads (0 = auto-detect)

std::string	tensor_split
	Multi-GPU tensor split ratios (empty = single GPU)

bool	flash_attn = true
	Enable flash attention.

std::optional< std::vector< std::string > >	allowed_tools
	Tool whitelist (nullopt = all)

std::filesystem::path	mmproj_path
	Vision projector GGUF path.

std::string	model_format = "gguf"
	Expected model format.

Detailed Description

Model configuration for a single tier.

Contains all parameters needed to load and configure a model, including llama.cpp pass-through fields for KV cache, batching, threading, and attention.

Version: 1.8.0

Definition at line 148 of file config.h.

Member Data Documentation

◆ adapter

std::string entropic::ModelConfig::adapter = "qwen35"

Chat adapter name.

Definition at line 150 of file config.h.

◆ allowed_tools

std::optional<std::vector<std::string> > entropic::ModelConfig::allowed_tools

Tool whitelist (nullopt = all)

Definition at line 166 of file config.h.

◆ cache_type_k

std::string entropic::ModelConfig::cache_type_k = "f16"

KV cache key quantization type.

Definition at line 158 of file config.h.

◆ cache_type_v

std::string entropic::ModelConfig::cache_type_v = "f16"

KV cache value quantization type.

Definition at line 159 of file config.h.

◆ context_length

int entropic::ModelConfig::context_length = 16384

Context window size (512–131072)

Definition at line 151 of file config.h.

◆ flash_attn

bool entropic::ModelConfig::flash_attn = true

Enable flash attention.

Definition at line 163 of file config.h.

◆ gpu_layers

int entropic::ModelConfig::gpu_layers = -1

GPU offload layers (-1 = all)

Definition at line 152 of file config.h.

◆ keep_warm

bool entropic::ModelConfig::keep_warm = false

Pre-warm model at startup.

Definition at line 153 of file config.h.

◆ mmproj_path

std::filesystem::path entropic::ModelConfig::mmproj_path

Vision projector GGUF path.

When non-empty, the backend loads an mtmd_context alongside the base model for multimodal inference. Empty (default) = text-only model.

Version: 1.9.11

Definition at line 174 of file config.h.

◆ model_format

std::string entropic::ModelConfig::model_format = "gguf"

Expected model format.

"gguf" (default), "axmodel", "onnx", or empty (auto-detect). The backend validates that the file matches the expected format during load(). Mismatch returns ENTROPIC_ERROR_LOAD_FAILED with a diagnostic message identifying the actual format.

Version: 1.9.13

Definition at line 184 of file config.h.

◆ n_batch

int entropic::ModelConfig::n_batch = 512

Batch size for prompt processing.

Definition at line 160 of file config.h.

◆ n_threads

int entropic::ModelConfig::n_threads = 0

CPU threads (0 = auto-detect)

Definition at line 161 of file config.h.

◆ path

std::filesystem::path entropic::ModelConfig::path

Resolved model file path.

Definition at line 149 of file config.h.

◆ reasoning_budget

int entropic::ModelConfig::reasoning_budget = -1

Think token budget (-1 = unlimited)

Definition at line 157 of file config.h.

◆ tensor_split

std::string entropic::ModelConfig::tensor_split

Multi-GPU tensor split ratios (empty = single GPU)

Definition at line 162 of file config.h.

◆ use_mlock

bool entropic::ModelConfig::use_mlock = true

Lock model in system RAM.

Definition at line 154 of file config.h.

The documentation for this struct was generated from the following file:

entropic/types/config.h

Public Attributes