inference Directory Reference

Directory dependency graph for inference:

Directories
	adapters

Files
	adapter_manager.cpp
	AdapterManager implementation — LoRA lifecycle and hot-swap.
	backend.cpp
	InferenceBackend base class implementation.
	batch_util.h
	gh#98 (v2.8.0) same-prefix batch-generation decision logic.
	grammar_registry.cpp
	GrammarRegistry implementation — named grammar management.
	image_preprocessor.cpp
	Image preprocessing implementation.
	inference_c_api.cpp
	C API wrappers for InferenceBackend.
	interface_factory.cpp
	InferenceInterface factory — bridges orchestrator to C callbacks.
	llama_cpp_backend.cpp
	LlamaCppBackend implementation — direct llama.cpp C API.
	llama_cpp_backend.h
	LlamaCppBackend — llama.cpp C API integration.
	llama_cpp_sampler.cpp
	LlamaCppSampler + LlamaCppSamplerFactory implementation (v2.3.10 seam impl).
	llama_cpp_sampler.h
	Concrete llama.cpp Sampler + SamplerFactory (v2.3.10 seam impl).
	llama_cpp_tokenizer.cpp
	LlamaCppTokenizer implementation (v2.3.10 seam impl).
	llama_cpp_tokenizer.h
	Concrete llama.cpp tokenizer (v2.3.10 seam impl).
	mtp_envelope.h
	Pure envelope check for the MTP speculative path (gh#108).
	orchestrator.cpp
	ModelOrchestrator implementation.
	profile_registry.cpp
	ProfileRegistry implementation – named GPU resource profiles.
	prompt_cache.cpp
	PromptCache implementation — LRU eviction, thread-safe.
	prompt_cache.h
	Host-memory KV cache state storage with LRU eviction.
	secondary_model_loader.cpp
	SecondaryModelLoader implementation.
	speculative_compat.cpp
	Implementation of the target/draft compatibility check.
	throughput_tracker.cpp
	ThroughputTracker implementation – EWMA throughput measurement.
	tool_call_markers.h
	gh#103 (v2.8.2): family-aware tool-call CLOSE markers, derived from the resolved common_chat format.
	tool_call_serialize.h
	Shared serialization of parsed tool calls to the C-ABI JSON array form.
	warm_keep_util.h
	gh#96 (v2.7.5) warm-keep / incremental-prefill decision logic.