entropic::SecondaryModelLoader Class Reference

Role-keyed lifecycle manager for non-primary models. More...

#include <entropic/inference/secondary_model_loader.h>

Public Member Functions
bool	ensure_loaded (const std::string &role, const ModelConfig &config)
	Lazily load and activate a model for a role.
InferenceBackend *	get (const std::string &role) const
	Get the backend for a role.
std::shared_ptr< InferenceBackend >	get_shared (const std::string &role) const
	Get the backend for a role as a shared_ptr.
bool	release_role (const std::string &role)
	Unload and drop a role.
bool	is_loaded (const std::string &role) const
	Check whether a role is currently loaded and active.
std::vector< std::string >	loaded_roles () const
	Names of all roles with a currently-loaded backend.
void	clear_all_prompt_caches ()
	Fanout: clear prompt/KV cache on every loaded backend.
void	shutdown ()
	Unload every role.

Detailed Description

Role-keyed lifecycle manager for non-primary models.

Replaces the per-role std::shared_ptr<InferenceBackend> members (router_) that previously lived directly on ModelOrchestrator. The router refactor is intentionally invisible to callers: existing router behavior is preserved via loader_.get("router").

Role names (conventional, not enforced):

• "router" — digit-classifier model used by ModelOrchestrator::route()
• "draft" — speculative-decoding draft (v2.1.11+)
• "thinking" — future thinking-model slot (gh#25)

Version

2.1.11

Definition at line 55 of file secondary_model_loader.h.

Member Function Documentation

clear_all_prompt_caches()

void entropic::SecondaryModelLoader::clear_all_prompt_caches

(

)

Fanout: clear prompt/KV cache on every loaded backend.

Used by ModelOrchestrator::clear_all_prompt_caches() so the router and draft caches invalidate alongside the main pool when identity content changes (P1-7, v2.0.6-rc16 contract).

@utility

Version

2.1.11

Secondary roles participate in pool-wide prompt-cache invalidation, so a cache flush cannot leave a router or draft model holding a stale prefix. A no-op when no role is loaded.

@req REQ-INFER-020

Version

2.1.11

Definition at line 162 of file secondary_model_loader.cpp.

ensure_loaded()

bool entropic::SecondaryModelLoader::ensure_loaded	(	const std::string &	role,
		const ModelConfig &	config
	)

Lazily load and activate a model for a role.

If the role is already loaded against the same config path, this is a no-op (idempotent). If the role is loaded against a different path, the existing backend is unloaded first.

Parameters

role	Role name (e.g. "router", "draft").
config	ModelConfig for the secondary model.

Returns

true on successful activation, false on failure.

Version

2.1.11

Idempotent against the same path (an already-loaded role short-circuits) and reloads when the configured path changes. A failed load is isolated: the role is simply not recorded, so a bogus draft path degrades the feature without blocking engine initialisation.

Parameters

role	Role name (e.g. "router", "draft").
config	ModelConfig for the secondary model.

Returns

true on activation success (including the already-loaded short-circuit); false if the backend failed to activate. @req REQ-INFER-020

Version

2.1.11

Definition at line 41 of file secondary_model_loader.cpp.

get()

InferenceBackend * entropic::SecondaryModelLoader::get

(

const std::string &

role

)

const

Get the backend for a role.

Parameters

role	Role name.

Returns

Backend pointer if loaded, nullptr otherwise. @utility

Version

2.1.11

Parameters

role	Role name.

Returns

Backend pointer, nullptr if role is unknown. @utility

Version

2.1.11

Definition at line 74 of file secondary_model_loader.cpp.

get_shared()

std::shared_ptr< InferenceBackend > entropic::SecondaryModelLoader::get_shared

(

const std::string &

role

)

const

Get the backend for a role as a shared_ptr.

Used when callers need to extend backend lifetime beyond the loader (e.g. holding a reference for the duration of a long generation while the loader could otherwise be released).

Parameters

role	Role name.

Returns

Backend shared_ptr (empty if not loaded). @utility

Version

2.1.11

Parameters

role	Role name.

Returns

Backend shared_ptr, empty if role is unknown. @utility

Version

2.1.11

Definition at line 87 of file secondary_model_loader.cpp.

is_loaded()

bool entropic::SecondaryModelLoader::is_loaded

(

const std::string &

role

)

const

Check whether a role is currently loaded and active.

Check whether a role is currently loaded and non-COLD.

Parameters

role	Role name.

Returns

true if get(role) != nullptr and the backend reports it is loaded (state != COLD). @utility

Version

2.1.11

Parameters

role	Role name.

Returns

true if backend is present and is_loaded(). @utility

Version

2.1.11

Definition at line 126 of file secondary_model_loader.cpp.

loaded_roles()

std::vector< std::string > entropic::SecondaryModelLoader::loaded_roles

(

)

const

Names of all roles with a currently-loaded backend.

Names of all loaded roles (sorted for deterministic output).

Returns

Sorted list of role names whose backend is non-COLD. @utility

Version

2.1.11

Returns

Sorted role names whose backend reports is_loaded(); a role whose load failed is absent, which is how callers observe the failure. @req REQ-INFER-020

Version

2.1.11

Definition at line 139 of file secondary_model_loader.cpp.

release_role()

bool entropic::SecondaryModelLoader::release_role

(

const std::string &

role

)

Unload and drop a role.

Parameters

role	Role name.

Returns

true if a role was unloaded, false if it was not loaded.

Version

2.1.11

Parameters

role	Role name.

Returns

true if a role was unloaded, false if none was loaded under that name (an unknown role is not an error). @req REQ-INFER-020 @req REQ-INFER-002

Version

2.1.11

Definition at line 104 of file secondary_model_loader.cpp.

shutdown()

void entropic::SecondaryModelLoader::shutdown

(

)

Unload every role.

Mirrors ModelOrchestrator::shutdown() — called during engine teardown. Safe to call repeatedly.

Version

2.1.11

Safe to call repeatedly.

Idempotent on an empty loader, so an orchestrator that never loaded a router or draft still tears down cleanly.

@req REQ-INFER-002 @req REQ-INFER-020

Version

2.1.11

Definition at line 179 of file secondary_model_loader.cpp.

---

The documentation for this class was generated from the following files:

• entropic/inference/secondary_model_loader.h
• inference/secondary_model_loader.cpp