backend.h File Reference

InferenceBackend concrete base class. More...

#include <entropic/types/backend_capability.h>
#include <entropic/types/config.h>
#include <entropic/types/generation_result.h>
#include <entropic/types/logprob_result.h>
#include <entropic/types/message.h>
#include <entropic/interfaces/i_hook_handler.h>
#include <atomic>
#include <cstdint>
#include <functional>
#include <mutex>
#include <string>
#include <string_view>
#include <vector>

Include dependency graph for backend.h:

This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Classes
class	entropic::InferenceBackend
	Concrete base class for inference backends (80% logic). More...

Namespaces
namespace	entropic
	Activate model on GPU (WARM → ACTIVE).

Detailed Description

InferenceBackend concrete base class.

Owns: lifecycle state machine, transition mutex, metrics, logging. Subclasses override: do_load, do_activate, do_deactivate, do_unload, do_generate, do_generate_streaming, do_complete, do_count_tokens, do_evaluate_logprobs.

Thread safety

• state() is lock-free (std::atomic<ModelState>)
• load/activate/deactivate/unload acquire transition_mutex_
• generate/generate_streaming/complete require ACTIVE state, do NOT acquire transition_mutex_ (generation is concurrent with state queries, but not with state transitions)
• evaluate_logprobs acquires eval_mutex_ (separate from transition and generation — evaluation can run concurrently with generation)

State machine

COLD ──load()──> WARM ──activate()──> ACTIVE
  ^               |                     |
  └──unload()─────┘<──deactivate()──────┘

Internal to inference .so — not exposed across boundaries.

Version

1.9.13

Definition in file backend.h.