A placement decision, with the reason it was reached. More...

#include </home/runner/work/entropic/entropic/src/inference/auto_placement.h>

Public Attributes
bool	known = false
	False = could not decide; leave config alone.

int	gpu_layers = 0
	Layers to offload, or -1 meaning ALL.

int	cpu_moe_layers = 0
	Layers whose experts stay host-side.

uint64_t	bytes = 0
	Estimated resident footprint of the choice.

bool	fully_resident = false
	True when `gpu_layers` is the -1 sentinel.

int	n_ubatch = 0
	n_ubatch auto chose, or 0 to leave the operator's value alone.

uint64_t	margin_bytes = 0
	Free VRAM left over once this placement is loaded, in bytes.

const char *	reason = ""
	Why this placement, for the operator's log.

Detailed Description

A placement decision, with the reason it was reached.

Version: 2.13.1

Definition at line 44 of file auto_placement.h.

Member Data Documentation

◆ bytes

uint64_t entropic::AutoPlacement::bytes = 0

Estimated resident footprint of the choice.

Definition at line 59 of file auto_placement.h.

◆ cpu_moe_layers

int entropic::AutoPlacement::cpu_moe_layers = 0

Layers whose experts stay host-side.

Definition at line 58 of file auto_placement.h.

◆ fully_resident

bool entropic::AutoPlacement::fully_resident = false

True when gpu_layers is the -1 sentinel.

Definition at line 60 of file auto_placement.h.

◆ gpu_layers

int entropic::AutoPlacement::gpu_layers = 0

Layers to offload, or -1 meaning ALL.

Full residency is reported as the -1 sentinel, not as a count equal to block_count, because -1 is what the rest of the engine reads as "all layers": classify_offload prices it as fully resident, and mlock_refused exempts it (gpu_layers < 0) since the host side is then transient rather than pinned. Emitting a concrete 30 would silently price a fully-resident model through the PARTIAL branch — omitting embeddings and the output head, an under-count — and strip the mlock exemption from a configuration that works today.

Definition at line 57 of file auto_placement.h.

◆ known

bool entropic::AutoPlacement::known = false

False = could not decide; leave config alone.

Definition at line 45 of file auto_placement.h.

◆ margin_bytes

uint64_t entropic::AutoPlacement::margin_bytes = 0

Free VRAM left over once this placement is loaded, in bytes.

v2.13.2. bytes says what the placement costs; this says what it does NOT consume, which is the number that decides whether the placement survives ordinary growth — a longer system prompt, one more tool schema, a KV change. A consumer put it exactly right: a placement that fits by fifteen megabytes is a coincidence, not a configuration. The engine takes it (it was measured running) and states the margin so the operator can decide otherwise.

Definition at line 80 of file auto_placement.h.

◆ n_ubatch

int entropic::AutoPlacement::n_ubatch = 0

n_ubatch auto chose, or 0 to leave the operator's value alone.

v2.13.2: auto gives up ubatch before it gives up a layer. Measured on a 26B-A4B at IQ2, four iterations, freeing the same ~1800 MiB either way: lowering ubatch costs prefill 49% and decode NOTHING (overlapping error bars); dropping six layers costs prefill 44% and decode 66%. Layers are the two-axis currency, so they are spent last.

Definition at line 69 of file auto_placement.h.

◆ reason

const char* entropic::AutoPlacement::reason = ""

Why this placement, for the operator's log.

Definition at line 81 of file auto_placement.h.

The documentation for this struct was generated from the following file:

inference/auto_placement.h

Public Attributes