device_memory.h File Reference

gh#142: ask the GPU how much VRAM is actually free. More...

#include <cstdint>

Include dependency graph for device_memory.h:

This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Namespaces
namespace	entropic
	Activate model on GPU (WARM → ACTIVE).

Functions
uint64_t	entropic::query_device_free_vram_bytes ()
	Free VRAM on the first GPU device ggml reports, in bytes.

Detailed Description

gh#142: ask the GPU how much VRAM is actually free.

Why this exists

orchestrator.h documented the VRAM budget as resolving "`ENTROPIC_VRAM_BUDGET_BYTES` -> CUDA `cudaMemGetInfo` -> 0", but only the env var was ever implemented — the header promised a capability the engine did not have. With no variable set the budget was 0, which the gate reads as "unknown, do not enforce", so every default deployment ran with the admission gate disabled and a too-large tier aborted the process inside llama.cpp instead of being refused.

This closes that gap through ggml's own device abstraction rather than the CUDA runtime directly, so it reports honestly for whichever backend variant was compiled in and needs no CUDA-specific build plumbing.

Free, not total

The query returns FREE VRAM, which is the number that matters: gh#142's reporter hit this on a GPU busier than the developer's, and a budget derived from total capacity would have admitted the load and aborted anyway. It is sampled once at initialize(), so it reflects the device as the engine found it — a later consumer of VRAM can still cause a failure this gate cannot see.

Version

2.11.0

Definition in file device_memory.h.