A GGUF's shape, as far as its metadata states it. More...

#include </home/runner/work/entropic/entropic/src/inference/gguf_metadata.h>

Public Member Functions
uint64_t	bytes_per_block () const
	Average bytes per transformer block.

uint64_t	expert_bytes_per_block () const
	Average expert bytes per block — what one `cpu_moe_layers` buys.

Public Attributes
bool	known = false
	False = read failed; fields are meaningless.

std::string	architecture
	`general.architecture`, "" when absent.

int	block_count = 0
	`<arch>.block_count` — the real layer count.

int	expert_count = 0
	`<arch>.expert_count`; 0 = dense, >0 = MoE.

uint64_t	block_bytes = 0
	Total bytes of tensors that belong to a numbered block.

uint64_t	expert_bytes = 0
	Bytes of expert (MoE FFN) tensors inside `block_bytes`.

uint64_t	non_block_bytes = 0
	Bytes not attributable to any block — always resident.

Detailed Description

A GGUF's shape, as far as its metadata states it.

Every field is "what the file says", never an inference. known == false means the file could not be read or did not declare its architecture, and callers must fall back rather than treat the zeros as measurements — the same contract FootprintEstimate uses.

Version: 2.13.1

Definition at line 49 of file gguf_metadata.h.

Member Function Documentation

◆ bytes_per_block()

uint64_t entropic::GgufShape::bytes_per_block ( ) const

inline

Average bytes per transformer block.

Returns: block_bytes / block_count, or 0 when unknown. @utility

Version: 2.13.1

Definition at line 79 of file gguf_metadata.h.

◆ expert_bytes_per_block()

uint64_t entropic::GgufShape::expert_bytes_per_block ( ) const

inline

Average expert bytes per block — what one cpu_moe_layers buys.

Returns: expert_bytes / block_count, or 0 when dense or unknown. @utility

Version: 2.13.1

Definition at line 90 of file gguf_metadata.h.

Member Data Documentation

◆ architecture

std::string entropic::GgufShape::architecture

general.architecture, "" when absent.

Definition at line 51 of file gguf_metadata.h.

◆ block_bytes

uint64_t entropic::GgufShape::block_bytes = 0

Total bytes of tensors that belong to a numbered block.

Excludes token embeddings, the output head and final norms, which are resident regardless of how many layers are offloaded. Pricing a layer as file_bytes / block_count folds those in and overstates per-layer cost — the error that makes auto conservative even when it knows the layer count.

Definition at line 62 of file gguf_metadata.h.

◆ block_count

int entropic::GgufShape::block_count = 0

<arch>.block_count — the real layer count.

Definition at line 52 of file gguf_metadata.h.

◆ expert_bytes

uint64_t entropic::GgufShape::expert_bytes = 0

Bytes of expert (MoE FFN) tensors inside block_bytes.

Summed over tensors matching llama.cpp's own expert pattern. 0 for a dense model. This is what a cpu_moe_layers split actually moves.

Definition at line 68 of file gguf_metadata.h.

◆ expert_count

int entropic::GgufShape::expert_count = 0

<arch>.expert_count; 0 = dense, >0 = MoE.

Definition at line 53 of file gguf_metadata.h.

◆ known

bool entropic::GgufShape::known = false

False = read failed; fields are meaningless.

Definition at line 50 of file gguf_metadata.h.

◆ non_block_bytes

uint64_t entropic::GgufShape::non_block_bytes = 0

Bytes not attributable to any block — always resident.

Definition at line 71 of file gguf_metadata.h.

The documentation for this struct was generated from the following file:

inference/gguf_metadata.h

Public Member Functions

Public Attributes