entropic::SpeculativeRunState Struct Reference

Bundles per-kernel-run mutable state to keep the loop body focused on its responsibility (knots: cognitive ≤ 15, ≤ 3 returns). More...

Public Attributes
common_speculative *	spec = nullptr
common_sampler *	smpl = nullptr
llama_context *	ctx_tgt = nullptr
llama_context *	ctx_dft = nullptr
llama_batch	batch_tgt {}
bool	batch_initialized = false
llama_seq_id	seq_id = 0
int	n_past = 0
llama_token	id_last = 0
std::vector< llama_token >	prompt_tgt
std::vector< llama_token >	draft
std::string	generated
std::vector< std::string >	stop
	gh#108: stop seqs (effective_stop); empty for gh#36
int	n_generated = 0
int	n_prefilled = 0
	gh#144 (v2.12.0): prompt tokens actually pushed through llama_decode during this run's prefill.
int	n_drafted = 0
int	n_accepted = 0
bool	has_eos = false
std::string	finish_reason
entropic_error_t	error_code = ENTROPIC_OK
std::string	error_message
bool	use_ckpt_tgt = false
bool	use_ckpt_dft = false
common_prompt_checkpoint	ckpt

Detailed Description

Bundles per-kernel-run mutable state to keep the loop body focused on its responsibility (knots: cognitive ≤ 15, ≤ 3 returns).

@dg_internal

Version

2.1.11

Definition at line 3804 of file llama_cpp_backend.cpp.

Member Data Documentation

batch_initialized

bool entropic::SpeculativeRunState::batch_initialized = false

Definition at line 3810 of file llama_cpp_backend.cpp.

batch_tgt

llama_batch entropic::SpeculativeRunState::batch_tgt {}

Definition at line 3809 of file llama_cpp_backend.cpp.

ckpt

common_prompt_checkpoint entropic::SpeculativeRunState::ckpt

Definition at line 3841 of file llama_cpp_backend.cpp.

ctx_dft

llama_context* entropic::SpeculativeRunState::ctx_dft = nullptr

Definition at line 3808 of file llama_cpp_backend.cpp.

ctx_tgt

llama_context* entropic::SpeculativeRunState::ctx_tgt = nullptr

Definition at line 3807 of file llama_cpp_backend.cpp.

draft

std::vector<llama_token> entropic::SpeculativeRunState::draft

Definition at line 3815 of file llama_cpp_backend.cpp.

error_code

entropic_error_t entropic::SpeculativeRunState::error_code = ENTROPIC_OK

Definition at line 3829 of file llama_cpp_backend.cpp.

error_message

std::string entropic::SpeculativeRunState::error_message

Definition at line 3830 of file llama_cpp_backend.cpp.

finish_reason

std::string entropic::SpeculativeRunState::finish_reason

Definition at line 3828 of file llama_cpp_backend.cpp.

generated

std::string entropic::SpeculativeRunState::generated

Definition at line 3816 of file llama_cpp_backend.cpp.

has_eos

bool entropic::SpeculativeRunState::has_eos = false

Definition at line 3827 of file llama_cpp_backend.cpp.

id_last

llama_token entropic::SpeculativeRunState::id_last = 0

Definition at line 3813 of file llama_cpp_backend.cpp.

n_accepted

int entropic::SpeculativeRunState::n_accepted = 0

Definition at line 3826 of file llama_cpp_backend.cpp.

n_drafted

int entropic::SpeculativeRunState::n_drafted = 0

Definition at line 3825 of file llama_cpp_backend.cpp.

n_generated

int entropic::SpeculativeRunState::n_generated = 0

Definition at line 3818 of file llama_cpp_backend.cpp.

n_past

int entropic::SpeculativeRunState::n_past = 0

Definition at line 3812 of file llama_cpp_backend.cpp.

n_prefilled

int entropic::SpeculativeRunState::n_prefilled = 0

gh#144 (v2.12.0): prompt tokens actually pushed through llama_decode during this run's prefill.

The MTP path counted NOTHING, so last_prefill_tokens() read 0 for every MTP turn and there was no way to measure whether a prefix was reused or re-decoded. Instrument before optimising.

Definition at line 3824 of file llama_cpp_backend.cpp.

prompt_tgt

std::vector<llama_token> entropic::SpeculativeRunState::prompt_tgt

Definition at line 3814 of file llama_cpp_backend.cpp.

seq_id

llama_seq_id entropic::SpeculativeRunState::seq_id = 0

Definition at line 3811 of file llama_cpp_backend.cpp.

smpl

common_sampler* entropic::SpeculativeRunState::smpl = nullptr

Definition at line 3806 of file llama_cpp_backend.cpp.

spec

common_speculative* entropic::SpeculativeRunState::spec = nullptr

Definition at line 3805 of file llama_cpp_backend.cpp.

stop

std::vector<std::string> entropic::SpeculativeRunState::stop

gh#108: stop seqs (effective_stop); empty for gh#36

Definition at line 3817 of file llama_cpp_backend.cpp.

use_ckpt_dft

bool entropic::SpeculativeRunState::use_ckpt_dft = false

Definition at line 3840 of file llama_cpp_backend.cpp.

use_ckpt_tgt

bool entropic::SpeculativeRunState::use_ckpt_tgt = false

Definition at line 3839 of file llama_cpp_backend.cpp.

---

The documentation for this struct was generated from the following file:

• inference/llama_cpp_backend.cpp