The numbers a finished generation reports, whatever produced it. More...

#include </home/runner/work/entropic/entropic/src/inference/generation_summary.h>

Public Attributes
int	token_count = 0
	Tokens actually generated.

std::string	finish_reason
	stop / length / error / cancel.

double	generation_time_ms = 0.0
	Wall clock for the generation.

double	throughput_tok_s = 0.0
	token_count / seconds.

int	n_drafted = 0
	Speculative: tokens proposed (0 = plain).

int	n_accepted = 0
	Speculative: tokens accepted.

Detailed Description

The numbers a finished generation reports, whatever produced it.

n_drafted/n_accepted are zero on a plain decode; the formatter omits the speculative clause entirely in that case rather than printing zeros, so a plain line stays byte-identical to what it has always been and existing log parsers keep working.

Version: 2.12.2

Definition at line 44 of file generation_summary.h.

Member Data Documentation

◆ finish_reason

std::string entropic::GenerationSummary::finish_reason

stop / length / error / cancel.

Definition at line 46 of file generation_summary.h.

◆ generation_time_ms

double entropic::GenerationSummary::generation_time_ms = 0.0

Wall clock for the generation.

Definition at line 47 of file generation_summary.h.

◆ n_accepted

int entropic::GenerationSummary::n_accepted = 0

Speculative: tokens accepted.

Definition at line 50 of file generation_summary.h.

◆ n_drafted

int entropic::GenerationSummary::n_drafted = 0

Speculative: tokens proposed (0 = plain).

Definition at line 49 of file generation_summary.h.

◆ throughput_tok_s

double entropic::GenerationSummary::throughput_tok_s = 0.0

token_count / seconds.

Definition at line 48 of file generation_summary.h.

◆ token_count

int entropic::GenerationSummary::token_count = 0

Tokens actually generated.

Definition at line 45 of file generation_summary.h.

The documentation for this struct was generated from the following file:

inference/generation_summary.h

Public Attributes