One post-generation summary line, formatted identically on every decode path. More...

#include <string>

Include dependency graph for generation_summary.h:

This graph shows which files directly or indirectly include this file:

Classes
struct	entropic::GenerationSummary
	The numbers a finished generation reports, whatever produced it. More...

Namespaces
namespace	entropic
	Activate model on GPU (WARM → ACTIVE).

Functions
std::string	entropic::format_generation_summary (const GenerationSummary &s)
	Format the one-line generation summary.

Detailed Description

One post-generation summary line, formatted identically on every decode path.

gh#151: the plain decode paths logged

Generated: 87 tokens, finish=stop, 3040ms, 28.5 tok/s

and the speculative path logged a differently shaped line that carried no throughput at all. Both paths computed the value — spec_finalize has populated throughput_tok_s since gh#108 — so this was never a missing measurement in the engine. It was a measurement a reader of the LOGS could take on one arm and not on the other, which is exactly the comparison speculative decoding exists to be judged by.

The formatter is shared rather than duplicated because two hand-rolled lines drifting apart IS the defect being fixed here; a second one would reintroduce it the first time either changed.

Pure and vendor-free so the format is CPU-unit-testable with no model and no GPU — same reason warm_keep_util.h and partial_offload.h are.

Version: 2.12.2

Definition in file generation_summary.h.

Classes

Namespaces

Functions

Detailed Description