Pure-virtual per-generation sampler used by the decode loop. More...
#include <entropic/inference/sampler.h>
Public Member Functions | |
| virtual int32_t | sample ()=0 |
| Sample one token from the current decode position. | |
| virtual void | reset () |
| Reset internal sampler state. | |
Detailed Description
Pure-virtual per-generation sampler used by the decode loop.
One Sampler instance represents one configured sampling chain — its lifetime is bounded by a single generate / streaming / speculative call. Implementations capture whatever backend state they need (e.g. the llama_context*) at construction and free their internal resources in their destructor; callers neither know nor care what's underneath.
Implementations must never throw and must never crash on repeated sample() invocations beyond model EOS.
- Version
- 2.3.10
Member Function Documentation
◆ reset()
|
inlinevirtual |
Reset internal sampler state.
Default no-op. Mock implementations may override to clear accumulated history (e.g. for tests that reuse a sampler across logical generations). Production llama.cpp samplers are created fresh per generation, so this is largely a test-facing hook.
Reimplemented in entropic::LlamaCppSampler.
◆ sample()
|
pure virtual |
Sample one token from the current decode position.
The backend has already positioned its KV cache to the token-to-be-sampled prior to calling sample(). The Sampler is responsible only for converting the latest logits into a token id.
- Returns
- Chosen token id, or a negative value on internal error (callers treat <0 as a hard failure).
- Version
- 2.3.10
Implemented in entropic::LlamaCppSampler.
The documentation for this class was generated from the following file:
- entropic/inference/sampler.h
