ai
Top-k
Top-k Sampling
Definition
Top-k sampling restricts token sampling to the k most probable tokens at each generation step, setting the probability of all other tokens to zero before re-normalizing and sampling. It prevents the model from selecting very low-probability tokens but is less adaptive than top-p (nucleus) sampling, which adjusts the candidate set size based on probability mass.
Top-k and top-p are often used together.
Can this ship? Are we exposed? Can you prove it?
Every CISO is asking these three questions. We answer them with facts, not guesses, from repository to runtime.