LLM Next-Token Probability Lab

Softmax & Nucleus Sampling
Sampling Hyperparameters
Temperature (T) 1.0
Controls randomness: Lower = Deterministic peak; Higher = Flat uniform distribution.
Top-K Truncation 5
Limits sampling to the highest K candidate logits.
Top-P Nucleus Mass 0.99
Dynamically truncates tokens after cumulative probability exceeds Top-P.
Raw Logits Scrubber
Token Logit ($z_i$) Exp $e^{z_i/T}$ Prob $P(t_i)$ Status
Softmax Scaling Equation:
$$P(t_i) = \frac{\exp(z_i / T)}{\sum_j \exp(z_j / T)}$$
Active Mass: 100% | Tokens Retained: 5
Context & Generated Sequence 0 tokens generated
The capital of France is
Visual Monte Carlo Token Sampler Tree p5.js Particle Tree
Normalized Token Probabilities
Cumulative Probability Curve (Nucleus)
Sampling Step Telemetry Summary
cumulative_p_retained_count: 3
sampled_token: Paris
top_prob: 0.98003
Step Sampled Token Probability Logit Temp Top-K Top-P

Temperature scaling and filtered normalization explain next-token sampling

Read the explanation

The lab divides logits by temperature, subtracts their maximum, exponentiates and normalizes. For hypothetical two tokens with logit gap one, temperature one gives the leading token about seventy-three point one percent; temperature point five gives eighty-eight point one. Bars share four pixels per percentage point. This is arithmetic on toy logits, not a backend language-model request. After sorting probabilities, the source keeps at most K tokens. Hypothetical two candidates with K two retain both; K one retains only the leader. Bars share one hundred pixels per token. The remaining distribution is then normalized so retained probabilities sum to one; removed candidates cannot be sampled from that filtered distribution. With equal logits, two candidates each have fifty percent raw probability and four candidates each have twenty-five. Bars share five pixels per token probability percentage. Equal logits remain equal under temperature scaling. The source teaches softmax, top-K and top-P with explicit pipeline stages; actual complete native JSON/reset and responsive first-footer checks preserve the default distribution. Random generation actions and full independent scene approval remain separate; no model API is called.

Super generates helpful tools and automates fact-checking across the internet proactively. If you enjoyed this tool, build your own with Super and share it with a friend.