ONE ROLLOUT · TOKEN EVIDENCE
See where the ratio leaves the clip.
Compare rollout-referenced and stale-policy importance ratios, then run transition-aligned GAE backwards across one token trace.
Direct ratio = exp(current logp − rollout logp). Stale comparison = exp(current logp − stale logp). This calculator is illustrative, not a paper reproduction.
COMPUTED TRACE
Ratio and GAE evidence
No trace computed yet.Use the four-token fixture to inspect every recurrence.
TOKENS
DIRECT MEAN
STALE MEAN
CLIPPED TOKENS
FIRST TOKEN GAE
Rollout-policy reference
Stale-policy comparison
POLICY UPDATE · 1
VALUE UPDATE · 1
VALUE UPDATE · 2
| TOKEN | DIRECT RATIO | STALE RATIO | CLIPPED | TD DELTA | GAE | SURROGATE |
|---|
Limits: standard illustrative formulas with a rollout-policy denominator. This does not reproduce Tsinghua training code, Direct Double-Sided IS details beyond symmetric clipping, Qwen3 MoE, frozen attention, value pretraining, ablations, or long-horizon stability.