Reinforcement learning workbench

How far ahead should a learner look?

Compare an agent that reacts now with one that predicts what comes next. Change the environment, run the same trial, and inspect where anticipation earns its keep.

Concept model. This illustrates learning dynamics; it is not a biological neural measurement.

Environment

Shape the trial

Learning rate0.28 How quickly new evidence changes the policy. Future discount0.82 How strongly the agent values delayed outcomes. Reward delay4 steps How long the useful outcome takes to arrive. Uncertainty24% How often the environment breaks expectation.

Scenario name

Training run

Moderate delay, clear signal

Step0/ 60

Choice

Reward

Decision, not decoration

What changed, and what should you test next?

Run the sample trial to compare both policies.

Next: keep the sample assumptions, then change one variable at a time.

Reactive policy0

Updates mainly from what just happened.

Reward: 0.0
Stability: 0%
Confidence: 0%

Anticipation-aware0

Propagates expected value back toward the choice.

Reward: 0.0
Stability: 0%
Confidence: 0%

Transparent by design

Open the model before trusting the result.

Credit assignment

Delayed rewards are propagated backward using the future-discount setting. Higher delay weakens the signal.

Environmental noise

Uncertainty lowers both policies' confidence and penalizes long-horizon predictions more heavily.

Scope boundary

This is a deterministic intuition aid. It does not model neurons, dopamine, clinical behavior, or production-grade RL.

Source coefficients shape two illustrative ceilings

Read the explanation

At saved learning rate point two eight, discount point eight two, delay four and uncertainty point two four, the written reactive ceiling is fifty two point eight one and predictive ceiling about sixty one point six one four. The bars use eight pixels per ceiling point. The source multiplies each ceiling by a different exponential progress curve and floors actual values at four. These are deterministic invented equations, not learned policy performance or biological measurements. Predictive stability is ninety four minus uncertainty times sixty seven minus delay times two point one, floored at five. The sample yields sixty nine point five two; edge uncertainty point seven eight and delay nine yields twenty two point eight four. These bars use six pixels per displayed stability unit. The percent-style labels are formula outputs, not calibrated confidence or observed reliability. The source recomputes arrays from index zero through the current step, so sixty advances give sixty one sampled values. These bars compare sixty intervals with sixty one points at seven pixels per count. Timed running schedules real fifty five millisecond advances; manual Step advances one. JSON exports assumptions, current step and finding but not the full arrays or scores. No trained model or real experiment is run.