Reinforcement learning workbench

How far ahead should a learner look?

Compare an agent that reacts now with one that predicts what comes next. Change the environment, run the same trial, and inspect where anticipation earns its keep.

Concept model. This illustrates learning dynamics; it is not a biological neural measurement.

Environment

Shape the trial

Learning rate0.28 How quickly new evidence changes the policy. Future discount0.82 How strongly the agent values delayed outcomes. Reward delay4 steps How long the useful outcome takes to arrive. Uncertainty24% How often the environment breaks expectation.

Scenario name

Training run

Moderate delay, clear signal

Step0/ 60

Choice

Reward

Decision, not decoration

What changed, and what should you test next?

Run the sample trial to compare both policies.

Next: keep the sample assumptions, then change one variable at a time.

Reactive policy0

Updates mainly from what just happened.

Reward: 0.0
Stability: 0%
Confidence: 0%

Anticipation-aware0

Propagates expected value back toward the choice.

Reward: 0.0
Stability: 0%
Confidence: 0%

Transparent by design

Open the model before trusting the result.

Credit assignment

Delayed rewards are propagated backward using the future-discount setting. Higher delay weakens the signal.

Environmental noise

Uncertainty lowers both policies' confidence and penalizes long-horizon predictions more heavily.

Scope boundary

This is a deterministic intuition aid. It does not model neurons, dopamine, clinical behavior, or production-grade RL.