Why More Demonstrations Made the Robot Worse

Behavior Cloning, honestly

Behavior cloning (BC) is supervised learning on demonstrations: at each timestep the policy is trained to output the action the human took in that state.

The catch: the loss treats a crisp grasp and a nervous mid-air hesitation as equally sacred training signal. BC has no concept of "that moment was a mistake."

Why more demos can hurt

• Early demos are usually recorded carefully by one expert — mostly clean.
• Scaling to 50+ demos means more operators, fatigue, retries, and pauses — the quality distribution widens.
• BC fits the average of that distribution. Averaging a clean trajectory with hesitant ones yields a slow, wobbly policy — drag the slider right in Vanilla mode and watch the executed path degrade.
• It also compounds: wobble puts the robot in states no demo covered (covariate shift), where errors snowball.

The WARP-RM fix: weight, don't worship

A self-supervised reward model scores each moment of each demo — no human labels. Signals it can learn from: motion smoothness, progress toward task completion, consistency with other demos, and time-to-effect. Training then becomes:

Low-weight moments barely influence the gradient, so the policy distills the best moments across all demos. That is how the reported result — 1.8× throughput vs vanilla BC on t-shirt folding — comes from the same data, just weighted. Flip the toggle above: the red segments fade, the executed path snaps clean, and throughput climbs.