How VR Full-Body Tracking Works

Lighthouse tracking, step by step

1. Sync flash. A base station floods the room with an IR flash to mark time zero.
2. Laser sweep. A spinning rotor sweeps a laser line across the room, horizontally then vertically, about 60 times per second.
3. Timing. Each photodiode on a tracker records when the beam hit it. Time since flash converts directly to an angle: angle = t × rotor speed.
4. Pose solve. With 20+ diodes at known positions on the tracker shell, two angles per diode are enough to solve full 6-DoF position and rotation to sub-millimetre precision.
5. IMU fusion. Between sweeps, a 1000 Hz gyro + accelerometer predicts motion; sweeps correct the drift.

Inverse kinematics fills the gaps

SteamVR only knows where the trackers are. Everything between them is solved by IK: given the position of a hand and shoulder, where must the elbow be?

• A chain of 2 bones with fixed lengths has a circle of valid elbow/knee positions. IK pickers use heuristics (elbows point down-and-out, knees point forward).
• 3-point tracking (head + hands): hips and legs are fully guessed from head height, so avatar legs drift and slide.
• Adding a hip tracker anchors the spine; foot trackers make legs a solvable 2-bone chain (only knees remain heuristic).
• Full 10-11 point setups leave almost nothing to guess, which is why dancers and mocap users wear them.

The latency budget

Total motion-to-photon delay is a pipeline. Roughly, for a lighthouse system:

Under ~20 ms feels instant. Past ~50 ms your own body feels wrong; past 100 ms full-body avatars visibly swim, which is exactly what the latency slider above simulates.

Tracking tech compared

Why avatars glitch: occlusion and drift

• Occlusion. Lighthouse trackers need line-of-sight to at least one base station. Crouch behind a couch and your foot tracker goes blind; the IMU coasts for a moment, then the pose snaps when a sweep is reacquired.
• Reflections. Mirrors and glass bounce the laser sweep, producing ghost hits. That is why setup guides say to cover mirrors.
• IMU drift. Gyroscopes integrate angular velocity, so tiny bias errors accumulate into yaw drift. Lighthouse corrects it every ~8–16 ms; IMU-only suits must periodically face forward and reset.
• Calibration slop. Trackers strap to clothing, and clothing moves. A hip tracker that rotates 10 degrees on a waistband makes the whole avatar lean. Good software lets friends or the wearer re-calibrate with a T-pose in seconds.

Quick self-check

Glossary

• 6-DoF — six degrees of freedom: position (x, y, z) plus rotation (pitch, yaw, roll).
• Pose — the complete set of joint positions and rotations for one frame.
• FBT — full-body tracking; commonly 3-point (no FBT), 6-point (hip + feet), or 10–11 point setups.
• IK / FK — inverse kinematics works backward from an end effector to joint angles; forward kinematics goes the other way.
• Motion-to-photon — time from your body moving to the display showing that movement.
• End effector — the tracked point an IK chain must reach, e.g. your hand or foot.
• Sync flash / sweep — the lighthouse timing reference and the rotating laser plane that follows it.

From mocap stages to living rooms

• Optical mocap (film/games): dozens of ceiling cameras track retroreflective dots at sub-millimetre accuracy. Superb, but a studio costs more than a house.
• 2016, Lighthouse 1.0: Valve inverted the problem - instead of cameras watching markers, cheap sensors on the body watch structured light from two small emitters. Room-scale 6-DoF for consumers.
• 2018+, Vive Trackers: the same pucks that tracked controllers strap to hips and feet, and social VR communities turned full-body avatars into a culture of dance, mirrors, and motion-captured performance.
• 2022+, IMU suits: open-source projects proved that gyro-only trackers plus clever skeletal constraints get 90% of the experience with zero base stations.
• Now: headset cameras plus ML body estimation infer legs with no hardware at all - the guessed joints in this demo are increasingly guessed by a neural network instead of simple heuristics.

Common setups and what they cost you

Try reproducing each row with the toggles above and watch the fidelity meter agree.