Teacher-guided corrective residuals support reflex-like recovery in simulated locomotion

Claim. In BrainCAD/ReflexBench locomotion experiments, cerebellar-inspired corrective modules improved reflex-like recovery only when trained with a structured teacher signal.

Reflexes are often described as simple loops, but real movement is not simple. A body with many joints and contacts has many ways to fail: a foot can slip, sensory input can become unreliable, or a shove can arrive while the controller is already committed to a step. Nervous systems solve this with layered circuits. Spinal circuits provide fast sensorimotor pathways, cortex supports flexible voluntary control, and the cerebellum is strongly associated with internal models and error-based motor adaptation Wolpert et al., 1998Ito, 2008Seidler et al., 2013.

Motivated by this organization, we implemented several corrective circuits in BrainCAD BrainCAD Project, 2026 and tested them in ReflexBench, a perturbation suite for MuJoCo locomotion. The first lesson was negative: PPO alone Schulman et al., 2017 often failed to discover a small, fast corrective action, even when the architecture was biologically plausible. The successful variant used a teacher. A robust policy was first trained under perturbations, and a cerebellar residual was then trained to imitate the teacher’s correction relative to a nominal cortex, following the broad logic of residual policy learning and distillation Johannink et al., 2018Rusu et al., 2015Schmitt et al., 2018. We therefore frame the result as a learning-signal result, not as evidence for a new principle of cerebellar biology.

The model has three pieces. A cortex policy proposes a base action,

where $o_t$ is the observation. A teacher policy, trained separately with perturbations, produces $u_t^{teacher}$. During imitation, the cerebellar residual is trained toward

At evaluation time, the residual controller executes

where $\alpha_t \in [0,1]$ is an optional engagement gate. A later consolidation step removes the residual machinery by training a new cortex to imitate the final action directly.

RL-only correction was not enough. We first tested spinal gain gating, residual correction trained only by PPO, curriculum perturbations, basal-ganglia-like engagement gates, and stability auxiliary terms. These variants were useful diagnostics, but none gave a reliable causal reduction in fall AUC. Thus, the circuit diagram alone was not the main explanation. The key variable was whether the module received enough information to learn a useful correction at the right time.

Teacher-guided residuals improved survival. The teacher-guided residual was tested causally: the same checkpoint and the same perturbation schedule were evaluated with the residual enabled or disabled. In the N=20 Humanoid campaign, fall AUC decreased for push (-0.313, 95% CI [-0.491, -0.147]), sensor (-0.447, 95% CI [-0.553, -0.338]), and slip (-0.325, 95% CI [-0.511, -0.155]) perturbations, where negative numbers mean fewer falls. Return AUC increased for push (58.8, 95% CI [27.6, 88.4]), sensor (103.6, 95% CI [82.7, 123.5]), and slip (61.4, 95% CI [27.4, 91.6]). Perturbation sanity checks verified that pushes, friction changes, and sensor corruption were actually applied, not only logged.

Teacher quality mattered. A strong teacher produced stabilizing residuals. A deliberately weak teacher produced residuals that could become harmful. This argues against a simple capacity explanation: the student learned a direction of correction from the teacher, and a bad teacher could teach the wrong direction.

Corrections could be consolidated. We trained a cortex-only policy to imitate the final action of the residual controller. This consolidated policy no longer used an online cerebellar residual, but it retained most of the fall reduction in Humanoid and generalized under a long-push schedule. This suggests that the residual pathway can act as a training scaffold: useful during learning, but not always necessary at inference.

Exploratory morphology analysis should be interpreted cautiously. Ant and Humanoid showed strong fall reductions, Walker2d replicated, and Hopper exposed a stability-performance tradeoff. This supports a modest morphology-dependent control hypothesis: layered corrections may become more useful as the body creates more recovery states for the controller to handle, but four simulated bodies cannot establish an evolutionary scaling law.

Methods in brief¶

Experiments used BrainCAD with MuJoCo locomotion environments Todorov et al., 2012Brockman et al., 2016. ReflexBench applied external pushes, friction reductions, and observation corruption. Fall AUC was computed as the area under the fall-probability curve across the predefined perturbation-severity grid: for Humanoid, push magnitudes 0-1 crossed with duration, slip severities 0-1.25 crossed with patch settings, and sensor dropout bursts 0-0.4. Return AUC was computed analogously from mean return across the same severity grid. Curves were computed per seed before paired ON-OFF deltas were summarized with bootstrap confidence intervals. Lower fall AUC is better, while higher return AUC is better. Source notebooks and validation summaries are included in the micropublication package.