Technology
Reinforcement Learning for Humanoid Robots: How Machines Learn to Move
Reinforcement learning has replaced classical control as the dominant method for humanoid robot locomotion. Here is how RL policies are trained in simulation, transferred to real hardware, and deployed at scale in factories and warehouses.
In April 2026, a humanoid robot completed a half-marathon in Beijing, running more than 21 kilometers powered by reinforcement learning policies trained entirely in simulation. Not pre-programmed motion sequences. Not carefully scripted choreography. A policy that learned to run by trying millions of times in a virtual world, then transferred to real hardware.
Reinforcement learning (RL) has become the dominant technique for teaching humanoid robots to move, balance, recover from falls, and execute complex physical tasks. This article explains how RL works for robotics, why it outperforms classical control in unstructured environments, the key technical challenges, and what the current state of deployment tells us about where the field is heading.
AI-generated image
Reinforcement learning policy networks learn through reward signals — the core mechanism behind modern humanoid locomotion. Credit: AI-generated
Key Stats
21 km
Half-Marathon Completed (Beijing, 2026)
1B+
Sim Steps per Training Run (Isaac Lab)
18K+
Humanoid Units Shipped in 2025 (China)
$38B
Physical AI Market by 2035
How Reinforcement Learning Works for Robots
Reinforcement learning is a training method where an agent learns by taking actions in an environment and receiving numerical rewards or penalties based on outcomes. There is no labeled dataset of correct movements, no human expert demonstrating each motion. The robot (or its simulated equivalent) starts by moving randomly, gradually discovers which actions lead to rewards, and iteratively refines a policy that maps observations to actions to maximize cumulative reward over time.
For a bipedal robot learning to walk, the setup looks roughly like this: the robot's observation includes joint angles, velocities, contact forces from foot sensors, and inertial measurements from an IMU. The action space is the torque commands sent to each motor. The reward function gives positive feedback for moving forward at target speed, staying upright, and minimizing energy consumption, and penalizes falls, excessive joint torques, and unnatural gait patterns.
The RL Training Loop
• Policy network: A neural network (typically a multi-layer MLP or transformer) that takes observations as input and outputs actions.
• Rollout: The policy acts in the simulated environment for a fixed number of timesteps, generating experience data.
• Reward computation: Each step's reward is computed from the robot's state and the reward function design.
• Policy gradient update: The policy network is updated using algorithms like PPO (Proximal Policy Optimization) to increase the probability of actions that led to high rewards.
• Repeat: Billions of environment steps are accumulated before the policy is stable enough for real-world transfer.
The key insight that made RL practical for full-body robot control is the use of physics simulators that can run thousands of robot instances in parallel, each accumulating experience simultaneously. NVIDIA's Isaac Lab runs on GPU clusters and can simulate billions of timesteps per training run, compressing years of real-world robot time into hours of compute time. This is why simulation-to-real (sim-to-real) transfer is the central strategy for robot locomotion training rather than collecting data on physical hardware.
Under the Hood: Sim-to-Real Transfer
Training in simulation is efficient but introduces a fundamental problem: real-world physics never exactly matches the simulator. Motor friction, cable elasticity, sensor noise, floor compliance, and hundreds of other physical details differ between the simulation model and the physical robot. A policy that achieves perfect walking in simulation may fail immediately when deployed on real hardware if these gaps are too large.
AI-generated image
Sim-to-real transfer bridges virtual training environments and physical deployment — the key technical challenge in RL-trained robotics. Credit: AI-generated
The standard solution is domain randomization: during training, the simulator randomly varies physical parameters within plausible ranges. Motor friction might vary from 0.5x to 2x nominal. Floor friction might vary from 0.3 to 1.2. The robot's mass might shift by plus or minus 20%. By training across this distribution of environments, the policy learns to be robust to variations rather than optimizing for any single set of conditions. When deployed on real hardware, the real parameters fall within the training distribution, and the policy generalizes.
Domain randomization alone is often not sufficient for high-performance locomotion. Several complementary techniques have emerged:
• System identification: Precisely measuring the real robot's physical parameters (mass distribution, joint stiffness, motor curves) and incorporating them into the simulator to narrow the reality gap.
• Adaptive policies: Training the policy with a latent variable representing environmental conditions, allowing the policy to implicitly identify which environment it is in and adapt.
• Online fine-tuning: Collecting data on the real robot and performing limited policy updates to close residual gaps that simulation cannot capture.
• Privileged learning: Training with access to ground-truth simulation state during training, then distilling to a policy that only uses observable sensor inputs for deployment.
| Simulator / Framework | Developer | Key Capability | Used By |
|---|---|---|---|
| Isaac Lab | NVIDIA | GPU-parallel, 1B+ steps | Figure AI, Agility, researchers |
| MuJoCo | Google DeepMind | High-fidelity contact physics | Research standard |
| Booster Gym | Booster Robotics | End-to-end RL for locomotion | Unitree H1/G1 platforms |
| Isaac GR00T N1 | NVIDIA | Foundation model + open datasets | General humanoid platforms |
Who's Leading RL-Driven Humanoid Robotics
The companies deploying RL-trained humanoids commercially today share a common infrastructure choice: NVIDIA's Isaac ecosystem for simulation and training, combined with proprietary policy architectures and reward function designs.
• Figure AI: Uses pure RL-trained locomotion (sim-to-real via Isaac Lab) on Figure 02, deployed at BMW's Spartanburg plant. Figure confirmed that its natural walking gait is entirely RL-derived, not motion-captured. Backed by $675 million in Series B funding.
• Agility Robotics: Digit humanoid uses a whole-body control foundation model trained with RL. Agility's approach integrates locomotion and manipulation under a unified RL policy, enabling Digit to handle Amazon warehouse tasks. Subsidiary of Hyundai.
• Unitree Robotics: The G1 and H1 platforms are the most widely deployed RL locomotion testbeds in research. Unitree's open-source compatibility with Booster Gym and Isaac Lab has made them the default hardware for RL locomotion research globally. Chinese firms shipped 18,000+ humanoid units in 2025.
• Boston Dynamics: The electric Atlas uses a combination of model predictive control (MPC) and RL components. Boston Dynamics has more than 30 years of bipedal locomotion research, but even they are integrating RL-trained components for manipulation and recovery behaviors.
• Tesla: Optimus development uses a large-scale RL training stack with data collected inside Tesla's own factories as the primary real-world signal. Tesla's advantage is vertical integration: the same AI chip team, compute infrastructure, and data flywheel used for Autopilot supports Optimus training.
What RL-Trained Robots Mean for Manufacturing and Beyond
The practical significance of RL-trained locomotion is that it makes humanoid robots genuinely deployable in unstructured environments, not just on flat floors with defined waypoints. A robot that can recover from a shove, step over a cable, maintain balance on a sloped factory floor, and adapt its gait to different surface materials is qualitatively more useful than one that requires precisely surveyed terrain.
For manufacturing, this translates to expanding the range of tasks a humanoid can handle without facility modification. Early deployments at BMW (Figure AI) and Amazon (Agility's Digit) are in structured factory environments with deliberately defined operating zones. As RL policies improve, the definition of "structured" expands, and the required facility preparation shrinks.
The Data Flywheel Emerging
Companies deploying humanoids at scale are beginning to accumulate the real-world data needed to fine-tune and improve their RL policies. Each robot in the field is a data-collection point. As fleet sizes grow from tens to thousands of units, the data advantage compounds. The companies that deploy earliest build a training data moat that is increasingly difficult for later entrants to close.
The labor market implications are real but gradual. Current RL-trained robots handle specific structured tasks: part transfers, quality inspection assistance, box moving in constrained zones. They are not general labor replacements. The 2026 deployment window targets tasks that are physically demanding, repetitive, and already hard to staff, a category that represents about 15-20% of manufacturing labor by volume but a much smaller share by economic value. Broader displacement, if it occurs, is a 2030s concern at the current capability trajectory.
What's Coming Next: The 12-Month Outlook
The next 12 months in RL-trained humanoid robotics center on three developments: larger-scale deployments generating real performance data, the integration of RL locomotion with vision-language-action (VLA) models for task-level reasoning, and the emergence of industry-specific RL policies trained on domain data rather than general simulation.
NVIDIA's GR00T N1 foundation model and WholeBodyVLA (presented at ICLR 2026) represent the frontier of unified locomotion-plus-manipulation policies. Rather than a separate locomotion controller and a separate manipulation controller, these architectures train a single policy conditioned on language instructions that can navigate, pick, place, and recover from disturbances under a common framework. Early results are promising but deployment-scale validation is still pending.
Watch for fleet announcements: Figure AI, Agility, and 1X Technologies have all indicated expanding deployments in 2026. When any company announces 1,000+ units in operation at a single customer site, it signals a transition from pilot to operational scale, which changes the economics and the data flywheel dynamics.
Frequently Asked Questions
Is reinforcement learning the same as the AI used in ChatGPT?
They share underlying neural network architectures but work differently. Large language models like those powering ChatGPT are trained primarily via supervised learning on text, with RL from human feedback (RLHF) used as a fine-tuning step. Robot locomotion RL trains policies from scratch using simulated physical experience and reward functions, without any labeled human data. The math (policy gradients, PPO, etc.) is distinct from language model training, though both fields are increasingly borrowing techniques from each other.
Why not just program the robot to walk instead of using RL?
Classical control methods, including model predictive control (MPC) and zero-moment point (ZMP) controllers, work well on flat, predictable terrain with accurate robot models. They struggle with unmodeled disturbances, irregular terrain, and the combinatorial complexity of whole-body control when both locomotion and manipulation must be coordinated. Writing explicit rules for every situation is intractable. RL discovers behaviors that human engineers would never think to code and generalizes better to novel conditions. Boston Dynamics used classical control for decades and built capable robots; the shift to RL-augmented methods is because the remaining hard problems, recovery from falls, unstructured terrain, loco-manipulation, are exactly where RL excels.
How much compute does it take to train a humanoid locomotion policy?
A competitive locomotion policy typically requires training on 1-10 billion simulated environment steps. On NVIDIA's Isaac Lab running on an A100 cluster, that translates to roughly 4-24 hours of compute time. A single A100 GPU runs around $2-3 per hour on cloud providers, so a full training run costs in the hundreds to low thousands of dollars in compute. This is cheap enough that companies run many parallel experiments. Fine-tuning on real-world data adds cost but is typically done with much smaller datasets. The total training compute budget for a state-of-the-art locomotion policy is modest compared to large language model training.
What is the biggest unsolved problem in robot RL today?
Contact-rich manipulation, specifically dexterous hand control for tasks requiring precise force feedback. Locomotion is largely a solved research problem, though engineering it to product-quality reliability is still difficult. Hands remain the frontier. Grasping arbitrary objects, using tools, and performing assembly tasks that require sub-millimeter precision are all substantially harder than walking. The tactile sensing hardware needed to provide the feedback signal for hand RL is also immature compared to the IMUs and encoders used for locomotion.
What cost should buyers expect for RL-trained humanoid robots in 2026?
Enterprise pricing for commercial deployments is not publicly disclosed, but industry estimates from IDC and others put early commercial units in the $50,000-$200,000 range depending on capability tier and contract structure. Chinese manufacturers, including Unitree, have targeted sub-$20,000 price points for their G1 platform at volume. IDC estimates costs could drop to roughly $13,000 per unit for high-volume Chinese production by 2026. US and European robots remain significantly more expensive due to labor and component costs.
The 12-Month Outlook
Reinforcement learning has moved humanoid robotics from a research curiosity to a commercially deployed technology within roughly five years. The combination of GPU-parallel simulation, sophisticated reward engineering, and domain randomization has solved the locomotion problem well enough to support real factory deployments at BMW, Amazon, and an expanding list of industrial customers.
The next phase is about scale, data, and the integration of locomotion with task-level intelligence. Companies deploying hundreds of robots in 2026 will be training on millions of hours of real-world data by 2027, compounding advantages that pure simulation cannot replicate. Watch the fleet size announcements and the manipulation benchmark results over the next 12 months. Those numbers tell the story of how fast this technology is actually maturing.
The Bottom Line: Reinforcement learning is the engine behind every commercially deployed humanoid robot in 2026, and the gap between RL-trained and classically programmed robots is widening, not narrowing. Companies that control the data flywheel from real-world deployments will set the capability floor for the next generation of physical AI.
For buyers evaluating humanoid robots today, RL capability is table stakes. The differentiating questions are now about reliability at scale, integration with existing factory control systems, and the vendor's retraining infrastructure. The robots that walk well enough. The question is which ones can learn fast enough.