Technology
How Humanoid Robots Learn: Sim, Imitation, and Real Data
Humanoid robots aren't programmed — they're trained. Sim-to-real transfer, imitation learning, and teleoperation data pipelines are the methods producing today's commercial capabilities. Here's how they work.
A humanoid robot that can pick up a cup, load a dishwasher, or sort packages on a warehouse conveyor has not been programmed with explicit rules for each task. It has been trained — a process that involves millions of simulated trials, hours of human demonstrations, and reinforcement learning loops that run continuously across server farms. How robots learn is the central engineering challenge of the humanoid era, and the methods being used today are producing capabilities that were not achievable five years ago.
The dominant training paradigm combines two approaches: sim-to-real transfer, where robots practice in physics simulation before ever touching real hardware, and imitation learning, where robots learn by watching and mimicking human demonstrations. Understanding how these methods work — and where they still fail — is essential context for evaluating every humanoid robot company's progress claims.
AI-generated image
Teleoperation data collection: a human operator wears haptic gloves while a robot mirrors the movements, capturing training demonstrations. Source: AI-generated
Key Stats
1000x
Faster-than-real-time simulation speed for training runs
~50h
Human demo data needed to teach a new manipulation task (typical)
10K+
Simulation instances run in parallel during massively parallel RL training
95%+
Task success rate achieved in sim vs. ~60-80% on real hardware (sim-to-real gap)
Why You Can't Just Program a Robot
The traditional approach to robot programming is explicit: write code that says "if the object is at position X, move arm to position Y, close gripper to force Z." This works for industrial robot arms bolted to the floor of a car factory, repeating the exact same weld at the exact same position millions of times. It fails the moment you introduce variability — objects of different shapes, sizes, orientations; surfaces that are cluttered; lighting that changes; parts that are slightly mis-positioned.
Humanoid robots are intended to operate in exactly those variable, unstructured environments. A warehouse picking robot needs to handle 10,000 different SKUs that arrive in unpredictable orientations. A home robot needs to deal with dishes stacked differently every time. Writing explicit rules for every possible configuration is not feasible. The solution is machine learning: let the robot figure out what to do through exposure to data, trial, and feedback.
But learning on a physical robot is expensive and slow. Physical robots break. Trial-and-error learning in the real world requires a human to reset the environment after each failed attempt. A robot learning to pick up objects might need tens of thousands of trials before it develops reliable behavior. Running those trials on real hardware at real-world speed would take months or years. Simulation solves this by running trials millions of times faster than real time, in parallel, without physical wear.
Simulation Training: The Physics Engine Foundation
Sim-to-real transfer starts with a physics simulation that accurately models robot dynamics, contact forces, and the properties of the objects the robot will manipulate. The major simulators used in robotics research and industry include Isaac Sim (NVIDIA), MuJoCo (now maintained by Google DeepMind), PyBullet, and Genesis (a newer GPU-accelerated option). Each offers different tradeoffs between simulation fidelity, speed, and ease of integration with learning frameworks.
In simulation, a robot can attempt a task — picking up a cup, stacking boxes, turning a valve — and receive immediate feedback on whether it succeeded. A reinforcement learning (RL) algorithm adjusts the robot's control policy based on rewards (success) and penalties (failure), gradually improving performance. Running 10,000 parallel simulation instances simultaneously, each with slightly randomized environment conditions, a good training cluster can accumulate millions of training episodes in hours rather than years.
AI-generated image
Massively parallel simulation training: thousands of robot instances learning simultaneously in a physics engine. Source: AI-generated
The critical technique that makes sim-to-real transfer work is domain randomization. Rather than training on a single fixed simulation environment, the training process randomizes physics parameters within plausible ranges: object mass, surface friction, joint damping, lighting conditions, camera positions, sensor noise levels. A policy trained across thousands of randomized variations becomes robust to the discrepancies between the simulation and the real world — because it has never relied on any single fixed set of conditions.
The Sim-to-Real Gap
Even the best physics simulations are imperfect models of reality. Soft contact dynamics — how a gripper deforms when squeezing a compliant object — are notoriously difficult to simulate accurately. Electrical noise in real sensors differs from simulated noise. Joint friction is inconsistent in real hardware. The policy that achieves 98% success in simulation typically achieves 60-80% on the real robot without additional adaptation. Closing this gap is an active research area at every major lab.
Imitation Learning: Teaching by Demonstration
Reinforcement learning from scratch is powerful but brittle for complex manipulation tasks. Defining a reward function that correctly incentivizes the exact behavior you want — without unintended shortcuts — is harder than it sounds. A robot rewarded only for getting a box from point A to point B might learn to knock it there rather than pick it up. Reward shaping is an art.
Imitation learning bypasses this problem by giving the robot expert demonstrations to learn from. A human operator performs the target task while the robot (or a simulation of it) records the full state-action trajectory — joint positions, gripper forces, camera images, every sensor reading at every timestep. The robot then trains a policy to reproduce these trajectories, learning the behavior directly rather than discovering it through random exploration.
The main imitation learning approaches used in humanoid robotics are:
• Behavioral Cloning (BC): The simplest form. Train a neural network to map observations to actions using supervised learning on the demonstration data. Fast to implement, but brittle — the robot fails when it encounters states not covered by the demonstrations, because it has no mechanism to recover from out-of-distribution situations.
• DAgger (Dataset Aggregation): An iterative improvement on BC. The robot runs its current policy and, when it encounters uncertain states, queries an expert for the correct action. Those new state-action pairs are added to the training dataset. Each iteration produces a better-rounded policy that handles more edge cases.
• ACT (Action Chunking with Transformers): Developed at Stanford's IRIS Lab, ACT trains a transformer model to predict sequences of future actions ("chunks") rather than single-step actions. By predicting 50-100ms of future behavior at once, ACT produces smoother, more coordinated movements and is more robust to delays between sensing and actuation. Figure AI and several other companies use transformer-based imitation learning architectures similar to ACT.
• Diffusion Policy: Uses diffusion models (the same class as image generation models) to learn a distribution over possible actions. Rather than predicting a single "best" action, diffusion policy models the full probability distribution, making it better at handling tasks with multiple valid solutions. MIT and Stanford research groups have demonstrated strong dexterous manipulation results with diffusion policy approaches.
Data Collection: The Teleoperation Pipeline
Getting high-quality training demonstrations at scale is one of the most operationally challenging parts of humanoid robotics. The dominant method today is teleoperation: a human operator wears a haptic exoskeleton or uses hand-tracking devices to control the robot's movements, while the robot records every joint angle, camera image, and force reading at high frequency (typically 50-200 Hz).
Physical Intelligence (Pi) — the robotics AI startup that raised $400 million in November 2024 at a reported $2.4 billion valuation — has built one of the largest teleoperation data collection operations in the industry. Their pi0 model was trained on data from multiple robot embodiments, making it one of the first foundation models designed to transfer across different robot hardware.
Figure AI uses a combination of teleoperation and in-factory data collection from its deployed BMW robots. Each Figure 02 robot at the BMW Spartanburg plant runs Figure's learning infrastructure, uploading task demonstrations and performance data continuously. This creates a feedback loop: deployed robots generate training data, which improves the policy, which gets pushed back to deployed robots as over-the-air updates.
1X Technologies has taken a different approach, using human-like bipedal robots as embodied data collection platforms. Their Neo robot is designed to be operated by remote human operators for data collection at scale, with the long-term goal of bootstrapping an autonomous capability from that demonstration data.
| Company | Training Approach | Key Data Source | Foundation Model? |
|---|---|---|---|
| Physical Intelligence (Pi) | Imitation + RL fine-tuning | Multi-embodiment teleoperation | Yes (pi0) |
| Figure AI | Imitation + continuous learning | Teleoperation + factory deployment | In development |
| Tesla (Optimus) | RL from video + simulation | FSD video data + sim | Partially |
| Boston Dynamics (Atlas) | RL + motion planning | Simulation (MuJoCo) | No (task-specific) |
| Agility Robotics (Digit) | RL for locomotion, imitation for manipulation | Simulation + teleoperation | No (task-specific) |
Combining Sim and Real: The Modern Playbook
The most effective training pipelines today don't choose between simulation and real-world data — they use both in sequence. A common workflow:
• Phase 1 — Massive sim pretraining: Train a policy in simulation with domain randomization, running millions of episodes across thousands of parallel environments. The policy learns basic task structure and physical intuition.
• Phase 2 — Imitation learning fine-tuning: Collect several hundred to several thousand teleoperated demonstrations on the real robot. Fine-tune the sim-pretrained policy on this real-world data to close the sim-to-real gap.
• Phase 3 — RL from deployment: Deploy the robot and collect performance data during real operation. Use online RL or offline RL (learning from stored experience) to continue improving the policy without requiring explicit demonstrations.
• Phase 4 — Continuous fleet learning: As a robot fleet scales, the aggregate data from all deployed robots feeds back into the training pipeline, creating a flywheel where more deployments produce better policies produce better deployments.
This flywheel effect is why scale matters so much in the current race. Companies with more deployed robots accumulate real-world data faster, train better policies, deploy more confidently, and deploy more robots. Tesla's advantage in autonomous driving — where the fleet of millions of vehicles generates petabytes of real-world driving data daily — is the model that every humanoid robotics company is trying to replicate at the embodied AI level.
Frequently Asked Questions
How long does it take to train a humanoid robot to do a new task?
It depends heavily on the task complexity and the training approach. With a good foundation model pretrained in simulation, fine-tuning on a new manipulation task can require as few as 50-200 teleoperated demonstrations — roughly 2-10 hours of data collection — plus compute time for fine-tuning (hours to days on a modern GPU cluster). From-scratch training for a genuinely novel task can require months of simulation pretraining first. The time-to-task is dropping rapidly as foundation models improve.
Why not just use ChatGPT-style language models to control robots?
Large language models (LLMs) and vision-language models (VLMs) are increasingly used as the "brain" layer for task planning and instruction understanding, but they can't directly control physical actions at the millisecond timescales needed for manipulation. A VLM might understand "pick up the red cup and place it in the sink" — but executing that instruction requires a separate low-level control policy that handles the actual motor commands, force feedback, and real-time adjustments. Most current systems use VLMs for high-level planning and task-specific learned policies for low-level execution.
What is the sim-to-real gap and how big is it?
The sim-to-real gap is the performance drop when a policy trained in simulation is deployed on a real robot. A policy might achieve 95% task success rate in simulation and only 60-70% on the real hardware, because the simulation's physics don't perfectly model real-world contact, sensor noise, and mechanical variability. Domain randomization (training across many randomized simulation parameters) and real-world fine-tuning are the main techniques for closing this gap. State-of-the-art methods can get real-world performance within 10-15 percentage points of simulation performance for many manipulation tasks.
Do robots trained in simulation need to re-learn everything for a new robot body?
Traditionally yes — policies are specific to the robot they were trained on. This is why Physical Intelligence's pi0 foundation model is significant: it was trained across multiple robot embodiments (different arm designs, gripper types, body forms) and can transfer capabilities across hardware. The field is moving toward "embodiment-agnostic" foundation models that encode physical reasoning separately from hardware-specific control parameters, but robust cross-embodiment transfer remains an open research problem.
How does Tesla train Optimus differently from other companies?
Tesla's approach leverages its autonomous driving infrastructure heavily. The same neural network architecture (transformer-based, trained on video data) used for Full Self-Driving is being adapted for Optimus. Tesla has also demonstrated using human video data — videos of humans performing tasks from the internet and from internal data collection — to bootstrap robot training without requiring robot-specific teleoperation for every task. Whether this video pretraining approach scales to reliable manipulation in unstructured environments remains to be seen in production deployments.
The 12-Month Outlook
The training pipeline for humanoid robots is improving faster than the hardware. Compute costs for simulation training continue to fall as GPU density increases and specialized training accelerators (NVIDIA H100, H200) become more available. Foundation models for robot manipulation are maturing rapidly — Physical Intelligence's pi0, released in late 2024, demonstrated generalization across tasks and embodiments that would have been state-of-the-art research results two years ago.
The key 2026 milestones to watch are deployment scale and task diversity. If Figure AI expands its BMW deployment from 70 to 1,000+ robots by end of 2026 as planned, that fleet will generate more real-world manipulation data in a single quarter than the entire robotics research community has collected historically. The resulting policy improvements should be measurable. Tesla's Optimus production ramp — targeting internal deployment at Gigafactories through 2025-2026 — will be the other major data point: whether training on video-scale data can produce reliable factory workers without the teleoperation bottleneck.
The Bottom Line: The companies that win the humanoid robot race will be the ones that most efficiently convert compute and human demonstration time into reliable robot capability — and the training pipeline, not the hardware, is where that conversion happens. Sim-to-real transfer and imitation learning are not solved problems, but they are good enough to produce commercial deployments today, and they are improving at a rate that suggests meaningful capability jumps every 6-12 months.
For anyone evaluating humanoid robot companies, the right questions are: How many demonstrations does it take to teach a new task? How well does performance hold up outside the training distribution? And how fast is the fleet learning loop running? The answers to those questions matter more than any single hardware spec.