NVIDIA Opens Flora Models to Label Robot Video and Score Actions

How Flora Turns Recordings Into Training Signals

A video archive is not automatically a useful training set. Consider a hypothetical two-handed packing task: one hand steadies a carton, the other places an object, and a later movement corrects a poor placement. Labeling the entire recording “pack the box” loses the sequence. Labeling every movement separately can lose the purpose. A learning system needs descriptions at an appropriate level, tied to the right moments.

Flora separates that work into temporal segmentation, action captioning and reward modeling. Temporal segmentation identifies the intervals occupied by meaningful operations. Captioning names the attempted operation. A reward model grades the result or the quality of execution. Keeping those outputs distinct prevents a common conceptual error: an instruction describing an attempted placement does not prove that placement succeeded.

For humanoid developers, the immediate relevance is manipulation data. A recording can inform how a future robot reaches or handles objects without establishing anything about walking, balance or safe collaboration with people. Improvements in the annotation layer need downstream testing before they become claims about a machine's capabilities.

Under the Hood: Three Models, Different Jobs

The Flora-TempSeg model card describes a segmenter built on Qwen3.5-9B using low-rank adaptation, a training method that updates a smaller set of added parameters. It predicts local interval boundaries and continuation flags. The surrounding pipeline joins compatible fragments into episode-level events. It does not generate the action descriptions itself.

The 27-billion-parameter captioner receives a specified target interval plus available neighboring events. Those surrounding events clarify the local operation; they should not become extra actions in the output. The reference input can combine a main camera and two wrist cameras, preserving their roles and timing. Its maximum frame budget is 64 frames across the target and context blocks.

That input design matters when hands obscure objects or a transfer only becomes clear in the next view. It also makes synchronization an engineering requirement. Mixing different moments into one apparent multi-camera observation would give an annotator contradictory evidence. Teams comparing annotation systems should preserve the original video clock and inspect ambiguous examples, rather than assume a fluent caption is accurate.

The rewarder's model card defines an episode outcome scale from 1 to 5. Its dense mode instead grades half-second steps within a 16-second window as failure, suboptimal or normal, with explanations for problematic steps. User-supplied rubrics provide another scoring route. These are different kinds of supervision: an overall partial-success score cannot identify every bad movement, while a locally normal action does not establish completion of the whole task.

NVIDIA's Data Strategy and the Research Around It

NVIDIA says Flora has been used to train GR00T models; its model cards specifically identify GR00T-2. The company is releasing a component of its data tooling, rather than announcing a new humanoid body or customer installation. The distinction gives researchers a concrete artifact to evaluate without turning an internal-use statement into evidence of a commercial fleet.

Reward learning also has an established research ecosystem. Robometer, a collaboration spanning universities and industry researchers, combines frame-level progress supervision with comparisons between trajectories, meaning complete sequences of robot behavior. Its project describes a dataset exceeding one million trajectories across 21 robot embodiments, including suboptimal and failed attempts. That approach highlights why failures are valuable training material, rather than merely recordings to discard.

The RoboReward dataset likewise pairs videos and task descriptions with graded outcomes. These research resources supply alternatives to treating every demonstration as equally good. They also show why evaluation needs several views of quality: understanding the instruction, detecting progress and recognizing failure are related but distinct problems.

Flora's public benchmark table reflects that unevenness. On DenseCaptionBench's human-video subset, NVIDIA reports joint annotation F1 of 37.4 for Flora versus 29.7 for GPT-6 Astra. On the robot subset, the ordering reverses: 51.0 versus 57.9. F1 combines precision and recall; these values are annotation metrics, not percentages of robot tasks completed. The evaluated pipelines also differ in visual inputs and inference budgets. The comparison does not support a universal claim that one model is better at robotics.

What Lower Labeling Costs Could Mean for Humanoids

The paper's cost analysis estimates $0.72 per source-video hour for Flora's segmentation-and-captioning pipeline versus $88.49 for maximum-effort Astra on a fixed 50-episode comparison. It does not price the full reward-labeling suite. The approximately 125-fold headline is conditional: Flora assumes eight GB300 GPUs priced at $10 per GPU-hour. Appendix D says the production GPU-hour denominator is unverified and excludes storage and deployment overhead. The API figure includes known usage, with some requests unpriced. These are different accounting bases, not complete invoices.

On an internal failure-labeling test, the paper reports 53.3% failure-step recall, alongside false failure labels on about 11% of normal steps. That tradeoff matters. Missing failures can contaminate training data; flagging normal behavior can waste review time. Neither outcome is captured by throughput alone.

The economic opportunity is therefore conditional too. If annotation becomes cheaper, a lab could inspect a broader range of tasks, retain more unsuccessful attempts and repeat experiments with revised labels. But it would still need to collect useful footage, respect data rights, maintain camera calibration and establish that the resulting training mixture improves behavior. Cheap labels on poorly chosen recordings do not solve those problems.

For an industrial buyer, the useful question is the total cost of producing a validated policy, not simply the price of describing an hour of video. A hypothetical evaluation should include human correction, decoding and storage, downstream training and physical testing. It should also examine whether savings persist on the buyer's own objects, lighting and work procedures. This is an evaluation framework, not a result demonstrated by the release.

A useful audit would keep examples of disagreement, rather than review only high-confidence successes. For instance, a camera may show an object entering a container while concealing whether it was released. An annotation could describe the attempt correctly yet overstate completion. Reviewing those cases against additional views or a human assessment would help identify which errors are tolerable for dataset search and which would distort a reward signal. The appropriate threshold depends on the later experiment and its consequences.

Frequently Asked Questions

Image attribution: Coolcaesar / Wikimedia Commons, with perspective and crop edits by Jacek Halicki, CC BY-SA 4.0. Photograph dated August 4, 2018. Archival company context, not Flora equipment. The downloaded rendition is unchanged; responsive resizing or display crops and any resulting image adaptation remain under the same license.

What's Coming Next: Test the Labels Against Behavior

The next meaningful evidence would connect annotation choices to downstream robot performance under a disclosed protocol. Researchers can compare manually reviewed labels, general-purpose models and specialized tools on the same held-out recordings, then test policies trained from those alternatives. Reporting correction effort alongside task outcomes would reveal whether apparent data-engine savings survive contact with physical experimentation.

For now, Flora makes an important part of robot development more inspectable. Its strongest contribution is a concrete separation of when an action happens, what is attempted and how well it goes. Its limits are equally concrete: research licensing, imperfect recognition and cost estimates that depend on substantial infrastructure.