RoboAtom: Large-scale Atomic-Action Supervision for Robot Learning

Paper GitHub Dataset Models

Abstract

Fine-grained action annotations can support skill reuse and compositional generalization in robot policies. However, robot datasets typically provide task-level instructions without subtask annotations. Although some include subtask labels, inconsistent formats and conventions hinder cross-source joint training. We propose RoboAtom-Data, which adds unified, time-aligned atomic-action annotations to existing data from multiple sources, covering 1.45 million trajectories, 6.88 million skill-labeled segments, and 16,882.1 hours. Our pipeline combines a unified protocol, large-scale manual annotation, propagation across repeated executions with the same ordered skill sequence, and multi-stage quality control. We evaluate the dataset through automatic atomic-action annotation and robot control. We construct RoboAtom-Bench to standardize evaluation of skill recognition and temporal boundary localization. Trained on the annotated data, RoboAtom-Anno achieves better automatic annotation performance than all evaluated open- and closed-source models on both RoboAtom-Bench and the external WGO-Bench. For robot control, atomic-action supervision paired with hierarchical execution improves task success over task-level baselines on LIBERO-Long, LIBERO-Plus, and RoboTwin

Annotated examples

Pick Picks up the first matryoshka doll

Order five nesting dolls

Pick Picks up the first bread slice and the meat patty

Make a sandwich

Open Opens the dishwasher door

Load a dishwasher

RoboAtom-Data

RoboAtom-Data unifies atomic-action annotations across robot datasets. Human-reviewed seed annotations, same-task extension, and quality control produce consistent supervision for skill-level learning

Construction

PhysExtend
Model	mIoU	Temporal F1@0.7	Skill F1@0.7	EA (%)	Cost (USD / video hour)
GPT-4o	48.2	20.7	11.1	10	0.888
Gemini 3.1 Pro	61.1	44.9	39.7	39	2.877
GPT-5.6-sol	58.9	37.8	35.0	46	3.510
Qwen3.8-27B	46.1	25.4	19.1	25	0.051
Cortex	66.5	49.8	48.5	37	4.8 × 10^-4
PhysExtend	76.8	71.6	71.0	69	2.8 × 10^-5

Metric definitions

mIoU: Mean temporal intersection over union between ground-truth and predicted segments, ignoring skill labels
Temporal F1@0.7: Segment detection F1 with temporal intersection over union of at least 0.7
Skill F1@0.7: Segment F1 requiring both temporal intersection over union of at least 0.7 and matching skill labels
Episode accuracy (EA): Percentage of trajectories in which every subtask has correct semantics and segmentation

Dataset composition

Datasets with subtask annotations
Dataset	Domain	Hours	Trajectories	Segments	Skills	Tasks	Embodiments
AgiBot World Beta	Real	2,976.4	141,563	1,001,552	77	217	1
AgiBot World 2026	Real	591.6	28,331	327,355	43	37	1
Galaxea Open-World	Real	500.0	19,183	108,775	58	150	1
RoboInter-Data	Real	2,417.1	235,880	752,388	15	232	6
BEHAVIOR-1K 2026	Sim	1,952.9	20,000	406,341	35	100	1
RoboAtom-Data	Mixed	16,882.1	1,446,974	6,876,282	29	3,334	32

RoboAtom-Bench

RoboAtom-Bench evaluates which actions occur, their order, and their temporal boundaries. Its subsets cover short, medium, and long action sequences, together with synchronized multi-camera videos

Concentric distributions of action counts, skill types, camera views, and benchmark subsets with 500 episodes in the center and the complete subset legend on the right

RoboAtom-Anno

RoboAtom-Anno learns from atomic-action supervision to annotate robot videos automatically, predicting skill categories, action descriptions, and temporal boundaries. The results below show annotation performance and inference cost

Performance & cost
Model	RoboAtom-Bench					WGO-Bench Robot
Model	mIoU	Temporal F1@0.7	Skill F1@0.7	Edit score	Cost (USD / video hour)	mIoU	Temporal F1@0.7	Cost (USD / video hour)
GPT-4o	31.5	13.9	7.2	47.7	0.971	46.1	14.6	0.800
GPT-5.6-sol	44.0	28.0	21.4	63.4	4.330	41.9	31.3	3.375
Gemini 3.1 Pro	42.5	24.4	16.1	61.8	4.516	52.1	29.6	4.389
Qwen3.8-27B	38.4	20.1	10.9	56.6	0.103	51.3	24.3	0.096
Qwen3.5-9B	9.6	6.2	3.5	28.7	0.008	12.8	1.7	0.007
RoboAtom-Anno-9B	55.1	41.0	30.1	76.7	0.006	61.7	48.3	0.004
RoboAtom-Anno-27B	56.4	43.2	33.8	76.8	0.078	66.0	49.2	0.054

Metric definitions

mIoU: Mean temporal intersection over union between ground-truth and predicted segments, ignoring skill labels
Temporal F1@0.7: Segment detection F1 with temporal intersection over union of at least 0.7
Skill F1@0.7: Segment F1 requiring both temporal intersection over union of at least 0.7 and matching skill labels
Edit score: Skill-sequence similarity based on normalized Levenshtein distance, independent of timestamps

Dual-system execution

Atomic-action supervision connects high-level planning with robot execution. System 2 selects the current action from observations and execution history, while System 1 converts that instruction into robot movements

Control success rates (%)
Method	LIBERO Long	LIBERO-Plus					RoboTwin
Method	LIBERO Long	Spatial	Object	Goal	Long	Overall	Clean	Random	Average
π_0.5	95.00	93.01	85.23	79.93	78.56	84.05	80.70	48.04	64.37
Ours (dual system)	98.20	92.46	91.86	86.34	82.29	88.18	82.02	50.32	66.17

Citation

If you use RoboAtom in your research, please cite:

@misc{wei2026roboatomlargescaleatomicactionsupervision,
      title={RoboAtom: Large-scale Atomic-Action Supervision for Robot Learning},
      author={Xueyuan Wei and Xuanxu Wang and Xudong Li and Yuping He and Wanshui Gan and Jiaqi Peng and Wenqi Guo and Jianjun Zhou and Junting Dong and Chunhua Shen and Haoyu Guo and Mulin Yu and Tao Lu},
      year={2026},
      eprint={2610.40207},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2610.40207},
}

RoboAtom