Physical AI Monograph

Physical AI Monograph

Concepts, Techniques, and Challenges

By Physical Intelligence Team,Shanghai AI Laboratory

A monograph on intelligence that perceives, models, and acts upon the physical world, connecting technical foundations with embodied systems and the challenges of Physical AGI.

Read the monographTable of contentsResourcesBibTeX

Final PDF edition · October 19, 2026

Parts
3
Chapters
12
Pages
388
Book cover of Physical AI Monograph
Physical AI Monograph

About the Book

A Unified View of Physical AI

What is Physical AI?What does its technical landscape look like?What challenges stand between today's systems and general physical intelligence?

Physical AI is developing rapidly, yet its definition, technical landscape, and central challenges remain open questions. This monograph brings these questions together through a unified framework: artificial systems that perceive, model, and act upon the physical world through continuous feedback under real-world constraints.

It distinguishes Physical AI from digital AI, embodied AI, cyber-physical systems, and digital twins, and develops its foundations across physics, control, cognition, learning, and reasoning. The technical survey connects multimodal perception, physical data and simulation, world models, embodied foundation models, motor control, physical agents, and hardware–software co-design.

Applications span robotics, manufacturing, infrastructure, scientific experimentation, and autonomous services. The monograph examines sim-to-real transfer, reliability, real-time execution, evaluation, safety, governance, and sustained deployment, before considering the research directions toward more general physical intelligence and Physical AGI.

Figure 1.3: perception, world modeling, decision-making, and control connected through physical feedback, data, and hardware
The system-level framework of Physical AI: models and control connected to the physical world through feedback. From Figure 1.3 of the monograph.

The Organizing Idea

Perception, prediction, decision-making, action, and adaptation form a continuous loop through the physical world.

An organizing framework introduced in Chapter 1.

01

Perceive

Estimate physical states from vision, touch, force, proprioception, and other sensing modalities.

02

Model & Predict

Represent physical relationships and anticipate the consequences of possible actions.

03

Decide & Act

Connect goals, planning, and control to feasible interventions in the physical world.

04

Adapt Through Feedback

Verify outcomes, revise behavior, and learn across control, task, and deployment timescales.

Inside the Monograph

Table of Contents

Three parts trace a path from concepts and foundations, through the technical stack, to future directions.
Expand a chapter to explore its sections.

Part I2 chapters

Overview and Foundations

Definitions, boundaries, and the foundations in physics, control, cognition, and learning.

01Introduction to Physical AI6 sections p. 11
  1. 1.1Background: A Paradigm Expansion of AI11
  2. 1.2Physical AI: Definition and Boundaries14
  3. 1.3Core Elements of Physical AI17
  4. 1.4The Role of Physical AI in the Evolution of AI20
  5. 1.5The Application Landscape of Physical AI21
  6. 1.6Summary and Organization of the Monograph27
02Technical Foundations4 sections p. 29
  1. 2.1Mathematical Descriptions of the Physical World29
  2. 2.2Control Theory and Autonomous Systems32
  3. 2.3Embodied Cognition and the Development of Intelligence41
  4. 2.4Learning and Reasoning in the Physical World44
Part II7 chapters

Methodology

The technical stack: perception, data, world models, manipulation, motor control, agents, and systems.

03Perception and Cognition4 sections p. 51
  1. 3.1Tasks and Characteristics of Physical Perception51
  2. 3.2Multimodal Perception and Fusion52
  3. 3.33D/4D Spatial Perception58
  4. 3.4High-Level Perception and Cognition60
04Data for Physical AI5 sections p. 66
  1. 4.1Data Foundations for Physical AI66
  2. 4.2Data Collection and Interaction68
  3. 4.3Physics Engines and Generative Worlds76
  4. 4.4Sim-to-Real Transfer and Real-World Alignment82
  5. 4.5Scaling Laws and the Data Flywheel for Physical AI87
05World Models5 sections p. 95
  1. 5.1Concepts and Theoretical Frameworks of World Models96
  2. 5.2Representations and Architectures of World Models100
  3. 5.3Training World Models107
  4. 5.4Using and Validating World Models119
  5. 5.5Current Challenges and Future Outlook125
06Foundation Models for Robot Manipulation6 sections p. 128
  1. 6.1Emergence and Taxonomy128
  2. 6.2Vision-Language-Action Model133
  3. 6.3Pre-Training, Post-Training, and Capability Shaping138
  4. 6.4Multi-Embodiment Modeling and Model Adaptation145
  5. 6.5Model Deployment and Efficiency Optimization152
  6. 6.6Dexterous Manipulation Foundation Models157
07Foundation Models for Whole-Body Motor Control5 sections p. 162
  1. 7.1Scope, Taxonomy, and Control Interfaces162
  2. 7.2Body Dynamics and State Estimation167
  3. 7.3Command-Conditioned Motor Control172
  4. 7.4Motor Pretraining and Dexterous Whole-Body Control178
  5. 7.5Real-Time Execution, Recovery, and Safety Control187
08Physical Agents4 sections p. 193
  1. 8.1Definition of Physical Agent193
  2. 8.2Physical Forms of Physical Agent199
  3. 8.3Physical Goal Representation and Task Grounding207
  4. 8.4Multi-Agent Collaboration and Distributed Physical AI212
09Hardware and Systems7 sections p. 219
  1. 9.1Hardware and Embodiment Taxonomy for Physical AI219
  2. 9.2Sensors, Actuators, and Intelligent Materials223
  3. 9.3Device-Edge-Cloud Collaboration and Real-Time Communication233
  4. 9.4Software Stacks and Development Infrastructure239
  5. 9.5Reliability and Deployment at Scale246
  6. 9.6Case Studies in Dexterous Hand Hardware250
  7. 9.7Case Studies in Humanoid System Integration257
Part III3 chapters

Future Outlook

General physical intelligence, evaluation and governance, and long-term development strategies.

10Physical AGI6 sections p. 263
  1. 10.1Core Capabilities of Physical AGI263
  2. 10.2Systematic Generalization in Open-World Settings266
  3. 10.3Human-AI Collaboration and Social Intelligence268
  4. 10.4Multi-Agent Systems and Collective Physical AI269
  5. 10.5Autonomous Evolution271
  6. 10.6Toward the Ultimate Goal of Physical AGI273
11Evaluation, Safety, and Governance for Physical AI5 sections p. 275
  1. 11.1Physical-World Evaluators275
  2. 11.2Capability Evaluation and Benchmarking279
  3. 11.3Physical Safety and Action Alignment289
  4. 11.4System Safety and Risk Governance295
  5. 11.5Law, Ethics, Standards, and Global Governance300
12Strategy and Civilizational Outlook3 sections p. 304
  1. 12.1Development Strategies for Physical AI304
  2. 12.2Physical AI and Human Civilization309
  3. 12.3Summary and Outlook315
  • Contributorsp. 316
  • Terminology and Abbreviationsp. 317
  • Notationp. 327
  • Referencesp. 343

Expand a chapter to view its sections. Page numbers match the final PDF edition.

Authors

About the Authors

Physical Intelligence Team, Shanghai AI Laboratory

The monograph is authored by the Physical Intelligence Team, Shanghai AI Laboratory. It brings together the concepts, techniques, and challenges of intelligence that interacts with the physical world.

Individual author profiles and acknowledgments will be added when available.

Read & Explore

The Monograph and Reader Feedback

Read the final PDF edition. Errata and reader feedback will be added here.

Open-Access Edition

Read and download the final PDF edition of the monograph.

Read the monograph

— clicks

Errata & Feedback

Corrections and a reader feedback channel will be added here when available.

Coming soon

Publication

Final Edition, with Open Access

The final PDF edition is available to read and download. Publisher and print edition details will be added when confirmed.

Publisher
To be announced
PDF edition date
October 19, 2026
Print edition
Coming soon

Citation

Cite the Monograph

Please use the following citation for the final PDF edition.

Zhang, W., Shen, C., Xue, T., et al. Physical AI: Concepts, Techniques, and Challenges. In press, with open access. 2026.

BibTeX

@misc{zhang2026physicalai,
  author = {Zhang, Weinan and Shen, Chunhua and Xue, Tianfan and Zhao, Bin and Jiang, Bingqi and Guo, Zihan and Cui, Zicai and Li, Yang and Zhou, Ming and Chen, Xinyuan and Ding, Lihe and Chen, Xiao and Wang, Yujin and Guo, Shi and Dong, Junting and Lu, Tao and Shen, Xing and Guo, Haoyu and Mu, Yao and Yu, Haiyong and Ma, Haoxiang and Huang, Fuxian and Yu, Qiaojun and Zhou, Yunsong and Li, Kailin and Wang, Zirui and Xue, Yufei and Zeng, Weishuai and Huang, Tao and Wang, Zhigang and Liu, Junli and Wu, Pengyuan and Zhang, Pingrui and Wang, Haotian and Zhou, Dongzhan and Cao, Jiafei and Yuan, Feng and Ye, Xinyi and Xuan, Quanli and Yu, Mulin and Gu, Shangding and Song, Kaiwen and Wang, Dong and Li, Jiarui and Liu, Kehui and Jia, Zhongjie and Li, Dongjiang and Bao, Jiacheng and Yang, Haoran and Chen, Zhaoqing and Sun, Yiming and Zhou, Bowen},
  title = {{Physical AI}: Concepts, Techniques, and Challenges},
  year = {2026},
  url = {https://internrobotics.github.io/physical-AI-book/paper/Physical-AI-Monograph.pdf},
}

Citation details refer to the final PDF edition dated October 19, 2026.

Contact

Get in Touch

For questions, corrections, or feedback about the monograph.

InternRobotics

physicalai@pjlab.org.cn