Physical AI

Forthcoming monograph · Preprint coming soon

Physical AI

Concepts, Techniques, and Challenges

By Physical Intelligence Team,Shanghai AI Laboratory

A monograph on intelligence that perceives, models, and acts upon the physical world, connecting technical foundations with embodied systems and the challenges of Physical AGI.

Preprint coming soonTable of contentsResourcesBibTeX

Preprint coming soon · Publication details to be announced

Parts
3
Chapters
12
Manuscript pages
362
Book cover of Physical AI: Concepts, Techniques, and Challenges
Physical AI: Concepts, Techniques, and Challenges

About the Book

A Unified View of Physical AI

What is Physical AI? What does its technical landscape look like?
What challenges stand between today's systems and general physical intelligence?

Physical AI is developing rapidly, yet its definition, technical landscape, and central challenges remain open questions. This monograph brings these questions together through a unified framework: artificial systems that perceive, model, and act upon the physical world through continuous feedback under real-world constraints.

It distinguishes Physical AI from digital AI, embodied AI, cyber-physical systems, and digital twins, and develops its foundations across physics, control, cognition, learning, and reasoning. The technical survey connects multimodal perception, physical data and simulation, world models, embodied foundation models, motor control, physical agents, and hardware–software co-design.

Applications span robotics, manufacturing, infrastructure, scientific experimentation, and autonomous services. The monograph examines sim-to-real transfer, reliability, real-time execution, evaluation, safety, governance, and sustained deployment, before considering the research directions toward more general physical intelligence and Physical AGI.

Figure 1.3: perception, world modeling, decision-making, and control connected through physical feedback, data, and hardware
The system-level framework of Physical AI: models and control connected to the physical world through feedback. From Figure 1.3 of the monograph.

The Organizing Idea

Perception, prediction, decision-making, action, and adaptation form a continuous loop through the physical world.

An organizing framework introduced in Chapter 1.
01

Perceive

Estimate physical states from vision, touch, force, proprioception, and other sensing modalities.

02

Model & Predict

Represent physical relationships and anticipate the consequences of possible actions.

03

Decide & Act

Connect goals, planning, and control to feasible interventions in the physical world.

04

Adapt Through Feedback

Verify outcomes, revise behavior, and learn across control, task, and deployment timescales.

Inside the Monograph

Table of Contents

Three parts trace a path from concepts and foundations, through the technical stack, to future directions.
Select a chapter title for its overview, or expand the row to explore its sections.

I2 chapters

Overview and Foundations

Definitions, boundaries, and the foundations in physics, control, cognition, and learning.

01Introduction to Physical AI5 sections p. 11
Chapter overview
  1. 1.1Background: A Paradigm Expansion of AI11
  2. 1.2Physical AI: Definition and Boundaries14
  3. 1.3Core Elements of Physical AI16
  4. 1.4The Role of Physical AI in the Evolution of AI19
  5. 1.5The Application Landscape of Physical AI21
02Technical Foundations4 sections p. 28
Chapter overview
  1. 2.1Mathematical Descriptions of the Physical World28
  2. 2.2Control Theory and Autonomous Systems31
  3. 2.3Embodied Cognition and the Development of Intelligence39
  4. 2.4Learning and Reasoning in the Physical World43
II7 chapters

Methodology

The technical stack: perception, data, world models, manipulation, motor control, agents, and systems.

03Perception and Cognition4 sections p. 49
Chapter overview
  1. 3.1Tasks and Characteristics of Physical Perception49
  2. 3.2Multimodal Perception and Fusion50
  3. 3.33D/4D Spatial Perception56
  4. 3.4High-Level Perception and Cognition58
04Data for Physical Intelligence5 sections p. 64
Chapter overview
  1. 4.1Data Foundations for Physical Intelligence64
  2. 4.2Data Collection and Interaction66
  3. 4.3Physics Engines and Generative Worlds74
  4. 4.4Sim-to-Real Transfer and Real-World Alignment80
  5. 4.5Scaling Laws and the Data Flywheel for Physical AI85
05World Models5 sections p. 94
Chapter overview
  1. 5.1Concepts and Theoretical Frameworks of World Models94
  2. 5.2Representations and Architectures of World Models99
  3. 5.3Training World Models106
  4. 5.4Using and Validating World Models119
  5. 5.5Current Challenges and Future Outlook124
06Foundation Models for Robot Manipulation6 sections p. 128
Chapter overview
  1. 6.1Emergence and Taxonomy128
  2. 6.2Vision-Language-Action Model132
  3. 6.3Pre-Training, Post-Training, and Capability Shaping138
  4. 6.4Multi-Embodiment Modeling and Model Adaptation145
  5. 6.5Model Deployment and Efficiency Optimization151
  6. 6.6Dexterous Manipulation Foundation Models157
07Foundation Models for Whole-Body Motor Control5 sections p. 161
Chapter overview
  1. 7.1Scope, Taxonomy, and Control Interfaces161
  2. 7.2Body Dynamics and State Estimation167
  3. 7.3Command-Conditioned Motor Control172
  4. 7.4Motor Pretraining and Dexterous Whole-Body Control178
  5. 7.5Real-Time Execution, Recovery, and Safety Control186
08Physical Agent4 sections p. 193
Chapter overview
  1. 8.1Definition of Physical Agent193
  2. 8.2Physical Forms of Physical Agent199
  3. 8.3Physical Goal Representation and Task Grounding206
  4. 8.4Multi-Agent Collaboration and Distributed Physical Intelligence212
09Hardware and Systems7 sections p. 219
Chapter overview
  1. 9.1Hardware and Embodiment Taxonomy for Physical Intelligence219
  2. 9.2Sensors, Actuators, and Intelligent Materials223
  3. 9.3Device–Edge–Cloud Collaboration and Real-Time Communication234
  4. 9.4Software Stacks and Development Infrastructure240
  5. 9.5Reliability and Deployment at Scale249
  6. 9.6Case Studies in Dexterous Hand Hardware253
  7. 9.7Case Studies in Humanoid System Integration259
III3 chapters

Future Outlook

General physical intelligence, evaluation and governance, and long-term development strategies.

10General Physical Intelligence6 sections p. 265
Chapter overview
  1. 10.1Core Capabilities of General Physical Intelligence265
  2. 10.2Systematic Generalization in Open-World Settings268
  3. 10.3Human–AI Collaboration and Social Intelligence270
  4. 10.4Multi-Agent Systems and Collective Physical Intelligence271
  5. 10.5Autonomous Evolution273
  6. 10.6Towards the Ultimate Goal of Physical AGI274
11Evaluation, Safety, and Governance for Physical AI5 sections p. 276
Chapter overview
  1. 11.1Physical-World Evaluators276
  2. 11.2Capability Evaluation and Benchmarking280
  3. 11.3Physical Safety and Action Alignment287
  4. 11.4System Safety and Risk Governance292
  5. 11.5Law, Ethics, Standards, and Global Governance297
12Strategy and Civilizational Outlook3 sections p. 301
Chapter overview
  1. 12.1Development Strategies for Physical Intelligence301
  2. 12.2Physical Intelligence and Human Civilization306
  3. 12.3Summary and Outlook312

References p. 313

Chapter pages introduce the forthcoming content. Page numbers refer to the draft manuscript.

Authors

About the Authors

Physical Intelligence Team, Shanghai AI Laboratory

The monograph is authored by the Physical Intelligence Team, Shanghai AI Laboratory. It brings together the concepts, techniques, and challenges of intelligence that interacts with the physical world.

Individual author profiles and acknowledgments will be added when available.

Read & Explore

The Book and Companion Resources

The preprint is coming soon. Companion materials will be shared here as they become available.

Preprint Manuscript

The preprint will be shared here when it is ready for public release.

Publication updates

Code & Examples

A space for companion implementations and examples. Links will be added when available.

Coming soon

Lecture Slides

A space for chapter slides and teaching materials to accompany the monograph.

Coming soon

Errata & Feedback

Corrections and a reader feedback channel will be added here when available.

Coming soon

Publication

Forthcoming

The manuscript is being prepared for public release. Publication information will appear here.

Publisher
To be announced
Publication date
To be announced
Print edition
Coming soon

Citation

Cite the Monograph

This is a provisional citation for the forthcoming monograph.

Physical Intelligence Team, Shanghai AI Laboratory. Physical AI: Concepts, Techniques, and Challenges. Forthcoming, 2026.

BibTeX

@misc{physicalai2026monograph,
  title  = {Physical AI: Concepts, Techniques, and Challenges},
  author = {{Physical Intelligence Team, Shanghai AI Laboratory}},
  year   = {2026},
  note   = {Forthcoming; manuscript not yet publicly released}
}

The citation will be updated when the manuscript and final publication details are available.

Explore Intelligence in the Physical World.

Explore the table of contents while the preprint is being prepared for release.

Preprint coming soonBrowse the contents

Contact

Get in Touch

For questions about the monograph and its companion materials.

InternRobotics

Our contact email will be announced here.

Email to be announced