Machine learning for robot decision making
In recent years, an increasing amount of research1 has focused on enabling robots to perform diverse tasks in complex environments using machine learning (ML)-based techniques. Major technology companies, such as NVIDIA and Tesla, are also advancing efforts to introduce service robots capable of extensive human interaction in daily life. The ability to make decisions based on task requirements and environmental changes has become a crucial factor to fully integrate into human life.
ML-based approaches, such as reinforcement learning and imitation learning, leverage training data to implicitly learn task requirements, as well as the dynamics of both the system and its environment, ultimately deriving adaptive strategies for various scenarios. However, these methods encounter significant challenges, particularly in terms of model training stability and their capacity to learn multimodal behaviors.
Diffusion models in robotic decision making
Deep generative models (DGMs) have demonstrated remarkable success in natural language processing and image generation, highlighting their potential for robot policy learning. Among the family of DGMs, diffusion models3 have been widely adopted in robotics, including trajectory planning4, control6, and grasping generation5, owing to their training stability and capability for long-horizon generation.
The core idea behind diffusion models is an iterative denoising procedure, where the neural network learn how to guide the samples from noise distribution to data distribution. In the forward phase, Gaussian noise is gradually injected to a clean data sample so the data is perturbed, while the neural network is trained to denoise and reconstruct the original sample in the reverse process, as depicted in Figure 1. Similar to image generation, the robot trajectory could also be learned by diffusion models and denoised to a task-performed path during generation.
Fig. 1. Forward process and reverse process in diffusion models. Adapted from: Ho et al. (2020), Denoising Diffusion Probabilistic Models3.
The challenge of diffusion-based decision making
Even though diffusion models have shown potential in robotic decision making due to the strong capability to learn high dimensional behavior, they still face several critical challenges in robotic decision-making. These include, but are not limited to:
- Real-time application challenges: The limited inference speed hinders practical deployment.
- Lack of safety guarantees: Diffusion-based policies do not inherently ensure safety.
- Generalization limitations: These models struggle to handle out-of-distribution scenarios beyond the training data.
Addressing these challenges remains an open research problem2, and its resolution could play a key role in enabling robots to seamlessly integrate into human environments.
Reference
[1] Ravichandar, Harish, et al. "Recent advances in robot learning from demonstration." Annual review of control, robotics, and autonomous systems 3.1 (2020): 297-330. (https://doi.org/10.1146/annurev-control-100819-063206)
[2] Huang, Tzu-Yuan, et al. "SAD-Flower: Flow Matching for Safe, Admissible, and Dynamically Consistent Planning." arXiv preprint arXiv:2511.05355 (2025). (https://doi.org/10.48550/arXiv.2511.05355)
[3] Ho, Jonathan, Ajay Jain, and Pieter Abbeel. "Denoising diffusion probabilistic models." Advances in neural information processing systems 33 (2020): 6840-6851. (https://doi.org/10.48550/arXiv.2006.11239)
[4] Janner, Michael, et al. "Planning with diffusion for flexible behavior synthesis." arXiv preprint arXiv:2205.09991 (2022). (https://doi.org/10.48550/arXiv.2205.09991)
[5] Urain, Julen, et al. "Se (3)-diffusionfields: Learning smooth cost functions for joint grasp and motion optimization through diffusion." 2023 IEEE international conference on robotics and automation (ICRA). IEEE, 2023. (https://doi.org/10.48550/arXiv.2209.03855)
[6] Huang, Tzu-Yuan, et al. "Toward near-globally optimal nonlinear model predictive control via diffusion models." arXiv preprint arXiv:2412.08278 (2024). (https://doi.org/10.48550/arXiv.2412.08278)
About the Author
Tzu-Yuan Huang, M.Sc. is a research associate and doctoral candidate at the Chair of Information-Oriented Control (ITR), Technical University of Munich (TUM), under the supervision of Prof. Sandra Hirche. As a member of relAI, his research focuses on data-driven control and safe, constraint-satisfying generative models for robotic systems, including his recent work on diffusion- and flow-matching-based planning.
