Lessons from aerospace

150 deaths per 10 billion miles

0.2 deaths per 10 billion miles

Artificial Intelligence

Why?

Bias-Variance trade-off

Bayesian Learning

How to handle uncertainty safely?

My research area

My Background

Safety

Safe control while learning

Given:

• Map and localization
• Desired trajectory as a plan
• Unsafe regions

Problem 1:

• Learn uncertainty aware robot system dynamics

Problem 2:

• Follow trajectory avoiding unsafe actions

Problem formulation

• $$\begin{array}{rl}\underset{\mathbf{u}\in \mathcal{U}}{min}& \text{ Cost function }\\ \text{s.t.}& \mathbb{P}(\text{ Safety constraint })\ge 1-ϵ,\end{array}$$

Control Barrier Functions

Deterministic condition for controller

• $$\begin{array}{rl}\underset{{\mathbf{u}}_k\in \mathcal{U}}{min}& \text{ Cost function }\\ \text{s.t.}& \mathbb{P}(\text{ Safety constraint }\mid {\mathbf{x}}_k,{\mathbf{u}}_k)\ge 1-ϵ,\end{array}$$
  $$\begin{array}{rl}\underset{{\mathbf{u}}_k\in \mathcal{U}}{min}& \text{ Quadratic cost function }\\ \text{s.t.}& \mathbb{P}(\text{CBC}({\mathbf{x}}_k,{\mathbf{u}}_k)>\zeta >0\mid {\mathbf{x}}_k,{\mathbf{u}}_k)\ge 1-ϵ,\end{array}$$
• Safe controller (an SOCP): $$\begin{array}{rl}\underset{{\mathbf{u}}_k\in \mathcal{U}}{min}& \text{ Quadratic cost function }\\ \text{s.t.}& \mathbb{E}[\text{CBC}]-\zeta \ge \sqrt{2\mathrm{Var}(\text{CBC})({\mathit{\text{erf}}}^{-1}(2ϵ-1){)}^2}\end{array}$$

Ackerman Drive Simulations

Ackerman Drive Simulations

2022 upgrade: Learning $h(\mathbf{x})$

Better math

Better Simulation: PyBullet

Results

Sample trajectories

Take away

• Bayesian learning enables uncertainty-awareness
• Uncertainty-aware controller can be formulated as SOCP controller

Future work

Other work

Future work

Collaborators

Questions?

• Shengyang Sun, Changyou Chen, and Lawrence Carin. Learning Structured Weight Uncertainty in Bayesian Neural Networks. In International Conference on Artificial Intelligence and Statistics (AISTATS), pages 1283–1292, 2017.
• A. D. Ames, S. Coogan, M. Egerstedt, G. Notomista, K. Sreenath, and P. Tabuada. Control barrier functions: Theory and applications. In 2019 18th European Control Conference (ECC), pages 3420–3431, June 2019. doi: 10.23919/ECC.2019.8796030.
• Mauricio A Alvarez, Lorenzo Rosasco, and Neil D Lawrence. Kernels for vector-valued functions: A review. Foundations and Trends in Machine Learning, 4(3):195–266, 2012.
• Niranjan Srinivas, Andreas Krause, Sham M Kakade, and Matthias Seeger. Gaussian process opti- mization in the bandit setting: No regret and experimental design. arXiv preprint arXiv:0912.3995, 2009.
• Quan Nguyen and Koushil Sreenath. Exponential control barrier functions for enforcing high relative- degree safety-critical constraints. In 2016 American Control Conference (ACC), pages 322–328. IEEE, 2016a.
• Louizos, Christos, and Max Welling. "Structured and efficient variational deep learning with matrix gaussian posteriors." International Conference on Machine Learning. 2016.
• Khojasteh, M. J., Dhiman, V., Franceschetti, M., & Atanasov, N. (2020). Probabilistic safety constraints for learned high relative degree system dynamics. L4DC 2020. available https://arXiv.org/abs/1912.10116.
• Learning from Interventions using Hierarchical Policies for Safe Learning J Bi, V Dhiman, T Xiao, C Xu - AAAI 2020. Available https://arXiv.org/abs/1912.02241
• Learning Navigation Costs from Demonstration in Partially Observable Environments T Wang, V Dhiman, N Atanasov. ICRA 2020. Available https://arXiv.org/abs/2002.11637
• Andrychowicz, Marcin, et al. "Hindsight experience replay." Advances in Neural Information Processing Systems. 2017.
• Mutual localization: Two camera relative 6-dof pose estimation from reciprocal fiducial observation. V Dhiman, J Ryde, JJ Corso. IROS 2013
• Learning Compositional Sparse Models of Bimodal Percepts. S Kumar, V Dhiman, JJ Corso AAAI, 2014
• Voxel planes: Rapid visualization and meshification of point cloud ensembles. J Ryde, V Dhiman, R Platt IROS, 2013
• Modern MAP inference methods for accurate and fast occupancy grid mapping on higher order factor graphs. V Dhiman, A Kundu, F Dellaert, JJ Corso ICRA 2014
• Continuous occlusion models for road scene understanding M Chandraker, V Dhiman. US Patent 9,821,813, 2017
• A continuous occlusion model for road scene understanding V Dhiman, QH Tran, JJ Corso, M Chandraker. CVPR 2016
• A Critical Investigation of DRL for Navigation V Dhiman, S Banerjee, B Griffin, JM Siskind, JJ Corso NeurIPS DRL Workshop, 2017.
• Learning Compositional Sparse Bimodal Models S Kumar, V Dhiman, PA Koch, JJ Corso. PAMI, 2017.
• (Mirowski et al. 2017) Learning to navigate in complex environments. In ICLR 2017.
• Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Research. Matthias Plappert and Marcin Andrychowicz and Alex Ray and Bob McGrew and Bowen Baker and Glenn Powell and Jonas Schneider and Josh Tobin and Maciek Chociej and Peter Welinder and Vikash Kumar and Wojciech Zaremba. ArXiV 2018. 1802.09464
• Kaelbling, Leslie Pack. "Learning to achieve goals." IJCAI. 1993.
• V. Dhiman, S. Banerjee, J. M. Siskind, and J. J. Corso. Learning goal-conditioned value functions with one-step path rewards rather than goal-rewards. In Submitted to ICLR, 2019. Under review.
• Zachariou, Peter et al. “SPEEDING Effects on hazard perception and reaction time.” (2011).
• Mnih, Volodymyr, et al. "Human-level control through deep reinforcement learning." Nature 518.7540 (2015): 529.
• Watkins, Christopher JCH, and Peter Dayan. "Q-learning." Machine learning 8.3-4 (1992): 279-292.
• Pearl, Judea. "Fusion, propagation, and structuring in belief networks." Artificial intelligence 29.3 (1986): 241-288.
• Jojic, Vladimir, Stephen Gould, and Daphne Koller. "Accelerated dual decomposition for MAP inference." ICML. 2010.
• Merali, Rehman S., and Timothy D. Barfoot. "Occupancy grid mapping with Markov chain monte carlo Gibbs sampling." Robotics and Automation (ICRA), 2013 IEEE International Conference on. IEEE, 2013.
• Shayle R Searle and Marvin HJ Gruber.Linear models. John Wiley & Sons, 1971
• Kehan Long, Vikas Dhiman, Melvin Leok, Jorge Cortés, Nikolay Atanasov: Safe Control Synthesis With Uncertain Dynamics and Constraints. IEEE Robotics Autom. Lett. 7(3): 7295-7302 (2022)

Bibliography

• DhKuDaCo ICRA 2014   :   Modern MAP inference methods for accurate and fast occupancy grid mapping on higher order factor graphs. V Dhiman, A Kundu, F Dellaert, JJ Corso ICRA 2014
• DhRyCo IROS 2013   :   Mutual localization: Two camera relative 6-dof pose estimation from reciprocal fiducial observation. V Dhiman, J Ryde, JJ Corso. IROS 2013
• DhBaGrSiCo NeurIPS Wkshp 2017   :   A Critical Investigation of DRL for Navigation V Dhiman, S Banerjee, B Griffin, JM Siskind, JJ Corso NeurIPS DRL Workshop, 2017.
• RyDhPl IROS 2013   :   Voxel planes: Rapid visualization and meshification of point cloud ensembles. J Ryde, V Dhiman, R Platt IROS, 2013
• KuDhCo AAAI 2014   :   Learning Compositional Sparse Models of Bimodal Percepts. S Kumar, V Dhiman, JJ Corso AAAI, 2014
• KuDhKoCo PAMI 2017   :   Learning Compositional Sparse Bimodal Models S Kumar, V Dhiman, PA Koch, JJ Corso. PAMI, 2017.
• DhBaSiCo ArXiV 2019   :   V. Dhiman, S. Banerjee, J. M. Siskind, and J. J. Corso. Learning goal-conditioned value functions with one-step path rewards rather than goal-rewards. In Submitted to ICLR, 2019. Under review.
• KhDhFrAt L4DC 2020   :   Khojasteh, M. J., Dhiman, V., Franceschetti, M., & Atanasov, N. (2020). Probabilistic safety constraints for learned high relative degree system dynamics. L4DC 2020. available https://arXiv.org/abs/1912.10116.
• BiDhXiXu AAAI 2020   :   Learning from Interventions using Hierarchical Policies for Safe Learning J Bi, V Dhiman, T Xiao, C Xu - AAAI 2020. Available https://arXiv.org/abs/1912.02241
• WaDhAt ICRA 2020   :   Learning Navigation Costs from Demonstration in Partially Observable Environments T Wang, V Dhiman, N Atanasov. ICRA 2020. Available https://arXiv.org/abs/2002.11637
• Ames et al (ECC 2019)   :   A. D. Ames, S. Coogan, M. Egerstedt, G. Notomista, K. Sreenath, and P. Tabuada. Control barrier functions: Theory and applications. In 2019 18th European Control Conference (ECC), pages 3420–3431, June 2019. doi: 10.23919/ECC.2019.8796030.
• Alvarez et al (FTML 2012)   :   Mauricio A Alvarez, Lorenzo Rosasco, and Neil D Lawrence. Kernels for vector-valued functions: A review. Foundations and Trends in Machine Learning, 4(3):195–266, 2012.
• Sun et al, AISTATS 2017   :   Shengyang Sun, Changyou Chen, and Lawrence Carin. Learning Structured Weight Uncertainty in Bayesian Neural Networks. In International Conference on Artificial Intelligence and Statistics (AISTATS), pages 1283–1292, 2017.
• Louizos and Welling (ICML 2016)   :   Louizos, Christos, and Max Welling. "Structured and efficient variational deep learning with matrix gaussian posteriors." International Conference on Machine Learning. 2016.
• Srinivas et al. (2009)   :   Niranjan Srinivas, Andreas Krause, Sham M Kakade, and Matthias Seeger. Gaussian process opti- mization in the bandit setting: No regret and experimental design. arXiv preprint arXiv:0912.3995, 2009.
• Nguyen and Sreenath (ACC 2016)   :   Quan Nguyen and Koushil Sreenath. Exponential control barrier functions for enforcing high relative- degree safety-critical constraints. In 2016 American Control Conference (ACC), pages 322–328. IEEE, 2016a.
• Long et al (RAL 2022)   :   Kehan Long, Vikas Dhiman, Melvin Leok, Jorge Cortés, Nikolay Atanasov: Safe Control Synthesis With Uncertain Dynamics and Constraints. IEEE Robotics Autom. Lett. 7(3): 7295-7302 (2022)
• DhTrChCo CVPR 2016   :   A continuous occlusion model for road scene understanding V Dhiman, QH Tran, JJ Corso, M Chandraker. CVPR 2016