Spatial Computing and Intuitive Interaction: Bringing Mixed Reality and Robotics Together

Jeffrey Delmerico, Roi Poranne, Federica Bogo, Helen Oleynikova, Eric Vollenweider, Stelian Coros, Juan Nieto, Marc Pollefeys

Research output: Contribution to journalArticlepeer-review


Spatial computing - the ability of devices to be aware of their surroundings and to represent this digitally - offers novel capabilities in human-robot interaction. In particular, the combination of spatial computing and egocentric sensing on mixed reality (MR) devices enables robots to capture and understand human behaviors and translate them to actions with spatial meaning, which offers exciting possibilities for collaboration between people and machines. This article presents several human-robot systems that utilize these capabilities to enable novel use cases: mission planning for inspection, gesture-based control, and immersive teleoperation. These works demonstrate the power of MR as a tool for human-robot interaction and the potential of spatial computing and MR to drive future developments.

Original languageEnglish
Pages (from-to)45-57
Number of pages13
JournalIEEE Robotics and Automation Magazine
Issue number1
StatePublished - 1 Mar 2022

Bibliographical note

Publisher Copyright:
© 1994-2011 IEEE.

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Computer Science Applications
  • Electrical and Electronic Engineering


Dive into the research topics of 'Spatial Computing and Intuitive Interaction: Bringing Mixed Reality and Robotics Together'. Together they form a unique fingerprint.

Cite this