Multimodal interaction for human-robot teams

Dustin Burke; Nathan Schurr; Jeanine Ayers; Jeff Rousseau; John Fertitta; Alan Carlin; Danielle Dumond

doi:10.1117/12.2016334

17 May 2013 Multimodal interaction for human-robot teams

Dustin Burke, Nathan Schurr, Jeanine Ayers, Jeff Rousseau, John Fertitta, Alan Carlin, Danielle Dumond

Proceedings Volume 8741, Unmanned Systems Technology XV; 87410E (2013) https://doi.org/10.1117/12.2016334
Event: SPIE Defense, Security, and Sensing, 2013, Baltimore, Maryland, United States

Abstract

Unmanned ground vehicles have the potential for supporting small dismounted teams in mapping facilities, maintaining security in cleared buildings, and extending the team’s reconnaissance and persistent surveillance capability. In order for such autonomous systems to integrate with the team, we must move beyond current interaction methods using heads-down teleoperation which require intensive human attention and affect the human operator’s ability to maintain local situational awareness and ensure their own safety. This paper focuses on the design, development and demonstration of a multimodal interaction system that incorporates naturalistic human gestures, voice commands, and a tablet interface. By providing multiple, partially redundant interaction modes, our system degrades gracefully in complex environments and enables the human operator to robustly select the most suitable interaction method given the situational demands. For instance, the human can silently use arm and hand gestures for commanding a team of robots when it is important to maintain stealth. The tablet interface provides an overhead situational map allowing waypoint-based navigation for multiple ground robots in beyond-line-of-sight conditions. Using lightweight, wearable motion sensing hardware either worn comfortably beneath the operator’s clothing or integrated within their uniform, our non-vision-based approach enables an accurate, continuous gesture recognition capability without line-of-sight constraints. To reduce the training necessary to operate the system, we designed the interactions around familiar arm and hand gestures.

Citation Download Citation

Dustin Burke, Nathan Schurr, Jeanine Ayers, Jeff Rousseau, John Fertitta, Alan Carlin, and Danielle Dumond "Multimodal interaction for human-robot teams", Proc. SPIE 8741, Unmanned Systems Technology XV, 87410E (17 May 2013); https://doi.org/10.1117/12.2016334

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available