Using arm and hand gestures to command robots during stealth operations

Adrian Stoica; Chris Assad; Michael Wolf; Ki Sung You; Marco Pavone; Terry Huntsberger; Yumi Iwashita

doi:10.1117/12.923690

10 May 2012 Using arm and hand gestures to command robots during stealth operations

Adrian Stoica, Chris Assad, Michael Wolf, Ki Sung You, Marco Pavone, Terry Huntsberger, Yumi Iwashita

Proceedings Volume 8407, Multisensor, Multisource Information Fusion: Architectures, Algorithms, and Applications 2012; 84070G (2012) https://doi.org/10.1117/12.923690
Event: SPIE Defense, Security, and Sensing, 2012, Baltimore, Maryland, United States

Abstract

Command of support robots by the warfighter requires intuitive interfaces to quickly communicate high degree-offreedom (DOF) information while leaving the hands unencumbered. Stealth operations rule out voice commands and vision-based gesture interpretation techniques, as they often entail silent operations at night or in other low visibility conditions. Targeted at using bio-signal inputs to set navigation and manipulation goals for the robot (say, simply by pointing), we developed a system based on an electromyography (EMG) "BioSleeve", a high density sensor array for robust, practical signal collection from forearm muscles. The EMG sensor array data is fused with inertial measurement unit (IMU) data. This paper describes the BioSleeve system and presents initial results of decoding robot commands from the EMG and IMU data using a BioSleeve prototype with up to sixteen bipolar surface EMG sensors. The BioSleeve is demonstrated on the recognition of static hand positions (e.g. palm facing front, fingers upwards) and on dynamic gestures (e.g. hand wave). In preliminary experiments, over 90% correct recognition was achieved on five static and nine dynamic gestures. We use the BioSleeve to control a team of five LANdroid robots in individual and group/squad behaviors. We define a gesture composition mechanism that allows the specification of complex robot behaviors with only a small vocabulary of gestures/commands, and we illustrate it with a set of complex orders.

Citation Download Citation

Adrian Stoica, Chris Assad, Michael Wolf, Ki Sung You, Marco Pavone, Terry Huntsberger, and Yumi Iwashita "Using arm and hand gestures to command robots during stealth operations", Proc. SPIE 8407, Multisensor, Multisource Information Fusion: Architectures, Algorithms, and Applications 2012, 84070G (10 May 2012); https://doi.org/10.1117/12.923690

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available