An integrated perception pipeline for robot mission execution in unstructured environments

Priya Narayanan; Bryanna Yeh; Emma Holmes; Scott Martucci; Karl Schmeckpeper; Christoph Mertz; Philip Osteen; Maggie Wigness

doi:10.1117/12.2560699

21 April 2020 An integrated perception pipeline for robot mission execution in unstructured environments

Priya Narayanan, Bryanna Yeh, Emma Holmes, Scott Martucci, Karl Schmeckpeper, Christoph Mertz, Philip Osteen, Maggie Wigness

Author Affiliations +

Proceedings Volume 11413, Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications II; 1141318 (2020) https://doi.org/10.1117/12.2560699
Event: SPIE Defense + Commercial Sensing, 2020, Online Only

Abstract

Visual perception has become core technology in autonomous robotics to identify and localize objects of interest to ensure successful and safe task execution. As part of the recently concluded Robotics Collaborative Technology Alliance (RCTA) program, a collaborative research effort among government, academic, and industry partners, a vision acquisition and processing pipeline was developed and demonstrated to support manned-unmanned team ing for Army relevant applications. The perception pipeline provided accurate and cohesive situational awareness to support autonomous robot capabilities for maneuver in dynamic and unstructured environments, collaborative human-robot mission planning and execution, and mobile manipulation. Development of the pipeline involved a) collecting domain specific data, b) curating ground truth annotations, e.g., bounding boxes, keypoints, c) retraining deep networks to obtain updated object detection and pose estimation models, and d) deploying and testing the trained models on ground robots. We discuss the process of delivering this perception pipeline under limited time and resource constraints due to lack of a priori knowledge of the operational environment. We focus on experiments conducted to optimize the models despite using data that was noisy and exhibited sparse examples for some object classes. Additionally, we discuss our augmentation techniques used to enhance the data set given skewed class distributions. These efforts highlight some initial work that directly relates to learning and updating visual perception systems quickly in the field under sudden environment or mission changes.

Conference Presentation

Citation Download Citation

Priya Narayanan, Bryanna Yeh, Emma Holmes, Scott Martucci, Karl Schmeckpeper, Christoph Mertz, Philip Osteen, and Maggie Wigness "An integrated perception pipeline for robot mission execution in unstructured environments", Proc. SPIE 11413, Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications II, 1141318 (21 April 2020); https://doi.org/10.1117/12.2560699

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available