Your eyes give you away: pupillary responses, EEG dynamics, and applications for BCI (Conference Presentation)

Paul Sajda

doi:10.1117/12.2262532

12 June 2017 Your eyes give you away: pupillary responses, EEG dynamics, and applications for BCI (Conference Presentation)

Paul Sajda

Author Affiliations +

Proceedings Volume 10194, Micro- and Nanotechnology Sensors, Systems, and Applications IX; 101941M (2017) https://doi.org/10.1117/12.2262532
Event: SPIE Defense + Security, 2017, Anaheim, CA, United States

Abstract

As we move through an environment, we are constantly making assessments, judgments, and decisions about the things we encounter. Some are acted upon immediately, but many more become mental notes or fleeting impressions -- our implicit "labeling" of the world. In this talk I will describe our work using physiological correlates of this labeling to construct a hybrid brain-computer interface (hBCI) system for efficient navigation of a 3-D environment. Specifically, we record electroencephalographic (EEG), saccadic, and pupillary data from subjects as they move through a small part of a 3-D virtual city under free-viewing conditions. Using machine learning, we integrate the neural and ocular signals evoked by the objects they encounter to infer which ones are of subjective interest. These inferred labels are propagated through a large computer vision graph of objects in the city, using semi-supervised learning to identify other, unseen objects that are visually similar to those that are labelled. Finally, the system plots an efficient route so that subjects visit similar objects of interest. We show that by exploiting the subjects' implicit labeling, the median search precision is increased from 25% to 97%, and the median subject need only travel 40% of the distance to see 84% of the objects of interest. We also find that the neural and ocular signals contribute in a complementary fashion to the classifiers' inference of subjects' implicit labeling. In summary, we show that neural and ocular signals reflecting subjective assessment of objects in a 3-D environment can be used to inform a graph-based learning model of that environment, resulting in an hBCI system that improves navigation and information delivery specific to the user's interests.

Conference Presentation

Citation Download Citation

Paul Sajda "Your eyes give you away: pupillary responses, EEG dynamics, and applications for BCI (Conference Presentation)", Proc. SPIE 10194, Micro- and Nanotechnology Sensors, Systems, and Applications IX, 101941M (12 June 2017); https://doi.org/10.1117/12.2262532

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
PRESENTATION

WATCH
PRESENTATION SAVE TO MY LIBRARY

GET CITATION

KEYWORDS

Navigation systems

Electroencephalography

3D modeling

Brain-machine interfaces

Computer vision technology

Eye models

Machine learning

Show All Keywords

Keywords/Phrases

Search In:

Publication Years