Restoration of light parameters and forming criteria for the quality estimation of visual perception in mixed reality systems using neural networks

M. Sorokin; V. Antonov; A. A. Laputko; D. D. Zhdanov

doi:10.1117/12.2597164

15 October 2021 Restoration of light parameters and forming criteria for the quality estimation of visual perception in mixed reality systems using neural networks

M. Sorokin, V. Antonov, A. A. Laputko, D. D. Zhdanov

Author Affiliations +

Proceedings Volume 11876, Optical Instrument Science, Technology, and Applications II; 118760I (2021) https://doi.org/10.1117/12.2597164
Event: SPIE Optical Systems Design, 2021, Online Only

Abstract

One of the main problems of mixed reality devices is the physically correct representation of the luminance distribution for virtual objects and their shadows in the real world. In other words, restoring the correct distribution of scene luminance is one of the key parameters that allows solving the problem of correct interaction between the virtual and real worlds. The paper proposes methods for restoring the parameters of light sources. Also, the work is aimed at the creation and formation of criteria for the quality of visual perception, allowing to evaluate the synthesized image of mixed reality and to decide how natural it is from the point of view of the observer. In this article, surveys were used to create datasets using realistic software tools. In work the neural network is trained to recognize those areas of the image that do not fit into the environment and to classify this image as a class that causes visual discomfort. As criteria for the quality of visual perception, it is proposed to use estimates of the mismatch of the parameters of shadows cast by virtual objects and the distribution of luminance over these objects in the images of scenes containing models of "real" and "virtual" objects. The level of misalignment is estimated concerning the true lighting conditions of the real world. In this work, criteria for the quality of visual perception were formed and a neural network was trained, which makes it possible to decide and analyze the quality of a synthesized mixed reality image.

Conference Presentation

Citation Download Citation

M. Sorokin, V. Antonov, A. A. Laputko, and D. D. Zhdanov "Restoration of light parameters and forming criteria for the quality estimation of visual perception in mixed reality systems using neural networks", Proc. SPIE 11876, Optical Instrument Science, Technology, and Applications II, 118760I (15 October 2021); https://doi.org/10.1117/12.2597164

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available