Defect detection and removal for depth map quality enhancement in manufacturing with deep learning

N. Gapon; V. Voronin; R. Sizyakin; M. Zhdanova; E. Semenishchev; A. Zelensky

doi:10.1117/12.2666560

15 June 2023 Defect detection and removal for depth map quality enhancement in manufacturing with deep learning

N. Gapon, V. Voronin, R. Sizyakin, M. Zhdanova, E. Semenishchev, A. Zelensky

Proceedings Volume 12524, Dimensional Optical Metrology and Inspection for Practical Applications XII ; 125240L (2023) https://doi.org/10.1117/12.2666560
Event: SPIE Defense + Commercial Sensing, 2023, Orlando, Florida, United States

Conference Poster

Abstract

A system for determination the distance from the robot to the scene is useful for object tracking, and 3-D reconstruction may be desired for many manufacturing and robotic tasks. While the robot is processing materials, such as welding parts, milling, drilling, fragments of materials fall on the camera installed on the robot, introducing unnecessary information when building a depth map, as well as the emergence of new lost areas, which leads to incorrect determination of the size of objects. There is a problem comprising a decrease in the accuracy of planning the movement trajectory caused by incorrect sections on the depth map because of incorrect distance determination of objects. We present a two-stage approach combining defect detection and depth reconstruction algorithms. The first step is image defects detection based on convolutional auto-encoder (U-Net) and deep feature fusion network (DFFN-Net). The second step is a depth map reconstruction with the exemplar-based and the anisotropic gradient concepts. The proposed modified block fusion algorithm uses a local image descriptor obtained by an automatic encoder for image reconstruction, which extracts image features and depth maps using a decoding network. Our technique outperforms the state-of-the-art methods quantitatively in reconstruction accuracy on RGB-D benchmark for evaluating manufacturing vision systems.

Citation Download Citation

N. Gapon, V. Voronin, R. Sizyakin, M. Zhdanova, E. Semenishchev, and A. Zelensky "Defect detection and removal for depth map quality enhancement in manufacturing with deep learning", Proc. SPIE 12524, Dimensional Optical Metrology and Inspection for Practical Applications XII , 125240L (15 June 2023); https://doi.org/10.1117/12.2666560

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available