High accuracy hole filling for Kinect depth maps

Jianxin Wang; Ping An; Yifan Zuo; Zhixiang You; Zhaoyang Zhang

doi:10.1117/12.2071437

31 October 2014 High accuracy hole filling for Kinect depth maps

Jianxin Wang, Ping An, Yifan Zuo, Zhixiang You, Zhaoyang Zhang

Proceedings Volume 9273, Optoelectronic Imaging and Multimedia Technology III; 92732L (2014) https://doi.org/10.1117/12.2071437
Event: SPIE/COS Photonics Asia, 2014, Beijing, China

Abstract

Hole filling of depth maps is a core technology of the Kinect based visual system. In this paper, we propose a hole filling algorithm for Kinect depth maps based on separately repairing of the foreground and background. There are two-part processing in the proposed algorithm. Firstly, a fast pre-processing to the Kinect depth map holes is performed. In this part, we fill the background holes of Kinect depth maps with the deepest depth image which is constructed by combining the spatio-temporal information of the pixels in Kinect depth map with the corresponding color information in the Kinect color image. The second step is the enhancement for the pre-processing depth maps. We propose a depth enhancement algorithm based on the joint information of geometry and color. Since the geometry information is more robust than the color, we correct the depth by affine transform in prior to utilizing the color cues. Then we determine the filter parameters adaptively based on the local features of the color image which solves the texture copy problem and protects the fine structures. Since L1 norm optimization is more robust to data outliers than L2 norm optimization, we force the filtered value to be the solution for L1 norm optimization. Experimental results show that the proposed algorithm can protect the intact foreground depth, improve the accuracy of depth at object edges, and eliminate the flashing phenomenon of depth at objects edges. In addition, the proposed algorithm can effectively fill the big depth map holes generated by optical reflection.

Citation Download Citation

Jianxin Wang, Ping An, Yifan Zuo, Zhixiang You, and Zhaoyang Zhang "High accuracy hole filling for Kinect depth maps", Proc. SPIE 9273, Optoelectronic Imaging and Multimedia Technology III, 92732L (31 October 2014); https://doi.org/10.1117/12.2071437

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available