Real-time arbitrary view interpolation and rendering system using ray-space

Norishige Fukushima; Tomohiro Yendo; Toshiaki Fujii; Masayuki Tanimoto

doi:10.1117/12.630760

16 November 2005 Real-time arbitrary view interpolation and rendering system using ray-space

Norishige Fukushima, Tomohiro Yendo, Toshiaki Fujii, Masayuki Tanimoto

Proceedings Volume 6016, Three-Dimensional TV, Video, and Display IV; 60160Q (2005) https://doi.org/10.1117/12.630760
Event: Optics East 2005, 2005, Boston, MA, United States

Abstract

We propose a 3D live video system that generates arbitrary viewpoints in real-time based on the ray-space, one of the image-based rendering. With this system, a remote user can freely change the viewpoint, not only according to the captured camera position, but also can synthesize views where a camera is not physical present using the ray-space interpolation. The basic idea of ray-space rendering is collecting and rearranging the partition of simultaneously captured images according to an arbitrarily specified virtual-view. If hundreds of cameras were arranged in significant density, synthesizing a free viewpoint away from the camera baseline require only camera geometric information. Since we cannot obtain such full information of ray according to plenoptic sampling, arbitrarily view generation necessitate interpolation of slightly missed rays. However, such view interpolation's cost is particularly huge. Therefore, we introduce three novel techniques of view interpolation: first, view centered interpolation framework, second, estimating disparity with smoothing, third, hierarchical searching of correspondences for fast computation. Moreover, we implement the experimental system with those algorithms. This free-view generating system includes sixteen cameras arranged straightforward. All cameras are connected with the consumer computers one by one. Whole the computers connect a server computer via Ethernet categorized star network. This system carries out four processes in real time: capture images, correct position of cameras with projective transformation, interpolate images on baseline, rendering arbitrary viewpoint. The experimental result shows that this system is rendering arbitrary viewpoint at 12fps (frames per second) set image resolution set to "320x240". We succeeded in synthesizing highly photo-realistic images.

Citation Download Citation

Norishige Fukushima, Tomohiro Yendo, Toshiaki Fujii, and Masayuki Tanimoto "Real-time arbitrary view interpolation and rendering system using ray-space", Proc. SPIE 6016, Three-Dimensional TV, Video, and Display IV, 60160Q (16 November 2005); https://doi.org/10.1117/12.630760

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available