As electronic documents are becoming more and more popular, variety of objects, including 2D and 3D objects such as articles, books and 3D-shapes, can be easily contained in a document. Conventional systems cannot capture these objects as a 3D form. We developed a new image capturing system with 3D information for deskwork. It is an assemblage of a normal digital camera and its docking station designed for easy operation. The docking station swings the attached camera tilting step by step. Within a few steps, it covers the whole object automatically. Then each frame is combined together into one complete image with full resolution. In order to get precise 3D structure, stripe patterns are projected by a modified flash light attached on the camera. The resolution rises by means of the swing to be slipped the patterns on the object. Using the obtained 3D data we can reconstruct a correct image from a splay surface image such as a book. Experimental Results of image mosaicking and 3D reconstruction shows that the system is practical.

As electronic documents are becoming more and more popular, variety of objects, including 2D and 3D objects such as articles, books and 3D-shapes, can be easily contained in a document. Conventional systems cannot capture these objects as a 3D form. We developed a new image capturing system with 3D information for deskwork. It is an assemblage of a normal digital camera and its docking station designed for easy operation. The docking station swings the attached camera tilting step by step. Within a few steps, it covers the whole object automatically. Then each frame is combined together into one complete image with full resolution. In order to get precise 3D structure, stripe patterns are projected by a modified flash light attached on the camera. The resolution rises by means of the swing to be slipped the patterns on the object. Using the obtained 3D data we can reconstruct a correct image from a splay surface image such as a book. Experimental Results of image mosaicing and 3D reconstruction shows that the system is practical.

We propose a novel range finding technique for measuring the 3D shape of objects. Experiments were carried out to verify the principle of this technique, and effectiveness of the technique was demonstrated through the experiments. The new range finding technique, which is based on a spatial encoding method, uses a single light pattern encoded with multiple values and enables the acquisition of range information of an object in a single shot. The conventional spatial encoding method uses a projector for projecting a light pattern onto an object and a CCD camera for measuring the 3D shape of the object. In this new technique, we use an additional CCD camera to cancel the influence of the texture of the object. The additional CCD camera is positioned so that the principal point and optical axis of the camera's lens become the same as those of the projector's lens. The light pattern projected onto the object is encoded again using image data captured by the additional CCD camera. Range image data acquired using this technique is highly accurate. We call this technique the Spatial Re-encoding method or SR method. We made an experimental model and captured range data of an object form different viewpoints to evaluate the effectiveness of the SR method. The whole 3D shape of the object was successfully built using those range data. This paper describes the new range finding technique called the SR method.

A novel 3D modeling system in which a target object is easily captured and modeled by using a hand-held camera with several reference panels is presented in this paper. The reference panels are designed to be able to obtain the camera position and discriminate between each other. A conventional 3D modeling system using a reference panel has several restrictions regarding the target object, specifically the size and its location. Our system uses multiple reference panels, which are set around the target object to remove these restrictions. The main features of this system are as follows: 1) The whole shape and photo-realistic textures of the target object can be digitized based on several still images or a movie captured by using a hand-held camera; as well as each location of the camera that can be calculated using the reference panels. 2) Our system can be provided as a software product only. That means there are no special requirements for hardware; even the reference panels , because they can be printed from image files or software. 3) This system can be applied to digitize a larger object. In the experiments, we developed and used an interactive region selection tool to detect the silhouette on each image instead of using the chroma -keying method. We have tested our system with a toy object. The calculation time is about 10 minutes (except for the capturing the images and extracting the silhouette by using our tool) on a personal computer with a Pentium-III processor (600MHz) and 320MB memory. However, it depends on how complex the images are and how many images you use. Our future plan is to evaluate the system with various kind of objects, specifically, large ones in outdoor environments.

Creation of accurate surface models of abdominal organs is essential for many developing technologies in medicine and surgery. One application we are working towards is augmented reality (AR) visualization for laparoscopic surgery. Our current system meets some, but not all, of the requirements. We use two custom built laparoscopes, a custom built miniature projector, a standard camera, and a standard video capture and processing card to implement a laparoscopic structured light range acquisition system. We will briefly show the custom hardware but will emphasize the structured light depth extraction techniques used for the unique properties of surfaces inside the body, particularly dealing with specular reflections. In early experiments, we studied the effectiveness of our algorithm in highly specular environments by creating range images acquired from fresh animal organs. These experiments used a large projector, open abdomens, and offline image processing. We report the results of experiments using our miniature projector, and on line processing.

Traditional visual communication systems convey only two-dimensional (2D) fixed field-of-view video information. The viewer is presented with a series of flat, non-stereoscopic images, which fail to provide a realistic sense of depth. Furthermore, traditional video is restricted to only a small part of the scene, based on the director's discretion and the user is not allowed to 'look around' in an environment. The major goal of our research is to develop new techniques for creating stereo panoramic video sequences. A stereo panoramic video sequence should be able to provide the viewer with stereo vision at any direction (complete 360 degrees field-of-view) at video rates. In this paper we explore the possibility of creating stereo panoramic video using a multi-camera approach, thus creating a high-resolution output. We present a setup that is an extension of a previously known approach, developed for the generation of still stereo panoramas, and demonstrate that it is capable of achieving our goal. We further explore the practical limitations involved in a practical implementation of the setup, namely the limited number of cameras and the non-zero physical size of real cameras. The relevant trade-offs are identified and studied. Experimental results are presented to support our conclusions.

We present a knowledge-based system for automatic 3D building reconstruction from aerial images. Our approach relies on combining pairs of stereo images with 2D GIS map and domain knowledge. Since most buildings can be described as aggregation of simple building types, the domain knowledge is represented in a building library containing building primitives (flat, gable, and hip roof building). The approach of modeling buildings using a set of basic building models suggests the usage of Constructive Solid Geometry representation for building description. The building reconstruction process is formulated as a hypothesis generation and verification scheme. It starts with the partitioning of a building in simple building parts based on the ground plan defined in the map. For each building partition different building hypotheses are generated corresponding to the building primitives defined in the building library. The evaluation of the generated building models is based on the formulation of the mutual information between the model and the images. The CSG tree representing a building is given by the best fit of the building models corresponding to the building partitions. We used this method to reconstruct buildings in suburban and urban scenes. The method worked well even in difficult conditions (noise, shadow).

Toward future 3D image communication, we have started studying the Multimedia Ambiance Communication, a kind of shared-space communication, and adopted an approach to design the 3D-image space using actual images of outdoor scenery, by introducing the concept of the three-layer model of long-, mid- and short-range views. The long- and mid-range views do not require precise representation of their 3D structure, and hence we employ the setting representation like stage settings to approximate their 3D structure according to the slanting-plane-model. We deal with an approach to produce the consistent setting representation for describing long- and mid-range views from range and texture data measured with a laser scanner and a digital camera located at multiple viewpoints. The production of such a representation requires the development of several techniques: nonlinear smoothing of raw range data, plane segmentation of range data, registration of multi-viewpoint range data, integration of multi-viewpoint setting representations and texture mapping onto each setting plane. In this paper, we concentrate on the plane segmentation and the multi-viewpoint data registration. Our plane segmentation method is based on the concept of the region competition, and can precisely extract fitting planes from the range data. Our registration method uses the equations of the segmented planes corresponding between two different viewpoints to determine the 3D Euclidean transformation between them. A unifying consistent setting representation can be constructed by integrating multiple setting representations for multiple viewpoints.

An approach for corresponding points matching based on multiresolution wavelet transform and phase matching is presented in this paper. In binocular vision, disparity is an important cue to reconstruct the 3D structure of the scene from two or more images. Disparity for a stereo pair can be calculated by solving the stereo correspondence problem in image matching. For phase matching, the difference between the complex phases at corresponding points is used to find binocular disparity. In this approach, we construct a complex-valued wavelet kernel, which satisfied the requirement to the filter used in phase matching, through the Hilbert transform. The multiresolution analysis is combined with this kernel to get multiresolution phases of local spatial frequency. Then the phase matching is done based on these multiresolution phases. According to the relationship between image features and phase congruency, and also the relationship between phase congruency and local energy, we modified the iteration approaches in phase matching so as to give high confidence on image features.

The goal of computing realistic 3-D models from image sequences is still a challenging problem. In recent years the demand for realistic reconstruction and modeling of objects and human bodies is increasing both for animation and medical applications. In this paper a system for the reconstruction of 3-D models of articulated objects, like human bodies, from uncalibrated images is presented. The scene is seen from different viewpoints and no pre-set knowledge is considered. To extract precise 3-D information from imagery, a calibration procedure must be performed. Therefore, first a camera calibration with Direct Linear Transformation (DLT) is done assuming few control points on the subject. The initial approximations of the interior and exterior orientation computed with DLT are then improved in a general photogrammetric bundle adjustment with self-calibration. Additionally a stereo matching process based on least squares matching extracts a dense set of image correspondences from the sequence. Finally a 3-D point cloud is computed by forward intersection using the calibration data and the matched points. The resulting 3-D model of human body is presented.

While creating a true 3D representation of object measured by any structure light projection system, the numerous clouds of points gathered fro various directions have to be merged. The techniques, which perform this task efficiently and correctly, are under constant investigation. In the paper we propose two merging methodologies: experimental - based on the knowledge about geometrical relations between the measurement system and measured object and numerical - based on the local similarity of the merged clouds. The first solution is connected with pre-calibration of the mechanical set-up in the measurement system coordinate space and/or the specific requirements for the accuracy and repeatability of the mechanical manipulators. The numerical solution is based on the assumption that at least one overlapping area with the characteristic points exist in the sequential pairs of the merged clouds. These points are used for calculation of the transformation matrix between the clouds. The analysis of influence of the applied method on the final virtual object representation is given and applied. We present several examples of implementation of these methodologies.

Object pose estimation is an important application in 3D recognition. A 3D object pose estimating method is developed for an automated manufacturing assembly application. The target parts are extracted from the original range images using the traditional edge detection and segmentation methods. The center position is then computed through the circle Hough transform algorithm. For the 3D orientation estimation, a 3D geometrical feature descriptor, Angle Distance Map (ADM), is proposed to describe the 3D local surface feature. A triangular mesh model of 3D object is used for reducing the computational complexity. The principal component analysis (PCA) method is applied on the ADM descriptions for efficient comparison. The orientation information is computed according to the extracted 3D feature points. The proposed method is tested in an application for flexible robot assembly. The experimental results show that accurate 3D pose estimation can be obtained.

The 3D imaging and display system based-on the integral photography(IP) method has been actively researched. But, It is a little bit difficult to implement the IP-based 3D display system to have the satisfiable 3D image resolution by using the conventional optical devices. Besides, it has some problems of limited viewing angle and observation distance. Accordingly, to alleviate these kind of problems, some sort of the design simulator for implementing the practical IP-based 3D display system is needed, in which the system parameters of the conventional optical devices can be optimized for given environments. Therefore, in this paper, the design simulator for the IP-based 3D display system is implemented by using the various display and design parameters through Visual C ++ program in GUI(graphic user interface) environment. From the analysis of relationship between the design and display parameters by using the implemented design simulator, the optimized design parameters for the cases of the various conditions can be derived. From some computer simulation, it is suggested that the optimum design parameters for any kind of given situations can be derived by using the IP-based 3D display design simulator.

Traditionally, medical geneticists have employed visual inspection (anthroposcopy) to clinically evaluate dysmorphology. In the last 20 years, there has been an increasing trend towards quantitative assessment to render diagnosis of anomalies more objective and reliable. These methods have focused on direct anthropometry, using a combination of classical physical anthropology tools and new instruments tailor-made to describe craniofacial morphometry. These methods are painstaking and require that the patient remain still for extended periods of time. Most recently, semiautomated techniques (e.g., structured light scanning) have been developed to capture the geometry of the face in a matter of seconds. In this paper, we establish that direct anthropometry and structured light scanning yield reliable measurements, with remarkably high levels of inter-rater and intra-rater reliability, as well as validity (contrasting the two methods).

With the growth in computing, storage, and networking infrastructure, it is becoming feasible for multimedia professionals, such as graphic designers in commercial, manufacturing, scientific, and entertainment areas, to work with 3D models of objects of their interest. As a result, the volume of 3D models available in digital form is rapidly growing. However, we lack a digital library system for these models, where they can be stored, searched and retrieved efficiently. As the size of data representing a 3D model is usually large, it presents a number of challenges in building an efficient digital library system. In this paper, we propose a digital library framework that is designed to provide storage services for 3D models, search and discovery services, and progressive retrieval services. The key to the digital library framework is a representation of a 3D model based on 'surface signatures'. This representation scheme captures the shape information of any free-form surface and encodes it into a 2D image corresponding to a certain point on the surface. The original object can be reconstructed from the intersection of the inverse mapping of few signatures with accuracy that depends on the location of the selected points and the number of signatures used in representing the object. This compressed representation allows for efficient storage and is amenable for progressive retrieval. Also, the 3-D objects can be checked for similarities in the compressed domain.

Modeling and measurement of the human face have been increasing by importance for various purposes. Laser scanning, coded light range digitizers, image-based approaches and digital stereo photogrammetry are the used methods currently employed in medical applications, computer animation, video surveillance, teleconferencing and virtual reality to produce three dimensional computer models of the human face. Depending on the application, different are the requirements. Ours are primarily high accuracy of the measurement and automation in the process. The method presented in this paper is based on multi-image photogrammetry. The equipment, the method and results achieved with this technique are here depicted. The process is composed of five steps: acquisition of multi-images, calibration of the system, establishment of corresponding points in the images, computation of their 3-D coordinates and generation of a surface model. The images captured by five CCD cameras arranged in front of the subject are digitized by a frame grabber. The complete system is calibrated using a reference object with coded target points, which can be measured fully automatically. To facilitate the establishment of correspondences in the images, texture in the form of random patterns can be projected from two directions onto the face. The multi-image matching process, based on a geometrical constrained least squares matching algorithm, produces a dense set of corresponding points in the five images. Neighborhood filters are then applied on the matching results to remove the errors. After filtering the data, the three-dimensional coordinates of the matched points are computed by forward intersection using the results of the calibration process; the achieved mean accuracy is about 0.2 mm in the sagittal direction and about 0.1 mm in the lateral direction. The last step of data processing is the generation of a surface model from the point cloud and the application of smooth filters. Moreover, a color texture image can be draped over the model to achieve a photorealistic visualization. The advantage of the presented method over laser scanning and coded light range digitizers is the acquisition of the source data in a fraction of a second, allowing the measurement of human faces with higher accuracy and the possibility to measure dynamic events like the speech of a person.

The objects captured with three-dimensional scanners are, by themselves, of limited value. The real power of 3D scanning emerges as applications derive useful information from the point clouds. Extracting measurements from 3D human body scans is an important capability for those interested in clothing and equipment design, human factors evaluation, and web commerce, among other applications. In order to be practical, measurement extraction functions must be fast, accurate, and reliable. Automation is critical for processing the large numbers of scans envisioned by most developers. In this paper we report two functions for identifying feducial points (landmarks) on the human face. First, we used a template-matching approach where a predefined template of 34 face landmarks is matched to a head scan using a small subset of the template landmarks. Once the template is in place, interrogating local surface geometry refines landmark location. This approach allows us to locate a large number of landmarks quickly, and, more importantly, it allows us to place important but hard to locate landmarks. In our second approach, we used image-processing methods to locate a small blue dot that has been positioned on the face prior to scanning.

Automatic extraction of anatomic landmarks from three-dimensional (3-D) head scan data is a typical, also challenging application of 3-D image analysis. This paper explored approaches to automatically identify landmarks based on their geometric appearance in a 3-D data set. We investigated the geometric features of most important landmarks of the head/face, especially invariant surface characteristics such as mean and Gaussian curvature, and other external characteristics as well. Based on the analysis of these features, we define a number of methods and operators to locate each extractable landmark from 3-D scan data. Starting from nose, the process to locate face landmarks can be conducted in a structural way and we reduced the image analysis of each landmark to a local area. Ideally, the characteristic map derived from a 3-D digital image should deliver a meaningful image for analysis. However, due to noise and void in the data set, it is not unusual the characteristic map has to be post-processed or re-computed. A number of experiments is conducted to find the suitable computational technique and additional steps are taken to obtain a satisfied characteristic map.

Traditionally, medical geneticists have employed visual inspection (anthroposcopy) to clinically evaluate dysmorphology. In the last 20 years, there has been an increasing trend towards quantitative assessment to render diagnosis of anomalies more objective and reliable. These methods have focused on direct anthropometry, using a combination of classical physical anthropology tools and new instruments tailor-made to describe craniofacial morphometry. These methods are painstaking and require that the patient remain still for extended periods of time. Most recently, semiautomated techniques (e.g., structured light scanning) have been developed to capture the geometry of the face in a matter of seconds. In this paper, we establish that direct anthropometry and structured light scanning yield reliable measurements, with remarkably high levels of inter-rater and intra-rater reliability, as well as validity (contrasting the two methods).

The problem of human face shape reconstruction basing on information about human skull is very important for many applications beginning with archeology and finishing with crime expertise. Recent progress in development of 3D scanning system provides the means and techniques for generation of metric 3D models for a skull thus creating a background for development a method for automated unknown human face virtual reconstruction. The paper presents the method, hardware, and original software for reconstruction of human face basing on metric skull 3D model. Skull 3D model is obtained using PC-based digital photogrammetric system for automated skull 3D reconstruction. It allows generating a whole textured metric skull 3D model by one step using images from three CCD cameras while turntable rotating. The proposed method for unknown face 3D reconstruction is based on the suppositions that every skull can be described by a set of reference points corresponding to a reference point set on a human face and that an arbitrary face 3D model can be transformed into reconstructed face by some kind of deformation with restriction on given tissue depth at the reference points. The description of the developed method and some results of the proposed technique applying are presented.

The advantage of optical 3D measurement systems like triangulation based fringe projection systems compared with tactile working systems is their contactless and fast 3D surface data acquisition. The results in the form of point clouds can be processed regarding a mathematical surface description for CAD data generation. In many cases the optical 3D surface data acquisition of complex workpieces cannot be achieved in one measuring process so that it is necessary to merge different measuring fields. This approach is essential when the object is larger than the measuring range or the object cannot be measured in one measuring process in consequence of reflections or shadings. According to these problems a fringe projection system was integrated in a coordinate measuring machine with 5 axes (3 linear axes and 2 rotation axes). By using these axes, every region of the workpiece can be reached with the optical 3D sensor for digitalization. To get the complete point cloud of the object surface the different measuring fields must be transformed in one coordinate system by using the relative axes positions of the coordinate measuring machine. Results concerning Reverse Engineering applications and considerations of the measurement uncertainty of the scanning fringe projection system are presented.

This paper presents a vision system whose purpose is to detect topographies of high reflective, metallic surfaces of minted tokens. We call this technique 'Selective Stereo Gradient Method' (SSGM). The objective is to decide whether the token belongs to a reference class or not. The most important property of the SSGM is that the classification can not be deceived by a photographic image and hence yields high fraud protection. To achieve this a 3 sector 120# LED illumination is used for generating three images under different illumination directions. The comparison between these three sequentially taken images leads to a discrimination between a real object with 3 D topography and a photographic image. The experimental setup and special illumination conditions are described. Rotation and translation invariance of the recognition and classification process are implemented. This is achieved by image transformation into a suitable coordinate system. A specimen will be identified to belong to the class of interest if, in a subsequent template matching step, selected patterns taken from the class reference object, can be successfully identified. If a first pattern is found additional patterns will be searched for. The classification statistics results will be reported for metallic tokens.

We are developing a multiple-angle 3D-video system that will allow an audience to enjoy a concert in real time at a distant location by simultaneously shooting multiple stereoscopic images from different angles and transmitting them through a high-speed network. At the receiving side, a decoder restores the original image signals from the received data, and different video images are shown on multiple stereoscopic displays. As a result, the audience can enjoy multiple images of a concert at the same time. The purpose of this system is to give a remote audience the opportunity to enjoy a live concert as though they were at the concert site. This paper examines the technical requirements for the camera arrangement and the transmission rate needed to transmit images of a musical performance with this system. We have built an experimental system with four stereo cameras, and have experimentally transmitted musical performance images to test the system performance. In this paper, we also propose a method for transmitting multiple camera images more efficiently.