n industrial application of structured light systems (SLS), besides the measurement accuracy, the robustness in long-time working situations is also significant. This requirement is hard to be satisfied due to the drifting of system parameters caused by factors such as the generated heat of the devices and the temperature variation of the environment. To ameliorate this drawback, this paper proposes a re-calibration method of SLS parameters in temperature-varying situations based on utilizing a series of circle marks as control-points. In this method, after initial calibration, a measurement of the whole workspace including these control-points is carried out as soon as possible, in which a series of two-directions phase images and an extra white image are captured. Through phase calculation and reconstruction, the precise poses of the control-points can be obtained and saved. Then, cost function is constructed by utilizing these control-points based on bundle adjustment and the system parameters are optimized. Compared with previous re-calibration methods, since extra constraint of relative pose between the camera and the projector is provided by registered circle marks, a target object with ground truth used to calculate the scale information in previous methods is not necessary anymore, which improves the compatibility of SLS application. Experiments are also carried out to verify the performance of this method.
KEYWORDS: Metals, Visibility, Monte Carlo methods, 3D modeling, Mathematical optimization, Quality measurement, Point clouds, Inspection, 3D metrology, Technology
The automatic 3D measurement technology based on surface structured light has been widely used in the production of metal sheet parts. Before the measurement, it is necessary to manually plan the robot's measurement viewpoint and path, and each viewpoint needs to take into account the measurement quality of the shape surface and holes at the same time. The planning is very difficult, and manual planning results in long offline debugging time and low measurement efficiency. To address this problem, this paper proposes a view planning method that focuses on holes of metal sheet part. Firstly, the relationship between view angle and hole reconstruction quality is constructed to analyze the influence of view angle on hole measurement. Then, candidate viewpoints, which meet the visibility and hole reconstruction quality requirement, are generated. Finally, the Markov decision process is used to model the coverage planning problem, and the Monte Carlo tree search method is used to generate the shortest viewpoint sequence which meets the coverage ratio requirements. The experimental results show that the method proposed in this paper can realize the complete measurement of complex surface features such as holes, and plan an efficient robotic measurement path.
Limited by the imaging dynamic range of the camera, the phenomenon of over-exposure and over-dark often occurs in the 3D measurement of strong reflective sheet metal parts, resulting in incomplete measurement result. One of existing methods such as multiple exposure can measure most of the visible area under a single viewpoint, but the visible area with too small or too large incidence angle still cannot be measured. To solve this problem, in this paper, a method of viewpoint planning for sheet metal parts with strong reflection is proposed. The method introduces the surface reflection model of reflective sheet metal parts into viewpoint planning to achieve the synchronous optimum of measurement efficiency and data integrity. Firstly, according to the measurable region of the surface structured light 3D measurement system and CAD model, the candidate viewpoint set is randomly generated in the sampling space, and the visibility matrix is constructed by analyzing whether each candidate viewpoint is visible to each patch of the model. Then, the surface reflection model of sheet metal parts with strong reflection is constructed, and the reflection coefficient of the visible patches under each viewpoint is calculated according to the reflection model. Based on this, the measurability of the visible patches of the viewpoint under multiple exposures is calculated, and the visibility matrix is updated. Lastly, through the viewpoint quality evaluation function constructed based on the data coverage increment and multiple-exposure time, the viewpoint with the highest quality is selected heuristically until the coverage requirement is met. Experiments show that the algorithm can improve the measurement efficiency and ensure the integrity of the measurement data.
Holes are common and important feature information in sheet metal parts and play a decisive role in product production and assembly. The 3D contour reconstruction method based on passive binocular vision has been widely used in hole measurement due to its high accuracy and robustness. It can obtain dimensional data such as diameter and center of round holes, but there are still problems in measurement accuracy and integrity, and it is difficult to accurately measure special-shaped holes. Aiming at these issues, a fringe projection-based 3D contour measurement method for holes on sheet metal parts is proposed. In this method, two constant movements are performed on the extracted contour curve to the surrounding area, and the 3D contour points can be reconstructed according to the phase matching. Therefore, the real 3D contour is reconstructed inversely according to the corresponding points of the previous two reconstructed contours. With this method, contours only need to be extracted on one image, so it can reduce the effect of inconsistency in contour extraction. In addition, the accuracy and integrity of the 3D contour can be guaranteed by using high-quality phase matching of the sheet metal area. The experimental results show that this method can achieve accurate and complete 3D contour reconstruction for sheet metal holes of various shapes, and the accuracy can reach 0.08mm.
Process signatures of the fusion area directly determine the quality of the part in Laser Powder Bed Fusion (LPBF). The geometric contour of the fusion area is one of the most important indicators of manufacturing quality. The accurate detection of the contour has been generating considerable interest. However, due to the complex operating condition in LPBF, the 2D images of the fusion area suffer from shortcomings such as poor contrast, high noise, and vary illumination. This makes the location of the contour extremely difficult for traditional detection methods. In this study, a robust contour detection method of the fusion area in LPBF is proposed. In order to raise the contrast of the contour, the phase image with clear contour details will be calculated from a series of fringe images with phase shift projected onto the fusion area. A phase-guided contour extraction method is conducted to accurately locate the center of contour which reduces significantly the impact of the severe manufacturing condition. Experimental results reveal that the proposed method can obtain the contours of the fusion area in a very short time, with higher accuracy and repeatability. In addition, it also holds the potential to be an effective way to monitor the geometric defects layer-wise.
Digital speckle correlation (DSC) solves the problem of searching corresponding points between two images, and it shows great application potential in pattern-projection based fast 3D shape measurement, because only one shot is enough to retrieve the 3D structure. As DSC relies on analyzing the spatial intensity distribution of a subset in image with a given point, it is likely to get false correspondences in low quality DSC area such as the background, because the searching range is hard to locate. So it is still hard to use DSC to realize fast 3D shape measurement. To solve this problem, the gray standard deviation of the subset is designed to recognize and remove the low-quality DSC area, and the principle of epipolar geometry and disparity constraint are utilized to determine the searching range, so the correspondences can be obtained. Moreover, in order to enhance the robustness of this method, a connected region method based on neighboring pixels possessing similar disparity is proposed to remove mismatched points after establishing initial disparity map by correspondences. Once the disparity map is obtained 3D structure can be retrieved based on the triangulation principle. The experiment on reconstructing Gorky plaster statue is performed, verifying that the proposed method can substantially reduce mismatched points and achieve robust single frame 3D shape measurement.
KEYWORDS: Detection and tracking algorithms, Target detection, Calibration, 3D acquisition, 3D metrology, Binary data, Evolutionary algorithms, Cameras, 3D image processing, Digital filtering
Distributing coded targets on the measured object is a reliable and common method for achieving optimum target location and accurate matching of corresponding targets among multi-view images. The circular coded targets which based on a central circular target surrounded by a coded band is widely used in vision measurement. However, it is difficult to decode the coded target while the number of pixels in the coded band is small or the projection angle is large. Aiming at solve this problem, a detection algorithm using the gray gradient to get the central angles of each coded section was proposed. In this algorithm, an accurate ellipse detection which can get sub-pixel locations was adopted to extract ellipse centers, and some false ellipses whose error in the fit of best fit is large will be rejected. Then, gray gradients in the coded band are calculated to get the central angle of each coded section, and the coded target will be decoded accurately. The experiment results show that the algorithm can locate and identify coded targets accurately under complex measurement conditions.
Phase measuring profilometry (PMP) has been widely used in many fields, like Computer Aided Verification (CAV), Flexible Manufacturing System (FMS) et al. High frame-rate (HFR) real-time vision-based feedback control will be a common demands in near future. However, the instruction time delay in the computer caused by numerous repetitive operations greatly limit the efficiency of data processing. FPGA has the advantages of pipeline architecture and parallel execution, and it fit for handling PMP algorithm. In this paper, we design a fully pipelined hardware architecture for PMP. The functions of hardware architecture includes rectification, phase calculation, phase shifting, and stereo matching. The experiment verified the performance of this method, and the factors that may influence the computation accuracy was analyzed.
Measuring high-reflective surfaces using optical method is always a big challenging problem. This paper presents a high-reflective surface measurement method based on conoscopic holography technology using a 4D motion platform equipped with a conoscopic holography optical probe. There are two key problems needed to solve before the automate scan of the complex shape surface: the coordinate calibration and the path planning. To improve the calibration efficiency and accuracy, the coordinate calibration is divided into two parts: the rough calibration and the accurate registration. The path planning consists of two aspects including: the path points generation and the path points verification. In addition, by scanning the objects having high-reflective surfaces, such as the metal blades, coins and other work-pieces, the efficiency of the measurement method has been verified.
Sparse bundle adjustment is widely applied in computer vision and photogrammetry. However, existing implementation is based on the model of n 3D points projecting onto m different camera imaging planes at m positions, which can't be applied to commonly monocular, binocular or trinocular imaging systems. A novel design and implementation of bundle adjustment algorithm is proposed in this paper, which is based on n 3D points projecting onto the same camera imaging plane at m positions .To improve the performance of the algorithm, a novel sparse block matrix partition method is proposed. Experiments show that the improved bundle adjustment is effective, robust and has a better tolerance to pixel coordinates error.
There is currently a very strong need for building 3D models of the visible surface for a wide variety of objects in shape and size. A hand-held 3D scanner is a useful tool in many situations, in case of the single measurement range size, the sensor-tracking devices or surface markers are need to realize multiple view alignment, thus limiting their functionality. We propose an efficient fast registration methods based on both texture and geometry which can bring in additional information and compensate for ambiguities in the other cues. Together, we can use rotation-invariant geometric or photometric feature descriptors to extract faithful corresponding points for matching without sensor-tracking devices or surface markers. Meanwhile, range data alignment based on photometric properties is performed better using a RANSAC algorithm to rule out mismatching. Experimental results with real objects indicate the effectiveness of the proposed approach. We have applied this method to hand-held structured light 3D scanner. It realizes a single view 3D measurement within 0.12S and Real-time global registration.
This paper presents a high-resolution 3D shape acquisition method for developing hand-held 3D measurement system by projecting structured and unstructured patterns. The structured patterns consist of three-step phase-shifting fringe patterns, and the phase can be computed pixel by pixel to achieve high-resolution 3D measurement, and the phaseshifting method commonly requires many additional images to implement a phase unwrapping procedure for obtaining absolute phase value and removing the correspondence ambiguity, but it would slow down the measurement speed and limits its applications in fast 3D measurement. To address this problem, an unstructured pattern using random speckle is employed to reject wrong correspondence and promises a correct 3D result. Therefore, only four images are required to reconstruct one 3D point cloud, which is suitable for fast 3D measurement in hand-held systems. Moreover, the proposed method can be speed up by parallel computing technology for real-time data processing. The experiments verify the performance of the proposed method.
Fast and precise 3D inspection system is in great demand in modern manufacturing processes. At present, the available sensors have their own pros and cons, and hardly exist an omnipotent sensor to handle the complex inspection task in an accurate and effective way. The prevailing solution is integrating multiple sensors and taking advantages of their strengths. For obtaining a holistic 3D profile, the data from different sensors should be registrated into a coherent coordinate system. However, some complex shape objects own thin wall feather such as blades, the ICP registration method would become unstable. Therefore, it is very important to calibrate the extrinsic parameters of each sensor in the integrated measurement system. This paper proposed an accurate and automatic extrinsic parameter calibration method for blade measurement system integrated by different optical sensors. In this system, fringe projection sensor (FPS) and conoscopic holography sensor (CHS) is integrated into a multi-axis motion platform, and the sensors can be optimally move to any desired position at the object’s surface. In order to simple the calibration process, a special calibration artifact is designed according to the characteristics of the two sensors. An automatic registration procedure based on correlation and segmentation is used to realize the artifact datasets obtaining by FPS and CHS rough alignment without any manual operation and data pro-processing, and then the Generalized Gauss-Markoff model is used to estimate the optimization transformation parameters. The experiments show the measurement result of a blade, where several sampled patches are merged into one point cloud, and it verifies the performance of the proposed method.
The multiview phase-shifting method shows its powerful capability in achieving high resolution three-dimensional (3-D) shape measurement. Unfortunately, this ability results in very high computation costs and 3-D computations have to be processed offline. To realize real-time 3-D shape measurement, a hybrid parallel computing architecture is proposed for multiview phase shifting. In this architecture, the central processing unit can co-operate with the graphic processing unit (GPU) to achieve hybrid parallel computing. The high computation cost procedures, including lens distortion rectification, phase computation, correspondence, and 3-D reconstruction, are implemented in GPU, and a three-layer kernel function model is designed to simultaneously realize coarse-grained and fine-grained paralleling computing. Experimental results verify that the developed system can perform 50 fps (frame per second) real-time 3-D measurement with 260 K 3-D points per frame. A speedup of up to 180 times is obtained for the performance of the proposed technique using a NVIDIA GT560Ti graphics card rather than a sequential C in a 3.4 GHZ Inter Core i7 3770.
KEYWORDS: 3D metrology, Cameras, Calibration, Spatial resolution, Temporal resolution, Projection systems, 3D image processing, Phase shifts, 3D acquisition, Computing systems
Numerous fast 3D shape measurement systems based on pattern projection method have been developed in the recent
years, but measuring arbitrary dynamic 3D shape with full resolution, including temporal resolution and spatial
resolution, is still a big challenging problem. This paper presents a real-time 3D measurement system with full spatial
resolution and temporal resolution. In this system, three-step phase-shifting algorithm is employed for full spatial
resolution measurement, and a multi-view phase-shifting correspondence is used to search the corresponding point
independently without additional images. So any adjacent three phase-shifting images in the continuous capturing stream
can be used to reconstruct arbitrary 3D shape, the 3D acquisition speed can be as fast as the camera capturing speed to
achieve full temporal resolution. Moreover, for developing an accurate measurement system the calibration method is
also presented, and it can obtain more accurate internal and external parameters than traditional method in presence of
inaccuracy of calibration target. And a hybrid computing architecture based accelerate calculation method is introduced
to achieve real-time 3D measurement, and the computation speed can be above 280 times faster than before. The
experiments indicate that our system can perform 1024 × 768 full spatial resolution and 220 fps full temporal resolution
3D measurement, and it can also realize real-time 3D measurement at average speed of 50 fps.
In real-time phase measurement profilometry (PMP) systems, finding point correspondence between phase-maps of
cameras or camera and projector is the key technology for 3D reconstruction. A novel method based on trifocal tensor
constrain, which can be directly implemented on wrapped phase-map and speed up by parallel computing technology, is
present. But fringe number, images noise and system non-linear response will lead to false correspondence. The
influence of these factors is deeply analyzed, and a feasible way is proposed to reject error candidate points. Finally, this
method indicated a satisfactory performance in the experiment.
KEYWORDS: Projection systems, Calibration, 3D modeling, Distortion, Cameras, Systems modeling, 3D acquisition, 3D image reconstruction, Imaging systems, 3D image processing
We present a general and effective projector calibration method using a ray-based generic model, which consists of
rays projected from all pixel elements of the projector. For computing the parameters of the rays, we propose a flexible
3D coordinates calculation method for the projected calibration target. Since the ray-based generic model does not rely
on any assumption, our approach is applicable to arbitrary projection system. The calibrated rays can be applied to
evaluate projector's actual distortion model, reconstruct 3D points of the scene and correct geometric distortion of the
projected image etc. Experiments are presented to verify the performance of the proposed technique.
System calibration is crucial for any 3-D shape measurement system. An accurate method is proposed to calibrate a 3-D shape measurement system based on a structured light technique. The projector is treated as a camera to unify the calibration procedures of a structured light system and a well-established stereo vision system. The key to realizing this method is to establish a highly accurate correspondence between camera pixels and projector pixels and generate digital micromirror device (DMD) image sets for projector calibration. A phase-shifting method is used to accomplish this task. A precalibrated lookup table and a linear interpolation algorithm are proposed to improve the accuracy of the generated DMD image, and accordingly improve the accuracy of the projector calibration and the 3-D measurement. Some experimental results are presented to demonstrate the performance of the system calibration.
In this paper a 3D shape measurement system based on the grating method was developed. The system consists of two
CCD cameras and a DLP projector. The multi-frequency heterodyne principle was presented to overcome the
unwrapping problem of phase functions, and a phase-to-coordinates conversion algorithm was proposed to reconstruct
the 3D geometry from the unwrapped phase map. To demonstrate the performance of this 3D measurement system, a
face model was measured, and the reconstructed 3D geometric surface is very smooth with details. Additionally, a ladder
with one step was measured. The result shows the system can measure the discontinuous object and get high
measurement precision.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.