Real-time image processing and regions of interest extraction are crucial in the non-standard welding process guided by structured light vision. However, due to the impact of detection speed, accuracy, and applicability, existing methods are difficult to apply directly. To address these issues, we have screened various improvement methods through experiments and provided a practical lightweight algorithm, YOLO-DGB, based on YOLOv5s, a current mainstream object detection algorithm. The proposed algorithm introduces depthwise separable convolution and Ghost modules into the backbone network to reduce the number of parameters and floating-point operations per second (FLOPs) in the detection process. To meet the accuracy requirements of the detection network, a bottleneck transformer is introduced after the spatial pyramid pooling fast, which improves the detection effect while ensuring a reasonable number of parameters and computation. To address the issue of insufficient datasets, we propose an improved DCGAN network to enhance the collected images. Compared with the original YOLOv5s network, our proposed algorithm reduces the number of parameters and FLOPs by 35.5% and 44.8%, respectively, while increasing the mAP of the model from 96.5% to 98.1%. Experimental results demonstrate that our algorithm can effectively meet the requirements of actual production processes.
A common problem in the field of object detection is that the image features could not be fully expressed. And another issue is that the static query selection in the detection transformer (DETR)-like models cannot adapt well to different datasets due to the fixed number of selected object queries. To solve these problems, hollow attention (HA) and dynamic query selection (DQS) modules were proposed, and a network HA-DQS-Net was further formed. HA integrates specially designed masks into self-attention to better combine channel and spatial directional feature information, thereby learning more complex and comprehensive target features. DQS improves the idea of static query selection in the current DETR-like model by dynamically selecting the number of object queries based on the actual number of targets in the image, which enhances the accuracy of the model. HA-DQS-Net, which combines the advantages of HA and DQS, has a competitive performance in the field of object detection. The excellent detection effectiveness of our viewpoint is validated based on PASVAL VOC and a homemade smoking dataset. It is worth noting that all APs have been improved when HA is applied to different DETR-like models, which improves the universality of the HA module.
Deep learning based on convolutional neural network (CNN) has attracted more and more attention in phase unwrapping of fringe projection three-dimensional (3D) measurement. However, due to the inherent limitations of convolutional operator, it is difficult to accurately determine the fringe order in wrapped phase patterns that rely on continuity and globality. To attack this problem, in this paper we develop a hybrid CNN-transformer model (Hformer) dedicated to phase unwrapping via fringe order prediction. The proposed Hformer model has a hybrid CNN-transformer architecture that is mainly composed of backbone, encoder, and decoder to take advantage of both CNN and transformer. Backbone is used as a wrapped phase pattern feature extractor. Encoder and decoder with cross attention are designed to enhance global dependency for the fringe order prediction. Experimental results show that the proposed Hformer model achieves better performance in fringe order prediction compared with the CNN models such as U-Net and DCNN. Our work opens an alternative way to the CNN-dominated deep learning phase unwrapping of fringe projection 3D measurement.
In view of the problem that traditional collaborative robots mostly work according to the prescribed path through teaching or offline programming cannot obtain the feedback information of the robot in real time, resulting in low intelligence. A multi-decision pressure compensation robot massage control system based on stereo vision imaging is proposed. It could reduce the execution time and positioning error of the robot when conducting human body positioning massage, and perform force compensation based on the massage pressure feedback in real time to improve the level of human-computer interaction. Firstly, hand-eye calibration is carried out through binocular vision sensor and collaborative robot, and the robot positioning control is realized by using high-precision real-time point cloud generated by structured light compensation. The massage pressure is fed back through the pressure sensor on the massage end which the massage force is trimmed to ensure the safety and comfort during the massage process. In the visual working range, the experimental data shows that the depth positioning error is within 0.277mm and the accuracy of the robot massage in the appropriate pressure range can reach 97.35%, which effectively solves the problems in robot massage and increases the standard of human-computer interaction and the immersive experience which is made innovations in the direction of stereo imaging technology and medical treatment.
Phase encoding and phase-shift profilometry are two commonly used 3D measurement techniques. However, the acquired phases in the techniques are subject to jump errors due to phase ambiguity and phase errors caused by multiple heterodyne. The phase-shifting profilometry also makes the selection of fringe period difficult. To overcome this problem and achieve high-precision measurement, a phase unwrapping method that combines dual-frequency heterodyne with double complementary phase encoding is proposed. First, two wrapped phases are obtained by two groups of sinusoidal fringes; the heterodyne phase is obtained after heterodyne processing, and the high-frequency phase is expanded by heterodyne phase. Second, the fringe levels are obtained using the complementary phase encoding fringes that are shifted by half an order, and then the absolute phase is obtained by selecting different phase coding levels according to different regions for the first phase unwrapping; Finally, the phase noise is removed by exploiting the difference between the phase slopes of adjacent pixels. Experimental results show that a system with the proposed method achieves an RMS error of 0.015 mm. In addition, the period of dual-frequency heterodyne synthesis does not need to cover the whole field of view, which breaks the limitation of frequency selection of the traditional dual-frequency heterodyne method and triple frequency heterodyne method, enabling high-precision measurement with higher frequency fringes. This method overcomes the limitations of the phase principal value error when using higher frequency fringes for high-precision measurement, improves the measurement effect of reflective objects, and effectively avoids the error caused by phase jump.
KEYWORDS: Image fusion, 3D metrology, 3D image processing, Calibration, Cameras, Clouds, High dynamic range imaging, Reflection, 3D image reconstruction, Image information entropy
To deal with the three-dimensional (3-D) point cloud loss caused by object reflection in the active fringe projection 3-D measurement, an active reflection suppression method for 3-D measurement is proposed. The method employs high-dynamic range images obtained by multiple exposure image fusion and a three-wavelength phase-shift profilometry method to achieve high-precision 3-D measurement of reflective objects. Experimental results show that, compared to traditional 3-D measurement methods, the proposed one can more effectively handle reflections thereby avoiding 3-D point cloud loss in the measurement of reflective objects.
KEYWORDS: 3D metrology, Calibration, Cameras, Clouds, 3D imaging standards, 3D modeling, 3D acquisition, Image registration, Optical engineering, Phase shift keying
A full-view three-dimensional (3-D) measurement method for complex surfaces is proposed, where 3-D data for standard balls with different angles are used to calibrate the rotation axes of a turntable and obtain transformation matrices of 3-D data of adjacent views. It can achieve accurate registration of 3-D data of views with different angles and obtains full-view 3-D data for complex surfaces in conjunction with the method for principal point calibration of cameras and modified triple-frequency six-step phase-shifting phase demodulation methods. Experiments show that the developed system based on the proposed method can achieve automatic registration of 3-D data of views with different angles, and good full-view 3-D measurement precision for complex surfaces.
An integrated method is proposed for the real-time measurement of filament lamp dimension based on machine vision (FLDMV). First, an online detection platform is built, and the image is acquired by telecentric lenses and charge-coupled diode (CCD). Second, a series of image processing, including filter, edge extraction, ellipse fitting, recursive minimum bounding rectangle, and curvature restrict estimation. Finally, the actual size of lamp is obtained by system calibration. The experimental analysis and comparison show that the maximum measurement error of this method is 0.21mm, which meets the requirements of filament lamp dimension measurement. The curvature restrict estimation based on ellipse fitting are proposed to guarantee the accuracy and real time. Compared with the traditional measurement method, our method has the advantages of fast measurement speed, high accuracy, and real time. It also can be widely used in other parts of the measurement.
The existing Gaussian Mixture Model(GMM) which is widely used in vehicle detection suffers inefficiency in detecting foreground image during the model phase, because it needs quite a long time to blend the shadows in the background. In order to overcome this problem, an improved method is proposed in this paper. First of all, each frame is divided into several areas(A, B, C and D), Where area A, B, C and D are decided by the frequency and the scale of the vehicle access. For each area, different new learning rate including weight, mean and variance is applied to accelerate the elimination of shadows. At the same time, the measure of adaptive change for Gaussian distribution is taken to decrease the total number of distributions and save memory space effectively. With this method, different threshold value and different number of Gaussian distribution are adopted for different areas. The results show that the speed of learning and the accuracy of the model using our proposed algorithm surpass the traditional GMM. Probably to the 50th frame, interference with the vehicle has been eliminated basically, and the model number only 35% to 43% of the standard, the processing speed for every frame approximately has a 20% increase than the standard. The proposed algorithm has good performance in terms of elimination of shadow and processing speed for vehicle detection, it can promote the development of intelligent transportation, which is very meaningful to the other Background modeling methods.
Fringe pattern profilometry (FPP) is one of the most promising 3D profile measurement techniques, which has been
widely applied in many areas. A challenge problem associated with FPP is the unwrapping of wrapped phase maps
resulted from complex object surface shapes. Although existing quality-guided phase unwrapping algorithms are able to
solve such a problem, they are usually extensively computational expensive and not able to be applied to fast 3D
measurement scenarios. This paper proposes a new quality-guided phase unwrapping algorithm with higher
computational efficiency than the conventional ones. In the proposed method, a threshold of quality value is used to
classify pixels on the phase maps into two types: high quality (HQ) pixels corresponding to smooth phase changes and
low quality (LQ) ones to rough phase variance. In order to improve the computational efficiency, the HQ pixels are
unwrapped by a computationally efficient fast phase unwrapping algorithm, and the LQ pixels are unwrapped by
computational expensive flood-fill algorithm. Experiments show that the proposed approach is able to recover complex
phase maps with the similar accuracy performance as the conventional quality-guided phase unwrapping algorithm but is
much faster than the later.
Phase shifting profilometry (PSP) technique is widely used as a 3-D shape measurement technique due to its robustness
and accuracy. However, PSP requires multiple fringe pattern images to be projected onto an object and a reference plane
to calculate the phase value, and also the object must maintain motionless when the measurement is taken. If the object
moves during the measurement, significant errors will be introduced when calculating the phase value. This paper
analyses the relationship between the object movement and the phase value, and proposes a method for compensating the
errors caused by two-dimensional movement of object. This method can eliminate the errors caused by two-dimensional
movement of object and reconstruct the object shape successfully. The effectiveness of the proposed method is verified
by simulations.
KEYWORDS: Cameras, 3D image processing, Video, Defect detection, Atomic force microscopy, 3D metrology, 3D modeling, Calibration, Reconstruction algorithms, Structured light
The fabric quality defect detection is very useful for improving the qualities of the products. It is also very important to
increase the reputation and the economic benefits of a company. However, there are some shortcomings in the traditional
manual detection methods, such as the low detection efficiency, the fatigue problem of the operator, and the detection
inaccuracy, etc. The existing 2D image processing methods are difficult to solve the interference which is caused by
non-defect case, just like the cloth folds, the flying thick silk floss, the noise from the background light and ambient
light, etc. In order to solve those problem, the BCCSL (Binocular Camera Color Structure Light) method and SFMS
(Shape from Multi Shading) method is proposed in this paper. The three-dimensional color coordinates of the fabric can
be quickly and highly-precision obtained, thus to judge the defects shape and location.
The BCCSL method and SFMS method can quickly obtain the three-dimensional coordinates' information of the fabric
defects. The BCCSL method collects the 3D skeleton's information of a fabric image through the binocular video
capture device and the color structured light projection device in real-time. And the details 3D coordinates of fabric
outside strip structural are obtained through the proposed method SFMS. The interference information, such as the cloth
fold, the flying thick silk floss, and the noise from the background light and ambient light can be excluded by using the
three-dimensional defect identification. What is more, according to the characteristics of 3D structure of the defect, the
fabric can be identified and classified. Further more, the possible problems from the production line can be summarized.
KEYWORDS: Surgery, 3D image processing, Medical imaging, Visualization, Image processing, Computed tomography, Image visualization, 3D modeling, Image segmentation, 3D displays
Although the CT device can give the doctors a series of 2D medical images, it is difficult to give vivid view for the
doctors to acknowledge the decrease part. In order to help the doctors to plot the surgery, the virtual surgery system is
researched based on the three-dimensional visualization technique. After the disease part of the patient is scanned by the
CT device, the 3D whole view will be set up based on the 3D reconstruction module of the system. TCut a part is the
usually used function for doctors in the real surgery. A curve will be created on the 3D space; and some points can be
added on the curve automatically or manually. The position of the point can change the shape of the cut curves. The
curve can be adjusted by controlling the points. If the result of the cut function is not satisfied, all the operation can be
cancelled to restart. The flexible virtual surgery gives more convenience to the real surgery. Contrast to the existing
medical image process system, the virtual surgery system is added to the system, and the virtual surgery can be plotted
for a lot of times, till the doctors have enough confidence to start the real surgery. Because the virtual surgery system can
give more 3D information of the disease part, some difficult surgery can be discussed by the expert doctors in different
city via internet. It is a useful function to understand the character of the disease part, thus to decrease the surgery risk.
Data Mining of Manufacturing Quality Information (MQI) is the key technology in Quality Lead Control. Of all the data mining methods, Neural Network and Genetic Algorithm is widely used for their strong advantages, such as non-linear, collateral, veracity etc. But if you singly use them, there will be some limitations preventing your research, such as convergence slowly, searching blindness etc. This paper combines their merits and use Genetic BP Algorithm in Data Mining of MQI. It has been successfully used in the key project of Natural Science Foundation of China (NSFC) - Quality Control and Zero-defect Engineering (Project No. 59735120).
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.