|
|
1.INTRODUCTIONWith the continuous development of optical inspection technology and researchers’ pursuit of imaging quality, the size of equipment lenses is showing a trend to become larger and larger, but larger single-lens lenses require more stringent processing techniques and higher costs. Since the traditional single-lens imaging effect is limited by cost, size, and performance, the concept of an array camera was born. The principle of array cameras is to place cameras at multiple locations in a four-dimensional light field to perform two-dimensional slicing of the four-dimensional light field, which is equivalent to a two-dimensional sampling of the light field from multiple angles. An 8 × 8 array camera was designed in the literature [1]. This device is characterized by the ability to perceive the moving light field. A 6 × 8 light field imaging device was designed in the literature [2]. This device can render images of viewpoints that are not at the 6 × 8 viewpoint location. It is done by rearranging the viewpoints of each sub-camera position. Better imaging quality is obtained by this means. The literature [3] produced an 8×12 array camera and the acquired light field data was made into a public dataset. A research team from Northwestern Polytechnic University developed an 8 × 8 camera array[4]. They conducted an in-depth study based on this array of the camera in topics such as camera calibration[5]. The array cameras mentioned above are large optical field imaging devices, which are expensive. On the other hand, as cameras are gradually moving towards miniaturization, there are teams aiming at the miniaturization of array cameras. PiCam is an ultra-thin 4 × 4 array imaging device with an overall size comparable to a coin, developed by Venkataraman’s team[6]. Camera arrays use multiple cameras located at different spatial locations to collect photographs with different perspectives, adding many possibilities for complementary information for applications such as post-capture refocusing[7,8], super-resolution[9], view synthesis[10], and de-obscuring[11]. Wilburn et al[12] used inexpensive cameras to build high-performance camera arrays for high-resolution and high dynamic range video capture, Spatio-temporal view interpolation, and nonlinear synthetic aperture photography. Vaish et al[13] also performed related work with the support of camera arrays. The camera array is calibrated by plane + parallax, which is used to view the surface of an obscured object behind a tree branch or a person. Yang et al[14] proposed a novel real-time hybrid synthetic aperture imaging monitoring technique for solving the blurring problem caused by capturing high-speed moving objects with small apertures in a low-light environment. Xiao et al[15] proposed a camera array-based image computational refocusing method, which can focus the light field image to any depth and improve the signal-to-noise ratio of the focused image. To improve the signal-to-noise ratio of infrared images, the parallax of sub-cameras needs to be calculated. Zeng et al[16] reviewed the camera array system in spatial target observation, which can bring low cost, wide field of view, high resolution, and high detection capability. Based on this, this paper proposes a refocusing-based infrared array camera dim target signal-to-noise ratio enhancement method to achieve the enhancement of signal-to-noise ratio for dim target detection using infrared array cameras to support infrared target detection research. 2.METHODOLOGY2.1Infrared image space noise theory analysisInfrared imaging systems consist of many system units, and each system unit introduces noise. Therefore, noise exists in various forms, and the spatial noise of infrared images is mainly background noise. Background noise is mainly the radiation noise of the scene and the noise caused by atmospheric jitter, etc. Background radiation mainly refers to the natural radiation sources that can radiate infrared light, such as air and clouds. Background radiation has a rise and fall, the background photon emission is irregular, the emitted power fluctuates around the average value, and the power fluctuation size can be expressed in mean square deviation. Emission radiation in the bandwidth of Δf The mean-squared noise voltage value generated at the time is the following equation. Xe is the emissivity of the background. e is the Stefan-Boltzmann constant. k is the Boltzmann constant. Tb is the background temperature. Ab is the background radiation area. The power of the background noise is independent of the frequency, and it is white noise. From the above analysis, it can be seen that the focal plane detector is basically in line with the Gaussian distribution except for the pretzel noise caused by the corresponding large or small individual image elements, the inherent image noise caused by the inconsistent response of each image element of the detector itself, and the image inhomogeneity noise caused by the image uniformity correction. 2.2Array camera dim target signal-to-noise ratio enhancement technique2.2.1Synthetic aperture imaging with array camerasThe array camera can make multiple observations of the current scene, the background noise can be effectively suppressed and the signal-to-noise ratio can be enhanced by the synthetic aperture imaging method. For the array camera, the following equation exists. Ik is the image of the scene acquired by the first camera in the array camera k acquired by the first camera, and Bk is the first k is the background in the first image and tk is the target in the first k is the target in the first image, and nk ∼ N(0, σ2) the noise in the k image. Due to the previous analysis, we assume that the noise between each camera is independent and obeys a Gaussian distribution. Using the synthetic aperture imaging method, we superimpose the images from each viewpoint of the array cameras, for which the following equation exists. Among them, d is the value of the difference in position of the target in the images of the different cameras (in pixels). b is the baseline length of the array cameras (distance between two adjacent camera optical axes). f is the focal length of the camera normalized to the pixel value. γ is the distance between the target and the camera. In the actual detection imaging process, the value of the distance from the target to the camera γ is much larger than the baseline length of the camera, b and therefore the spatial position of the target between cameras is essentially the same (d → 0). We superimpose the images of each viewpoint acquired by the array cameras, which can effectively suppress the Gaussian random noise while keeping the intensity of the target essentially unchanged. 2.2.2Signal-to-noise ratio enhancementThe formula for calculating the signal-to-noise ratio is defined as follows. Et is the energy of the target, the En is the noise energy, and Ires is the residual image after background suppression, and T is the set of image elements occupied by the target, and U is the set of full image elements. |U| and |T| denote the number of image elements in the set, respectively. From the above, it can be seen that the noise in the synthetic aperture image obeys the distribution In the above equation C1 is a constant, it is known from the above equation that the noise energy is inversely proportional to the 1/2 power of the number of cameras. And the target energy remains constant in the synthetic imaging process, so By theoretical derivation, we can conclude that the signal-to-noise ratio of the image obtained by synthetic aperture imaging is proportional to the 0.5 power of the number of cameras. 2.2.3Signal-to-noise ratio enhancementThe array camera can enhance the signal-to-noise ratio of the target by synthetic aperture imaging to make the target easier to detect. Since the signal-to-noise ratio of the target decreases with the increase of the detection distance, the synthetic aperture imaging method with array cameras can increase the ultimate detection distance by increasing the signal-to-noise ratio. This subsection focuses on the relationship between the limiting detection distance and the number of cameras in an array camera. Assuming that the target is a point source with constant radiated power in space and propagates uniformly outward along the sphere, the energy of the target acquired per unit area per unit time is the following equation. P0 is the target energy detected by the infrared detector per unit time. S0 is the aperture area of the infrared detector. P is the radiated power of the target. r is the distance between the IR detector and the target. Notice that the grayscale value of the IR image is P0 is proportional to the average energy of the target in the infrared image, so the average energy of the target in the infrared image Et is directly proportional to where C2 is a constant. Bringing the above equation into the signal-to-noise ratio equation, we get Let the limiting signal-to-noise ratio of target detection be SNRmin then the relationship between the detection distance and the number of cameras in the array at a fixed limiting S/N ratio can be written as It follows that the limiting detection distance of the detector is proportional to the 0.25 power of the number of cameras. This theoretically demonstrates that the imaging regime using array cameras can increase the detection distance by enhancing the signal-to-noise ratio. 3.EXPERIMENTAL RESULTS AND ANALYSIS3.1Experimental validationIn 2020, the literature[17] proposed a dynamic parameter method that allows focusing the light field data to different depths. In 2004, the literature[18] proposed to back-project the light field image onto an artificially specified reference plane based on the single-strain method with the premise that the outer parameters are obtained by camera correction, and the subimages on the reference plane are reprojected to the reference camera plane for translational overlap summation to obtain a refocused image focused on the reference plane at the specified depth. Based on this, experiments will be conducted in this section using the refocusing-based signal-to-noise ratio enhancement method for dim targets of infrared array cameras proposed in this paper. The flowchart and principle of the method are shown in Fig. 1. Using the reprojection idea to back-project the light field image onto a reference plane at a specified depth, the sub images on the reference plane are reprojected onto the reference camera plane for translation overlap summation to obtain the refocused image focused at the specified depth. In the focusing process, the interpolated image is obtained by one interpolation method for the pixel positions that are not strictly aligned. Compared with multiple interpolation methods, one interpolation can minimize the error generated by interpolation. 3.2Experimental analysisIn this experiment, we choose four sub-cameras to form the camera matrix, and this section shows the dim target signal-to-noise ratio enhancement effect of the method and analyzes it. As shown in Figure 2, the imaging effects of the four sub-cameras of the array camera are shown, and the dim target signal-to-noise ratios of each sub-camera are calculated as SNR1 = 98.19, SNR2 = 75.72, SNR3 = 109.39 and SNR4 = 92.04. Figure 2 shows the imaging effect obtained after the SNR enhancement method, and the SNR can be obtained as SNR = 150.32. The following analysis of its limit detection distance, through the analysis of Section 2.2.3 detection distance r satisfies the equation (13). By analyzing the experimental results, the enhancement effect of the refocusing-based IR array camera dim target signal-to-noise ratio enhancement method is shown in Table 1. Table 1.Demonstration of the enhancement effect of dim target signal-to-noise ratio enhancement methods.
4.CONCLUSIONIn this paper, we propose a refocusing-based infrared array camera dim target signal-to-noise ratio enhancement method, which can effectively improve the detection distance of the imaging system for dim targets. This paper introduces the feasibility of the method through theory and experimental validation at the same time. The experimental results show that the method achieves the enhancement of the signal-to-noise ratio of dim targets and overcomes the limitations of the traditional single-lens dim target detection imaging system in terms of cost, size and performance, and can be applied to the relevant subsequent processing tasks. REFERENCESYang J C, Everett M, Buehler C McMillan, L,
“A real-time dis-tributed light field camera,”
Eurographics Workshop on Ren-dering, 77
–86
(2002). Google Scholar
Zhang C, Chen T,
“A self-reconfigurable camera array,”
in Pro-ceedings of the Fifteenth Eurographics conference on Render-ing Techniques,
243
–254
(2004). Google Scholar
Wilburn B, Joshi N, Vaish V, Talvala E V, Antunez E, Barth A, Adams A, Horowitz M, Levoy M,
“High performance imaging using large camera arrays,”
acm Transactions on Graphics (TOG), 24
(3), 765
–776
(2005). https://doi.org/10.1145/1073204.1073259 Google Scholar
Xiao Z, Si L, Zhou G,
“Seeing Beyond Foreground Occlusion: a Joint Framework for SAP-Based Scene Depth and Appearance Reconstruction,”
IEEE Journal of Selected Topics in Signal Pro-cessing, 11
(7), 979
–991
(2017). https://doi.org/10.1109/JSTSP.2017.2715012 Google Scholar
Zhang Q, Zhang C, Ling J, Wang Q, Yu J,
“A Generic Mul-ti-Projection-Center Model and Calibration Method for Light Field Cameras,”
IEEE transactions on pattern analysis and machine intelligence,
(2018). Google Scholar
Venkataraman K, Lelescu D, Duparré J, McMahon A, Molina G, Chatterjee P, Mullis R, Nayar S,
“Picam: An ultra-thin high performance monolithic camera array,”
ACM Transactions on Graphics (TOG), 32
(6), 166
(2013). https://doi.org/10.1145/2508363.2508390 Google Scholar
Yang, Jungang, et al.,
“High-precision refocusing method with one interpolation for camera array images,”
IET Image Processing, 14
(5),
(2020). Google Scholar
Wang Y., et al.,
“Selective Light Field Refocusing for Camera Arrays Using Bokeh Rendering and Superresolution,”
IEEE Signal Processing Letters, 204
–208
(2018). Google Scholar
Wang, Y, et al.,
“Spatial-Angular Interaction for Light Field Image Super-Resolution,”
(2020). https://doi.org/10.1007/978-3-030-58592-1 Google Scholar
Wu, G., et al.,
“Spatial-Angular Attention Network for Light Field Reconstruction,”
IEEE Trans on Image Processing,
(2021). https://doi.org/10.1109/TIP.2021.3122089 Google Scholar
Wang, Y. et al.,
“DeOccNet: Learning to See Through Foreground Occlusions in Light Fields,”
in 2020 IEEE Winter Conference on Applications of Computer Vision (WACV) IEEE,
(2020). https://doi.org/10.1109/WACV45572.2020 Google Scholar
Wilburn, Bennett, et al.,
“High performance imaging using large camera arrays,”
ACM Transactions on Graphics (TOG),
(2005). https://doi.org/10.1145/1073204.1073259 Google Scholar
Vaish, V., et al.,
“Using plane+ parallax for calibrating dense camera arrays,”
in Computer Vision and Pattern Recognition, 2004. (CVPR). Proceedings of the 2004 IEEE Computer Society Conference on,
(2004). https://doi.org/10.1109/CVPR.2004.1315006 Google Scholar
Yang, T., et al.,
“Continuously tracking and see-through occlusion based on a new hybrid synthetic aperture imaging model,”
in Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on,
(2011). https://doi.org/10.1109/CVPR.2011.5995417 Google Scholar
Chao, X., et al.,
“A Flexible Refocusing Method for Camera Arrays Based on Reprojection,”
in International Conference on Computer, Engineering and Information Science,
(2017). Google Scholar
Hairui Z. et al.,
“Survey of the application of camera array in space target ovservation,”
Laser and Infrared, 48
(11), 1323
–1329
(2018). Google Scholar
Isaksen A,
“Dynamically reparameterized light fields,”
297
–306
(2000). Google Scholar
Vaish V, Wilburn B, Joshi N, Levoy M,
“Using plane+ parallax for calibrating dense camera arrays,”
null, 2
–9 IEEE,2004). Google Scholar
|
