In this work, we will share the main results achieved with a Long Wave Infrared (LWIR) Light Field (LF) imaging system with two novel capabilities relevant to IR image science applications: The capability of digitally refocusing to any nearby object planes with a high Signal to Noise Ratio (SNR), this is, to achieve refocused image object planes almost free of Fixed-Pattern Noise (FPN) and blur artifacts. And, the capability of achieving multispectral LWIR imaging for the global scene and for all the refocused nearby object planes required, this is, LWIR radiometry refocusing capacity. The built-in LWIR LF imaging system is implemented with an LWIR microbolometer Xenics camera 8-12 micrometers spectral band, and a high precision scanning system (Newport). LWIR multispectral capacity is achieved with an array of narrow-band LWIR interference optical filters.
This research shows a prototype for crowd location and counting for earthquakes based on deep learning and the infrastructure of a state-of-the-art 5G standalone network deployed at the Universidad de Concepcion, Chile. The system uses an 8 MP panoramic network camera to capture real-time crowd images, which are sent to a Deep Learning Server (DLS) over the 5G network. The camera provides visible color images, and its sensor technology can provide color images even at night. The DLS uses frames from the video feed and generates Focal Inverse Distance Transform (FIDT) maps, in which the counting and location of people are carried out. In particular, the FIDT maps are generated from the crowd images using a deep-learning model composed of two cascaded autoencoders. The 5G technology allows the system to transfer data from the camera to DLS at high speed, an essential feature for a system that will help authorities make critical decisions during natural disasters. Under this scenario, and considering that the number of rescuers is usually limited, our system enables a better distribution of them among several crowded places by instantly knowing the number of people at any time of the day or night.
This paper presents SAFE, a prototype system for supporting the fish landings control of small-scale fishing boats in Chile. SAFE is a modern solution for fishery inspection that automatically discriminates fish species using machine learning. Here, we present a version of SAFE that classifies five target pelagic fish species in Chile: anchovy, Chilean jack mackerel, hake, mote sculpin, and sardine. The system has two stages; the first detects and segments all fish appearing in an image. These segmented images then feed the second stage, which perform species classification. A database of approximately 266 images from these five fish species was constructed for training, validation, and testing purposes. For the fish detection stage, we exploited transfer learning to train Mask R-CNN architectures, an instance segmentation model. As for the fish species classification stage, we exploited transfer learning to train ResNet50 and VGG16 deep learning architectures. Results show that SAFE achieves between 90% and 96.3% macro-average precision (MP) when classifying the five fish species mentioned above. The best architecture, composed of a Mask R-CNN-based detector and a VGG16-based classifier, achieves an MP of 96.3%, which could process a single fish as quick as 16.67 FPS, and one whole 1920x1080-pixel image as quick as 2 FPS.
Earthquakes, and their cascading threats to economic and social sustainability, are a common problem between China and Chile. In such emergencies, automatic image recognition systems have become critical tools for preventing and reducing civilian casualties. Human crowd detection and estimation are fundamental for automatic recognition under life-threatening natural disasters. However, detecting and estimating crowds in scenes is nontrivial due to occlusion, complex behaviors, posture changes, and camera angles, among other issues. This paper presents the first steps in developing an intelligent Earthquake Early Warning System (EEWS) between China and Chile. The EEWS exploits the ability of deep learning architectures to properly model different spatial scales of people and the varying degrees of crowd densities. We propose an autoencoder architecture for crowd detection and estimation because it creates compressed representations for the original crowd input images in its latent space. The proposed architecture considers two cascaded autoencoders. The first performs reconstructive masking of the input images, while the second generates Focal Inverse Distance Transform (FIDT) maps. Thus, the cascaded autoencoders improve the ability of the network to locate people and crowds, thereby generating high-quality crowd maps and more reliable count estimates.
In this work, mid wavelength infrared microscopy imaging videos of several index finger pads, from voluntary people, are recorded to obtain their thermoregulation curves. The proposed non-invasive technique is able to capture spatial and temporal thermal information emitted from blood vessels under-skin, and the irrigation finger pad system, making possible to capture features that a visual-spectrum microscopy cannot detect. Using an infrared laboratory prepared method several voluntary patients exposed theirs fingers to thermal stress while the infrared data is recorded. Using standard infrared imaging and signal processing techniques the thermoregulation curves are estimated. The Cold/Hot Stress experiments have shown infrared data with exponential trend curves, with different recovering slopes for each voluntary person, and sometimes with two steps increasing slope in one person thermoregulation curve response.
A non-contact infrared imaging-based measurement technique is applied to quantify the enzymatic reaction of glucokinase. The method is implemented by a long-wave (8-12 [μm]) infrared microbolometer imaging array and a germanium-based infrared optical vision system adjusted to the size of a small biological sample. The enzymatic reaction is carried out by the glucokinase enzyme, which is representative of the internal dynamics of the cell. Such reactions produce a spontaneous exothermal release of energy detected by the infrared imaging system as a non-contact measurement technique. It is shown by stoichiometry computations and infrared thermal resolution metrics that the infrared imaging system can detect the energy release at the [mK] range. This allows to quantify the spontaneity of the enzymatic reaction in a three dimensional (surface and time) single and noncontact real- time measurement. The camera is characterized for disclosing its sensibility, and the fixed pattern noise is compensated by a two point calibration method. On the other hand, the glucokinase enzyme is isolated from Pyrococcus furiosus. Therefore, the experiment is carried out by manual injection with graduated micropipettes using 40 [μl] of glucokinase at the surface of the substrate contained in an eppendorf tube. For recording, the infrared camera is adjusted in-focus at 25.4 [mm] from the superficial level of the substrate. The obtained values of energy release are 139 ± 22 [mK] at room temperature and 274 ± 22 [mK] for a bath temperature of 334 [K].
In this paper, a prior knowledge model is proposed in order to increase the effectiveness of a multidimensional striping noise compensation (SNC) algorithm. This is accomplished by considering an optoelectronic approach, thereby generating a more accurate mathematical representation of the hyperspectral acquisition process. The proposed model includes knowledge on the system spectral response, which can be obtained by means of an input with known spectral radiation. Further, the model also considers the dependence of the noise structure on the analog-digital conversion process, that is, schemes such as active-pixel sensor (APS) and passive-pixel sensor (PPS) have been considered. Finally, the model takes advantage of the degree of crosstalk between consecutive bands in order to determinate how much of this spectral information is contributing to the read out data obtained in a particular band. All prior knowledge is obtained by a series of experimental analysis, and then integrated into the model. After estimating the required parameters, the applicability of the multidimensional SNC is illustrated by compensating for stripping noise in hyperspectral images acquired using an experimental setup. A laboratory prototype, based on both a Photonfocus Hurricane hyperspectral camera and a Xeva Xenics NIR hyperspectral camera, has been implemented to acquire data in the range of 400-1000 [nm] and 900-1700 [nm], respectively. Also, a mobile platform has been used to simulate and synchronize the scanning procedure of the cameras and an uniform tungsten lamp has been installed to ensure an equal spectral radiance between the different bands for calibration purpose.
This paper presents a digital hardware filter that estimates the nonuniformity (NU) noise in an Infrared Focal Plane Array (IRFPA) and corrects it in real time. Implementing the algorithm in hardware results in a fast, compact, low-power nonuniformity correction (NUC) system that can be embedded into an intelligent imager at a very low cost. Because it does not use an external reference, our NUC circuit works in real time during normal operation, and can track parameter drift over time. Our NUC system models NU noise as a spatially regular source of additive noise, uses a Kalman filter to estimate the offset in each detector of the array and applies an inverse model to recover the original information captured by the detector. The NUC board uses a low-cost Xilinx Spartan 3E XC3S500E FPGA operating at 75MHz. The NUC circuit consumes 17.3mW of dynamic power and uses only 10% of the logic resources of the FPGA. Despite ignoring the multiplicative effects of nonuniformity, our NUC circuit reaches a Peak Signal-to-Noise Ratio (PSNR) of 35dB in under 50 frames, referenced to two-point calibration using black bodies. This performance lies within 0.35dB of a double-precision Matlab implementation of the algorithm. Without the bandwidth limitations currently imposed by the external RAM that stores the offset estimations, our circuit can correct 320x240-pixel video at up to 1,254 frames per second.
In this paper the effects of the internal temperature on the response of uncooled microbolometer cameras have
been studied. To this end, different temperature profiles steering the internal temperature of the cameras have
been generated, and black-body radiator sources have been employed as time and temperature constant radiation
inputs. The analysis conducted over the empirical data has shown the existence of statistical correlation between
camera's internal temperature and the fluctuations in the read-out data. Thus, when measurements of the
internal temperature are available, effective methods for compensating the fluctuations in the read-out data can
be developed. This claim has been tested by developing a signal processing scheme, based on a polynomial
model, to compensate for the output of infrared cameras equipped with amorphous-Silicon and Vanadium-Oxide
microbolometers.
The aim of this research is to experimentally validate a Gauss-Markov model, previously developed by our
group, for the non-uniformity parameters of infrared (IR) focal plane arrays (FPAs). The Gauss-Markov model
assumed that both, the gain and the offset parameters at each detector, are random state-variables modeled by a
recursive discrete-time process. For simplicity, however, we have regarded here the gain parameter as a constant
and assumed that solely the offset parameter follows a Gauss-Markov model. Experiments have been conducted
at room temperature and IR data was collected from black-body radiator sources using microbolometer-based
IR cameras operating in the 8 to 12 μm. Next, well-known statistical techniques were used to analyze the offset
time series and determinate whether the Gauss-Markov model truly fits the temporal dynamics of the offset. The
validity of the Gauss-Markov model for the offset parameter was tested at two time scales: seconds and minutes.
It is worth mentioning that the statistical analysis conducted in this work is a key in providing mechanisms for
capturing the drift in the fixed pattern noise parameters.
KEYWORDS: Cameras, Staring arrays, Near infrared, Thermal modeling, Imaging systems, Temperature metrology, Cooling systems, Data modeling, Systems modeling, Hyperspectral imaging
Our group has developed a Planck physics-based model for the input/output behavior of near infrared (NIR)
hyperspectral cameras. During the validation of the model, experiments conducted using an NIR hyperspectral
camera have shown that, when thermal radiation is used as the camera input and no illumination is present,
the output offset happens to be thermally dependent, yet independent of the wavelengths in the NIR band. In
this work, the effect of the incident temperature on the amount of output offset in NIR hyperspectral cameras
has been experimentally studied and introduced in our previous model for such cameras. The experimental
study has been conducted using an NIR hyperspectral camera in the range of 900 to 1700 [nm] and a controlled
illumination set-up, while different input temperatures have been controlled by means of black-body radiator
sources. The thermal-dependent offset is modeled phenomenologically from experimental data. Initial results
have shown a non-linear dependence between the offset and the temperature. This thermal-offset dependence
can be used to generate new NIR hyperspectral models, new non-linear calibration procedures, and establish a
basis for the study of time dependent variations of the NIR thermal-offset.
Quality control of clams considers the detection of foreign objects like shell pieces, sand and even parasites.
Particularly, Mulinia edulis clams are susceptible to have a parasite infection caused by the isopoda Edotea
magellanica, which represents a serious commercial problem commonly addressed by manual inspection. In this
work a machine vision system capable of automatically detect the parasite using a clam image is presented. The
parasite visualization inside the clam is achieved by an optoelectronic imaging system based on an transillumination
technique. Furthermore, automatic parasite detection in the clam's image is accomplished by a pattern
recognition system designed to quantitatively describe parasite candidate zones. The extracted features are used
to predict the parasite presence by means of a binary decision tree classifier. A real sample dataset of more than
155000 patterns of parasite candidate zones was generated using 190 shell-off cooked clams from the Chilean
south pacific coasts. This data collection was used to train a test the classifier using cross-validation. Primary
results have shown a mean parasite detection rate of 85% and a mean total correct classification of 87%, which
represent a substantive improvement to the existing solutions.
The accuracy achieved by applications employing hyperspectral data collected by hyperspectral cameras depends
heavily on a proper estimation of the true spectral signal. Beyond question, a proper knowledge about the sensor
response is key in this process. It is argued here that the common first order representation for hyperspectral
NIR sensors does not represent accurately their thermal wavelength-dependent response, hence calling for more
sophisticated and precise models. In this work, a wavelength-dependent, nonlinear model for a near infrared
(NIR) hyperspectral camera is proposed based on its experimental characterization. Experiments have shown
that when temperature is used as the input signal, the camera response is almost linear at low wavelengths,
while as the wavelength increases the response becomes exponential. This wavelength-dependent behavior is
attributed to the nonlinear responsivity of the sensors in the NIR spectrum. As a result, the proposed model
considers different nonlinear input/output responses, at different wavelengths. To complete the representation,
both the nonuniform response of neighboring detectors in the camera and the time varying behavior of the input
temperature have also been modeled. The experimental characterization and the proposed model assessment
have been conducted using a NIR hyperspectral camera in the range of 900 to 1700 [nm] and a black body
radiator source. The proposed model was utilized to successfully compensate for both: (i) the nonuniformity
noise inherent to the NIR camera, and (ii) the stripping noise induced by the nonuniformity and the scanning
process of the camera while rendering hyperspectral images.
The multiplicative and additive components of the fixed-pattern noise (FPN) in infrared (IR) focal plane arrays
(FPAs) are typically modeled as time-stationary, spatially unstructured random processes. Even though the latter
assumption is convenient, it is also inaccurate due to FPN is indeed observed as a spatial pattern, with random
intensity values, superimposed over the true images. In this paper, the spatial structure in both the multiplicative
and the additive components of the FPN has been modeled in the frequency domain. The key observation in the
proposed models is that regular spatial patterns manifest themselves as narrowband components in the magnitude
spectrum of an image. Thus, the spatial structure of FPN can be abstracted in a straightforward manner by
approximating the spectral response of the FPN. Moreover, the random intensity of the FPN has been also
modeled by matching the empirically estimated distributions of the intensity values of both multiplicative and
additive components of the FPN. Experimental characterization of FPN has been conducted using black-body
radiator sources, and the theoretical as well as practical applicability of the proposed models has been illustrated
by both synthesizing FPN from three different IR cameras and by proposing a simple yet effective metric to
assess the amount of FPN in FPA-based cameras.
Algorithms for striping noise compensation (SNC) for push-broom hyperspectral cameras (PBHCs) are primarily
based on image processing techniques. These algorithms rely on the spatial and temporal information available
at the readout data; however, they disregard the large amount of spectral information also available at the data.
In this paper such flaw has been tackled and a multidimensional approach for SNC is proposed. The main
assumption of the proposed approach is the short-term stationary behavior of the spatial, spectral, and temporal
input information. This assumption is justified after analyzing the optoelectronic sampling mechanism carried
out by PBHCs. Namely, when the wavelength-resolution of hyperspectral cameras is high enough with respect
to the target application, the spectral information at neighboring photodetectors in adjacent spectral bands can
be regarded as a stationary input. Moreover, when the temporal scanning of hyperspectral information is fast
enough, consecutive temporal and spectral data samples can also be regarded as a stationary input at a single
photodetector. The strength and applicability of the multidimensional approach presented here is illustrated by
compensating for stripping noise real hyperspectral images. To this end, a laboratory prototype, based on a
Photonfocus Hurricane hyperspectral camera, has been implemented to acquire data in the range of 400-1000
[nm], at a wavelength resolution of 1.04 [nm]. A mobile platform has been also constructed to simulate and
synchronize the scanning procedure of the camera. Finally, an image-processing-based SNC algorithm has been
extended yielding an approach that employs all the multidimensional information collected by the camera.
In this paper a novel nonuniformity correction method that compensates for the fixed-pattern noise (FPN)
in infrared focal-plane array (IRFPA) sensors is developed. The proposed NUC method compensates for the
additive component of the FPN statistically processing the read-out signal using a noise-cancellation system.
The main assumption of the method is that a source of noise correlated to the additive noise of the IRFPA is
available to the system. Under this assumption, a finite impulse response (FIR) filter is designed to synthesize
an estimate of the additive noise. Moreover, exploiting the fact that the assumed source of noise is constant
in time, we derive a simple expression to calculate the estimate of the additive noise. Finally, the estimate
is subtracted to the raw IR imagery to obtain the corrected version of the images. The performance of the
proposed system and its ability to compensate for the FPN are tested with infrared images corrupted by both
real and simulated nonuniformity.
In this paper, a novel color space transform is presented. It is an adaptive transform based on the application of
independent component analysis to the RGB data of an entire color image. The result is a linear and reversible
color space transform that provides three new coordinate axes where the projected data is as much as statistically
independent as possible, and therefore highly uncorrelated. Compared to many non-linear color space transforms
such as the HSV or CIE-Lab, the proposed one has the advantage of being a linear transform from the RGB
color space, much like the XYZ or YIQ. However, its adaptiveness has the drawback of needing an estimate of
the transform matrix for each image, which is sometimes computationally expensive for larger images due to the
common iterative nature of the independent component analysis implementations. Then, an image subsampling
method is also proposed to enhance the novel color space transform speed, efficiency and robustness. The new
color space is used for a large set of test color images, and it is compared to traditional color space transforms,
where we can clearly visualize its vast potential as a promising tool for segmentation purposes for example.
The non-uniform response in infrared focal plane array (IRFPA)
detectors produces corrupted images with a fixed-pattern noise. In
this paper we present an enhanced adaptive scene-based
non-uniformity correction (NUC) technique. The method
simultaneously estimates detector's parameters and performs the
non-uniformity compensation using a neural network approach. In
addition, the proposed method doesn't make any assumption on the
kind or amount of non-uniformity presented on the raw data. The
strength and robustness of the proposed method relies in avoiding
the presence of ghosting artifacts through the use of optimization
techniques in the parameter estimation learning process, such as:
momentum, regularization, and adaptive learning rate. The proposed
method has been tested with video sequences of simulated and real
infrared data taken with an InSb IRFPA, reaching high correction
levels, reducing the fixed pattern noise, decreasing the ghosting,
and obtaining an effective frame by frame adaptive estimation of
each detector's gain and offset.
A Kalman filter is developed to estimate the temporal drift in the gain and the offset of detectors in focal-plane array sensors from scene data. The novelty of this approach is that the gain and the offset are modeled by random sequences (state variables) which must be estimated from the current and past noisy scene data. The gain and the offset are assumed constant over fixed-length blocks of frames; however, these parameters may slowly drift from block to block according to a temporal discrete-time Gauss-Markov process. The input to the Kalman filter consists of a sequence of blocks of frames and the output at any time is a vector containing current estimates of the bias and the offset for each detector. Once these estimates are generated, the true image is restored by means of a least- mean-square error temporal FIR filter. The efficacy of the reported technique is demonstrated by applying it to two sets of real infrared data and the advantage rendered by the Gauss-Markov model is shown.
It is well known that non-uniformity noise in focal-plane array (FPA) sensors, which is due to the pixel-to-pixel variation in the detectors' responses, can considerably degrade the quality of images since it results in a fixed pattern that is superimposed on the true scene. This paper addresses the benefits of non-uniformity correction (NUC) pre- processing on the performance of a number of existing post- processing algorithms. The post-processing applications considered are image registration (motion compensation) and high-resolution image reconstruction from a sequence of blurred and undersampled images. The accuracy of motion estimation in the presence of non-uniformity noise is investigated in terms of the standard deviation of the gain and the offset of the FPA detectors. Two recently reported scene-based NUC techniques are employed to pre-process the FPA output prior post-processing. The first NUC technique is based on the assumption that all pixels experience the same range of irradiance over a sufficiently long sequence of frames (constant-range assumption), and the second NUC technique is registration-based and can be used recursively. It is shown that NUC improves the accuracy of registration, and consequently, enhances the benefit of high-resolution reconstruction. It is also shown that under cases of severe non-uniformity noise, a recursive application of the registration-based NUC may further improve the accuracy of registration and high-resolution image reconstruction. The results are illustrated using real and simulated data.
A statistical model for the focal-plane array (FPA) output is developed characterizing the random nature of nonuniformity in time and space. The rationale of this method is that current and past outputs of the FPA bear information about the nonuniformity. Using a statistical algorithm, this hidden information about the random nonuniformity can be extracted and used to restore the true image. The proposed algorithm consists of two main parts. The first part involves a periodic statistical estimation of the model parameters using current data. The second part involves utilizing the estimated parameters in restoring the true image by means of a least mean square FIR filter whose coefficients remain unchanged between the rounds of parameter estimation. This model-based approach exploits the slow drift of the sensors' offset voltage, gain, and circuit noise in order to reduce the necessary computations to a minimum.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.