1.

Introduction

Cell function correlates strongly with structure or morphology. For example, cellular activities such as differentiation and apoptosis can be distinguished by changes in morphology. Consequently, morphological features provide powerful and often critical markers for cell assay and clinical applications.^1,2 Because of the complexity in cell morphology, conventional microscopy methods have to be used, which are labor-intensive for data acquisition and analysis. Therefore, it is highly desired to develop an automated method for rapid assay of large populations of single cells about their morphology with flow cytometry technology. Conventional and commercially available flow cytometers, however, are designed to acquire fluorescence signals or images as the major source of information for rapid cell assay. This approach, however, provides very limited information on cell morphology. The three-dimensional (3-D) morphology and molecular polarization information of single cells have been shown to correlate strongly with the spatial distribution of scattered light from the cell illuminated by highly coherent light.^3–8 We have previously developed a flow cytometry method for rapid acquisition of cross-polarized diffraction image pairs with two cameras.^9–13 The new method, termed as polarization diffraction imaging flow cytometry (p-DIFC), has been shown to be capable of distinguishing cell lines of high similarity in their 3-D morphology.¹⁰

Despite the capacity for morphology-based cell assay, a significant problem still exists that affects performance of the p-DIFC method using conventional CCD cameras. The images of the cells are acquired under the flow condition, which could markedly reduce the image contrast by motion blur (MB) with these cameras. The image blurring makes it harder to perform accurate image analysis and cell assay. A time-delay-integration (TDI) imaging method can be applied to control and reduce the MB. A TDI camera has multiple lines of pixels whose charges are transferred at a speed controlled with an external clock pulse-train. If the charge transfer speed of pixel lines is equal to the speed of the moving image, MB can be eliminated.¹⁴ A TDI camera has been developed and used to investigate the effect of MB on characteristic parameters extracted from cross-polarized diffraction images of microspheres and biological cells using different image-processing algorithms.¹⁵

In this report, we focus our attention on the quantitative analysis of the MB in measured diffraction images in the frequency domain with the modulation transfer function (MTF). The MTF-based analysis enables us to validate a tool for numerically “blurring” diffraction images acquired with the matched speeds between line transfer and moving image in p-DIFC measurements, and investigate the effect of MB on pattern analysis of the images from the same cell. We have also proposed a single parameter of frequency modulus for quantification of MB based on the Fourier transform of the measured diffraction images. The MB analysis was carried out on measured diffraction images acquired with MCF-7 and K562 cells moving at seven different speeds. These results provide insights into the effect and evaluation of MB to further improve the p-DIFC method with the TDI camera, and allow the use of the average frequency modulus to assess MB quantitatively.

2.

Methods

2.1.

Motion Blur Assessment

We designed and developed a TDI-camera-based imaging system for variation and elimination of MB in the acquired diffraction images from the moving cells.¹⁵ Quantitative assessment of MB is needed to characterize blurring in diffraction images. Often, a spatial domain image analysis approach is used for blur assessment in conventional image data. But the diffraction images concerned here carry light scattering noise, and the frequency domain approach with MTF becomes more suitable for MB assessment.

MB could occur when the object moves relative to the imaging sensor. In this study, the diffraction images from the moving object are acquired at off-focus position to improve image contrast.^16,17 A diffraction image $g(z,y)$ acquired at an off-focus position with MB can be described in frequency domain by its spatial frequency spectrum $G(u,w)$ as

Eq. (1)

$$G(u,w)=F(u,w)·{\mathrm{OTF}}_f(u,w)·{\mathrm{OTF}}_m(u,w)+N(u,w),$$

where $F(u,w)$ and $N(u,w)$ are, respectively, the spatial frequency spectrum of acquired image without blur and electronic noise; ${\mathrm{OTF}}_f(u,w)$ and ${\mathrm{OTF}}_m(u,w)$ are, respectively, the optical transfer function of the imaging system translated to off-focus positions and that with MB, both are recognized as the causes of reduction in high-frequency information of the acquired images in comparison to the images acquired at the focusing position and without MB. We have previously shown that for coherent imaging, ${\mathrm{OTF}}_f$ actually increases image contrast and content of information carried by the diffraction images.¹⁷ Our purpose here is to analyze the effect of MB given by the modulus of ${\mathrm{OTF}}_m$ from the measured diffraction images and quantify MB with a TDI camera as the imaging sensor.

The degree of MB can be determined from the distance and orientation of the image’s movement on the image sensor during an exposure time $T$. For our study, the cell and its image on the camera sensor move along the $y$-direction (see Fig. 2), while the incident beam propagates along the $z$-axis. Therefore, the degraded image $g(z,y)$ in terms of incident light energy can be expressed as integration of a series of light rays shifting through different positions of $\mathrm{\Delta }y(t)$ or

Eq. (2)

$$g(z,y)={\int }_0^{T}P[z,y+\mathrm{\Delta }y(t)]\mathrm{d}t,$$

where $z$ and $y$ are integers representing a pixel location, $P$ is the distribution of incident light power on the sensor, and $\mathrm{\Delta }y(t)$ describes the shift of a moving “light ray” contributing to the pixel intensity $g(z,y)$ at time $t$. Fourier transform of $g(z,y)$ yields the following:

Eq. (3)

$$\begin{gathered} G(u,w)=\frac{1}{MN}\sum _{z=1}^M\sum _{y=1}^{N}g(z,y)e^{-j2\pi (\frac{uz}{M}+\frac{wy}{N})} \\ =\frac{1}{MN}\sum _{z=1}^M\sum _{y=1}^N[{\int }_0^{T}P[z,y+\mathrm{\Delta }y(t)]\mathrm{d}t]e^{-j2\pi (\frac{uz}{M}+\frac{wy}{N})} \\ =F(u,w)\frac{1}{T}{\int }_0^T{e}^{-j2\pi \frac{w\mathrm{\Delta }y(t)}{N}}\mathrm{d}t, \end{gathered}$$

where $M$ and $N$ are the number of pixels in $z$- and $y$-direction, respectively; $u$ and $w$ are the corresponding component of spatial frequency; and $F(u,w)$ is the Fourier transform of $P(z,y)T$ or the image $g(z,y)$ without MB as given in Eq. (2). Given $d$ as the pixel size, the pixel shift $\mathrm{\Delta }y(t)$ is given by

Eq. (4)

$$\mathrm{\Delta }y(t)=\{\begin{array}{ll}\frac{V_{i}t}{d},& \text{for}\mathrm{CCD}\\ \frac{\mathrm{\Delta }Vt}{d}=\frac{\mathrm{\Delta }Vt}{T{V}_{\mathrm{ch}}/S},& \text{for}\mathrm{TDI}\end{array},$$

where $V_i$ is the image’s moving speed on a conventional CCD sensor and $\mathrm{\Delta }V$ is the difference between the line charge transfer speed $V_{ch}=df$ and $V_i$, if a TDI sensor is used instead with $f$ as the line transfer frequency and $S$ as the total line number of the TDI sensor. Comparison of Eqs. (1) and (3) provides the modulation transfer function ${\mathrm{MTF}}_m(u,w)$ for a blurred image as^18–20

Eq. (5)

$${\mathrm{MTF}}_m(u,w)=|{\mathrm{OTF}}_m|=\left|\frac{1}{T}{\int }_0^T{e}^{-j2\pi \frac{w\mathrm{\Delta }y(t)}{N}}\mathrm{d}t\right|=\{\begin{array}{ll}\sin c\left(\frac{\pi w}{Nd}T{V}_i\right),& \text{for}\mathrm{CCD}\\ \sin c\left(\frac{\pi wS}{N}\frac{\mathrm{\Delta }V}{V_{\mathrm{ch}}}\right),& \text{for}\mathrm{TDI}\end{array}.$$

For this study, we denote $\mathrm{\Delta }V/V_{ch}$ as the speed-mismatch ratio. From Eq. (2) we can verify that at synchronization, the speed difference $\mathrm{\Delta }V$ for a TDI camera becomes zero and the MB is eliminated with ${\mathrm{MTF}}_m=1$. Figure 1 presents four cases of ${\mathrm{MTF}}_m$ calculated by Eq. (5) for imaging systems with TDI cameras and different mismatch ratios, and a comparison to a system with a conventional CCD camera of the same image speed and equivalent exposure time. These results show quantitatively that the TDI camera can reduce and eliminate MB effectively and produce images of larger bandwidths than those acquired with conventional CCD cameras.

Fig. 1

${\mathrm{MTF}}_m$ of imaging systems using TDI camera with different values of speed-mismatch ratio and of an imaging system using a conventional CCD camera with equivalent exposure time of $T=2\mathrm{ms}$ and image speed of $V_i=200\mathrm{mm}/\mathrm{s}$.

Figure 1 shows that the MB can be eliminated by synchronization at $\mathrm{\Delta }V/V_{\mathrm{ch}}=0$ with a TDI camera, and the ${\mathrm{MTF}}_m$ remains maximized at 1 for all frequencies. Once the mismatch ratio starts to increase, MB occurs with the values of ${\mathrm{MTF}}_m$ markedly reduced in the frequency domain. We can quantify the frequency decrease of a blurred image to evaluation of the degree of MB by an averaged modulus of its Fourier transform ${|G|}_{\mathrm{av}}$, defined as

Eq. (6)

$${|G|}_{\mathrm{av}}=\frac{\sum _{i=0}^{M-1}\sum _{i=0}^{N-1}|G(u_i,w_i)|}{MN},$$

which is proportional to ${\mathrm{MTF}}_m$.

2.2.

Polarization Diffraction Imaging Flow Cytometry System With One Time-Delay-Integration Camera

Figure 2 shows the p-DIFC system constructed for this study based on previous research.^10,12,21 A cw laser of 532 nm in wavelength (MGL-III-532-100 mW) is focused by two cylindrical lenses on the core fluid in an elliptical spot with major diameter of about 300 μm along the $y$-axis and minor diameter of about $50\mu \mathrm{m}$, and the polarization and power of the incident beam are controlled by one half-wavelength plate and one Glan prism. The coherent side-scatter by the illuminated cells is collected by an infinity-corrected, nontelecentric $50\times$ objective lens (378-805-3, Mitutoyo), followed by a Wollaston prism (LSP-3A14, Laser Institute, QFNU) of $14\mathrm{mm}\times 14\mathrm{mm}$ aperture. The p-polarized and s-polarized scattered light after the Wollaston prism, separated by 10 deg in directions, are focused with a tube lens of focal length $F=75\mathrm{mm}$ onto the TDI-CCD sensor (S10201-4 Hamamatsu). The imaging system can be translated along the $x$-axis toward the flow chamber from the focused position of $\mathrm{\Delta }x=0$ at which the core fluid is on the object plane of the imaging system.

Fig. 2

Schematic of the p-DIFC system: HWP, half-wave plate; $\mathrm{GP}=\text{Glan prism}$; M, mirrors; CL, cylindrical lens; FC, flow chamber; OBJ, objective lens; WP, Wollaston prism; IF, 532 nm interference filter; TL, tube lens. The green arrow and shaded lines indicate the incident laser beam and scattered light of s- or p-polarization.

Different from conventional CCD cameras, the TDI cameras allow synchronization between the cell speed and line-pixel transfer frequency as discussed above. To determine cell speed, we measured the flow rate of the sheath fluid, which is proportional to the cell speed ²² and can be controlled by the sheath fluid pressure. The line-pixel transfer frequency $f$ of the TDI camera was set to be 16.67 KHz with an in-house developed software. Given the number of pixel lines $S$ of the TDI-CCD sensor as 128 and the sensor height $L_{\text{tdi}}$ as 1.536 mm, the synchronous image speed $V_{\mathrm{i}\_\mathrm{cal}}$ can be calculated from

Eq. (7)

$$V_{\mathrm{i}\_\mathrm{cal}}=\frac{L_{\mathrm{tdi}}f}{S},$$

which yields $V_{\mathrm{i}\_\mathrm{cal}}=200\mathrm{mm}/\mathrm{s}$.

We used polystyrene microspheres of $1\mu \mathrm{m}$ diameter and a conventional CCD camera (Lumenera Lt225) of a 6-mm height ($L_{\mathrm{t}}$) sensor for flow-speed measurement. The shadow lengths $L_{\mathrm{s}}$ of the imaged microspheres, proportional to the exposure time, were used to determine the speed of microspheres. The exposure time $T$ of the conventional camera was set to 3 ms, so the image speed $V_{\mathrm{i}\_\mathrm{meas}}$ can be determined by

Eq. (8)

$$V_{\mathrm{i}\_\mathrm{meas}}=\frac{L_{\mathrm{s}}}{T}.$$

For the $50\times$ objective and tube lens of focal length $F=75\mathrm{mm}$, the magnification of the imaging system ($M_{\mathrm{i}}$) is given by 18.75, and the microspheres’ speed or core-fluid speed is given by

Eq. (9)

$$V_{\mathrm{o}}=V_{\mathrm{i}}/M_{\mathrm{i}}.$$

After sample injection into the core-fluid reservoir, the particle or the core-fluid speed $V_0$ was adjusted by the sheath fluid pressure and measured by the shadow lengths. For three different speed settings, five measurements were performed to obtain the averaged core-fluid speed with an interval of 20 min between measurements. The averaged values and standard deviations show that the maximum relative change is 2%, which indicates that the flow speed was stably controlled.

3.

Results and Discussion

The MB similarly affects the different polarized diffraction images. The raw diffraction images from the TDI camera were cropped to obtain $200\times 200$ pixels near the maximum pixel of each polarized diffraction image for analysis. Figures 3(a) and 3(b) present examples of p-polarized diffraction images $g(z,y)$ acquired at three different values of core-fluid speed $V_0$ with one TDI camera and similar images with a conventional CCD camera of exposure time $T=0.3\mathrm{ms}$ and $V_0=10.6\mathrm{mm}/\mathrm{s}$. It is clear that the diffraction images acquired with the TDI camera at $V_0=6.8$ or $9.4\mathrm{mm}/\mathrm{s}$ show severe degradation due to MB, which is also the case for the one acquired with a conventional CCD camera. In contrast, the images acquired with the TDI camera at the zero speed-mismatch ratio of $V_0=10.6\mathrm{mm}/\mathrm{s}$ display minimal blur. Figure 3(c) further presents two diffraction images with numerically simulated MB using different values of $m$ for pixel numbers of window-smoothing along the vertical direction on the TDI-acquired image shown in Fig. 3(a). One can observe that the two related images in Figs. 3(b) and 3(c) with $V_0=9.4\mathrm{mm}/\mathrm{s}$ versus $m=15$ pixels or $6.8\mathrm{mm}/\mathrm{s}$ versus 48 pixels present similar MB even though they were acquired from different cells.

Fig. 3

The p-polarized diffraction images $g(z,y)$ of single K562 (top row) or MCF-7 (bottom row) cells of $200\times 200$ pixels: (a) measured with TDI of $f=16.67\mathrm{kHz}$ or conventional CCD camera at $V_0=10.6\mathrm{mm}/\mathrm{s}$; (b) measured with the same TDI camera; (c) simulated MB on image acquired with the TDI camera in (a). The labels at the lower left corners are values of $V_0$ in $\mathrm{mm}/\mathrm{s}$ in (a) and (b) or numbers of smoothed pixels $m$ in (c) for images in both rows.

To analyze the p-polarized diffraction images $g(z,y)$ in the frequency domain, 2-D Fourier transforms were performed on $g(z,y)$ to obtain $|G(u,w)|$, and the results of $|G(u,w)|$ normalized by the central peak or DC component are shown in Fig. 4 with the vertical $w$-axis corresponding to the vertical $y$-axis of motion in $g(z,y)$ images of Fig. 3. Since the Fourier components are concentrated in the low-frequency range or the central region, the spectral images in Fig. 4 present only the middle part of the 2-D frequency image $|G(u,w)|$ with $100\times 100$ pixels and values of central peaks as 255. Comparison of the images in Figs. 4(a) and 4(b) indicates clearly that MB reduces the frequency information along the $w$-axis in frequency domain and agrees well with the MTF analysis. The results of frequency analysis of the p-DIFC image data acquired with the TDI camera can also be compared with those acquired with the conventional CCD camera in Fig. 4(a), which shows that larger frequency band or information can be obtained with the TDI camera than with the conventional CCD. Furthermore, comparison of the spectral images in Figs. 4(b) and 4(c) proves that the simulated MB on diffraction images acquired with $\mathrm{\Delta }V=0$ produces distributions in frequency space similar to those in measured images with $\mathrm{\Delta }V\ne 0$. It should be noted that the similarity is achieved with the images acquired from different cells of K562 or MCF-7. These results validate a very useful tool to accurately model the effect of MB on image patterns and cell classification with polarized diffraction images acquired with a TDI camera. A study is underway with this tool to investigate the effects of MB on GLCM parameters extracted from the diffraction images of cells acquired under the condition of zero speed-mismatch.

Fig. 4

The central regions of normalized Fourier transform modulus images $|G(u,w)|$ of $100\times 100$ pixels obtained by from the diffraction images $g(z,y)$ in Fig. 3 with identical labels. The values of $|G(u,w)|$ in the images are indicated in color as the legends present.

In addition to the modulus image $|G(u,w)|$, one can also quantify MB by a single parameter of averaged frequency modulus ${|G|}_{\mathrm{av}}$ defined in Eq. (6). The mean values and standard deviations of ${|G|}_{\mathrm{av}}$ were obtained from five randomly selected p-polarized images $g(z,y)$ of $200\times 200$ pixels at different values of core-fluid speed $V_0$ and are plotted in Fig. 5(a). In the same figure, we also plot the same modulus parameter versus core flow speed from the images obtained with simulated MB performed on those images acquired with zero speed-mismatch by window-smoothing over $m$ pixels. It can be seen that both the mean values and standard deviations of spectral amplitude reduce with the increased speed-mismatch, which significantly affects the accuracy of pattern analysis for cell assay and classification. The high similarity between the two sets of curves of diffraction images acquired with a TDI camera at different core flow speed and those by simulated MB shows further that the simulated MB can be used for analysis of MB using only the diffraction images acquired with zero speed-mismatch. We note that the conventional blur evaluation parameter of averaged gradient ${|\delta g|}_{\mathrm{av}}$ does not perform well in the case of diffraction images $g(z,y)$ presented in this study. The parameter ${|\delta g|}_{\mathrm{av}}$ is typically defined by the following for a given image $g(z,y)$ along the direction of MB or $y$-axis¹⁸

Fig. 5

The dependence of (a) averaged modulus ${|G|}_{\mathrm{av}}$ of diffraction images acquired by TDI camera or of simulated blur and (b) averaged gradient ${|\delta g|}_{\mathrm{av}}$ of acquired diffraction image $g(z,y)$ on core-fluid speed $V_0$. The symbols and error bars represent the mean values and standard deviations obtained from five randomly selected diffraction images and the lines are for visual guide. Inset: The number of pixels $m$ for simulated blur on the images acquired with $V_0=10.6\mathrm{mm}/\mathrm{s}$ versus $V_0$.

Finally, it is interesting to examine whether the MB can be effectively removed by deconvolution. For this purpose, a MATLAB^® function (deconvwnr) was applied to the measured diffraction images to perform Wiener-filter-based deconvolution. The deconvolution function requires appropriate selection of three parameters of pixel-smoothing length $m$, direction of motion in degree from the horizontal direction of the blurred image, and value of $S_n/S_f$ as the noise-signal-power-ratio (NSPR) to achieve best effect of deblur. The optimized results of deconvolution are shown in Fig. 6 by applying the MATLAB^® function on the blurred images acquired with a flow speed of $V_0=9.4$ and $6.8\mathrm{mm}/\mathrm{s}$ as presented in Fig. 3(b). One can clearly see that even with the optimized parameters of deconvolution, the MB cannot be fully removed if the speed-mismatch becomes large. Furthermore, the deconvolution calculations are computationally expensive and are difficult to apply for rapid image analysis required in p-DIFC method.

Fig. 6

(a) The deconvoluted images by performing Wiener-filtering deconvolution on the blurred images shown in Fig. 3(b) with two K562 cells (top row) and two MCF-7 cells (bottom row); (b) central regions of $|G(u,w)|$ with $100\times 100$ pixels of the deblurred images in (a) with the values of $|G(u,w)|$ indicated in color as the legends present. The values of $m$, direction, and NSPR are labeled on each image.

4.

Conclusion

A diffraction imaging method and system with one TDI camera has been developed for acquisition of diffraction images of flowing cells. We performed MTF analysis of the measured diffraction images from two different cancer cell lines and validated a numerical method for the simulation of motion-blurring effects on images acquired with zero speed-mismatch. These results show that the MTF analysis provides an accurate approach for characterization of the MB effect on diffraction images and significant insights on improvement of image analysis by eliminating blur with a TDI camera. We further show that the averaged modulus of a diffraction image in the frequency domain yields a much improved single parameter than does the averaged gradient calculated directly from the diffraction image for the quantification of MB.