2.2.1.

Ray propagation

The incident rays are refracted when hitting an interface of changing RI, e.g., the sample surface. Snell’s law is applied in this simulation. The RIs of the sample and medium can be set to any value between 1 and 99. The slope of the surface at each incident point is determined by polynomial regression with a variable order and variable amount of considered neighbors. The exit angle represents the divergence from the refraction-free beam path. It is calculated with simple trigonometric functions.

The ray is then propagated through the cross section of the input structure. It is assumed that the sample has a constant RI and that the rays are not refracted inside the sample. A bi-linear interpolation of the four closest neighboring pixels of the in-between-pixel position of the ray after it propagated the distance of one pixel is performed repetitively until the ray hits the surface on the backside. The values of these bi-linear interpolations are summed to make up the brightness of the pixel of the incident position of the ray in the projection image. This propagation is repeated for each incident position of each projection angle and results in a sinogram of the input structure.

The deflection at the exit point in the specimen is not considered for this work. The fluorescent light, which is the main information to be measured, is unidirectional and collected by a photo multiplier tube underneath the glass cuvette. The deflection of the illumination beam at the back of the specimen is therefore irrelevant for the fluorescent light. The measurement of the transmitted light is influenced by the deflection at the back end of the specimen. This could lead to a total loss of light when the accumulated deflection is strong enough that no light hits the photo diode, which measures the transmission intensity. This is undesirable and remains a challenge to be overcome. Moving the photo diode closer to the glass cuvette would increase the acceptance angle and reduce the effect of strong deflection on the transmitted light intensity.

2.2.2.

Rearrangement of sinogram

Because the sample is rotated during data acquisition, there is information from every incident angle (projection angle) for each surface position. All of this data is saved in the sinogram of the input structure. When the RI is not matched, this information is simply associated to the wrong position.^15,16 The goal of the rearrangement software is to correct these positioning errors by searching for the matching surface position and projection angle. The ray path through the sample when refraction occurs can be translated into a path without refraction at a different rotation angle of the sample. Fig. 2 visualizes the problem and the proposed solution. By digitally rotating the sample utilizing the OpenCV library,¹⁸ the projection angle is found; the incident position is hit by the ray in a way that the ray travels vertically through the sample at the original projection angle of the sample.

Fig. 2

Refraction during RI mismatched SLOT measurement. Dark blue: immersion medium; light blue: specimen. Left: the ray at entryPoint $P$ is refracted because the RI of the medium does not match the RI of the sample. Because of that, the real beam path (solid red line; RBP) is unequal to the expected beam path (dotted red line; EBP) and subsequently incorrect information is measured. Right: after digitally rotating the sample by $\mathrm{\Delta }\theta$, the RBP at ${\theta }_1$ (arbitrary) is equal to the EBP at ${\theta }_0$. This means that the information that was supposed to be recorded at entryPoint $P$ was actually recorded at $P^{\prime }$ at a different rotation angle (${\theta }_1$). In other words, $\overrightarrow{PK}$ contains the same information as $\overrightarrow{P^{\prime }{K}^{\prime }}$.¹⁷

A preemptive surface is generated by reconstructing the uncorrected transmission measurement. Although the sinogram is potentially distorted and yields no useful information about the inside structure of the specimen, the shape (or surface) of the specimen is always reconstruction correctly. The boundary of the specimen is visible in SLOT in the transmission even when the specimen is optically cleared. The surface roughness of a real specimen is responsible for some absorption or deflection, and this leads to the visibility of the boundary of the cleared specimen. Before a rearrangement of the sinogram is possible, each ray’s refraction angle for each projection angle has to be calculated. This is done by rotating the input surface to each projection angle and performing ray propagation calculations. This information is temporally saved in a multidimensional list called entryPoints. In the next step, $\mathrm{\Delta }\theta$ (compare Fig. 2) is calculated for each incident point and angle with the following equation (the derivation is explained in Eqs. 2(a)–2(d) and in Fig. 3)

Fig. 3

Sketch to show the derivation of the equation to determine how much steeper or flatter the incident angle has to be to get an exit angle as if no refraction occurred. The values are arbitrary and just for illustration.

$\mathrm{\Delta }\theta$ is computed as (compare Fig. 3)

The exit angle is equal to incident angle:

Snell’s Law is used to solve for ${\alpha }_2$ as

Inserting Eqs. (2b) into (2c) and Eqs. (2c) into (2a) results in

The RI of the specimen has to be relatively homogeneous. Refraction inside the specimen cannot be corrected by the presented method. Although this is unfortunate, it should be noted that refraction inside a specimen also reduces the performance of every other optical imaging method. A sufficiently cleared biological specimen (which has to be assumed to have a homogeneous RI distribution) or a technical sample, such as the polydimethylsiloxan (PDMS) waveguides used as an example in this work, is needed for SLOT.

To derive an RI of a specimen that is not exactly known, a method is implemented into the correction software to allow for the correction of a sinogram with different specimen RIs. The user can compare the image quality of the resulting reconstructions to estimate the RI of the specimen. Some preemptive knowledge of the expected structures inside the specimen and user experience is necessary for this.

2.2.3.

Reconstruction

The tomograms (or cross sections) were reconstructed utilizing filtered back projection,^4,19 which is a standard method for the reconstruction of a volume from projections in CT. Mathematically, it is based on the inverse Radon transform, which was published over 100 years ago.²⁰ Fiji,²¹ which is a library for ImageJ,²² and the IMOD²³ library, which is an open source library for tomographic reconstruction, were used in this process. The tool for reconstruction was written specifically for SLOT but works for any set of projections that are acquired in a parallel projection setup.

2.2.4.

Artificial dataset

SLOT requires the sample to be embedded in an optical adhesive to mount the sample on the rotary stage. Although this compound can, in theory, be of any geometrical or random shape, the practice in the lab is standardized. To execute the CRISTAL clearing protocol successfully, the sample and optical adhesive are put in a thin Teflon tube for curing. This results in a cylindrical shaped compound with a constant RI (Fig. 4).

Fig. 4

Input surface (a) and input structure (b) used for the simulated results (Sec. 3). 8-bit gray value format with a resolution of $650\times 688\text{pixels}$ each.¹⁷

A hand drawn shape was used as the digital phantom outline with this fact in mind. But to account for potential changes in radius and curvature of a real compound, the phantom is not perfectly round but rather potato shaped. The phantom is two-dimensional because of reasons discussed in Sec. 2.1. In short, SLOT is not truly three-dimensional but spatial stacking of thin optical cross sections through precise imaging planes. If the outer boundary of the compound in the vertical direction (along the rotary axis) is not close to perpendicular to the incident beam, the presented correction method fails. This is because filtered back projection only works for parallel beam geometries and a precise two-dimensional image plane. If the outer boundary in the vertical direction is not close to ninety degrees and there is an RI mismatch, the laser beam gets refracted away from the precise imaging plane, and this derivation cannot be corrected digitally. Therefore, only a two-dimensional phantom is considered in this work.

The inside of the digital phantom is the logo of the Laser Zentrum Hannover e.V., where bright parts represent a high fluorescence signal. The phantom contains large homogeneous areas as well as very small and precise structures, e.g., the letters in the lower half. The input surface and input structure are illustrated in Fig. 4.

3.1.1.

Qualitative analysis

Figure 6 shows reconstructions of simulations of the digital phantom with different mismatches between the medium and sample. The RI of the medium was 1.4 in all cases. 800 projections were performed for each result. Rows 1 and 3 show the results without correction, and row 2 and 4 show results in which a rearrangement of the sinogram based on ray propagation was performed. The lower two rows are zoomed-in images of the upper ones, indicated by the red square. Column (c) shows the result of reconstruction when no RI mismatch is present; the contrast in these images is the highest of all columns, and the structure is best resolved.

Fig. 6

(1a)–(4e) Comparison of reconstructions of different RI mismatches: the numeric values on top indicate the RI of the sample. The RI of the medium was 1.4 in all cases. Rows 1 and 3 show reconstructions without correction of the sinogram. Rows 2 and 4 show reconstructions with the sinogram corrected for refraction. Rows 3 and 4 display zoomed-in images from rows 1 and 2, respectively. 800 projections over one rotation were simulated for these images.¹⁷

The sample appears smaller in the reconstruction in Figs. 6(1a) and 6(1b) than in the matched case displayed in Fig. 6(1c). This effect is called shrinking in the following. The opposite effect is visible in Figs. 6(1d) and 6(1e), where the inner part of the sample appears enlarged. There is no shrinking or enlargement visible in Fig. 6 row 2, where the sinograms were rearranged prior to reconstruction. A few line-shaped artifacts are visible in Fig. 6(4a), and the correctly reconstructed area is reduced in Figs. 6(2d) and 6(2e). In these cases, information is lost at the edges of the phantom.

3.1.2.

Quantitative analysis

Figure 7 shows the results of APD applied to the images shown in Fig. 6 and additional simulations complementing the shown results with an increment of 0.01 mismatch in RI. The lower the value is, the higher the similarity to the result in which no RI mismatch was present is. The subjective improvement of the result shown in the previous section when the sinogram is rearranged is quantitatively confirmed here. Going back to the qualitative analysis, it can be seen that the corrected images suffer a lower contrast that is homogeneous over the image, meaning the main portion of the APD value results from the lower contrast and not from worse structural resolution. The performance of the rearrangement results in APD would be even greater if the contrast of the images would have been adjusted manually.

Fig. 7

APD analysis of simulated reconstructions. The sample RI is indicated on the horizontal axis. The RI of the medium is 1.4 in all cases. The averaged APD over the respective region of interest is shown. The ground truth image as well as the different regions of interest are depicted in Figs. 4 and 5.¹⁷

In contrast to APD, SSIM accounts more for structural similarity and less for total and regional differences in brightness. The effect of this difference in evaluation is visible in Fig. 8. In the case of a mismatch of 1.4 (medium) to 1.3 (sample), the reconstruction of the corrected sinogram is twice as good as the uncorrected result when only looking the logo region of the sample. The SSIM values of the uncorrected simulations drop below 0.5, which indicates very strong differences in appearance to the phantom image. Although the performance of the uncorrected simulations is very similar for the whole sample and logo region, the performance of the corrected simulations is much better in the logo region.

Fig. 8

SSIM analysis of the same simulations as in Fig. 7. The greater the value is, the closer the evaluated image is to the reference image (perfect RI matched simulation).¹⁷

3.2.1.

Agarose cylinder

Exemplary SLOT measurements of fluorescent beads in an agarose cylinder were performed. Water was used as the immersion medium to illustrate an RI matched case. A silicone oil with an RI of 1.422 was used to showcase the issues of a poorly RI matched measurement and to test the capabilities of the correction algorithm in a real application.

Figure 9 shows the results of these measurements. The matched measurement shown in Fig. 9(a) displays spherical beads, which is the expected outcome. The mismatched measurement Fig. 9(b) shows strong artifacts, especially at the outer parts of the cylinder. The spherical shape of the beads is completely lost. Only the center part of the image is reasonably close to the matched case shown in Fig. 9(a). The measurement with the sinograms that were rearranged in post processing, shown in Fig. 9(c), is very close to the matched measurement. The fluorescent beads have a spherical shape, and the image quality is comparable to Fig. 9(a). The intensity of the fluorescence signal is falling off a bit at the outer parts of the image.

Fig. 9

SLOT measurements of an agarose block with fluorescent polystyrene beads with different RI-matching configurations. All images show the average of 100 cross sections. The intensity of fluorescence was color-coded with the “fire” look-up table (ImageJ) for images (a)–(d). (a) Water was used as the immersion medium, which has a negligible RI difference compared to the agarose cylinder. (b) A silicone oil with an RI of 1.422 was used. (c) The measurement of (b) was corrected using the presented rearrangement method. (d) The same image as (a). Panels (e) and (f) have a different color-code: they are composites in which the results of the matched measurement are shown in red, and the mismatched measurement is shown in green. (e) Composite of the matched (red) and mismatched (green) measurement. (f) Composite of the matched (red) and the corrected (green) mismatched measurement.

A different visualization of the same data is shown in the bottom row. The composites make it easier to compare the results. The effect of shrinking is much more visible in Fig. 9(e) than in Fig. 9(b). Figure 9(b) shows the composite of the matched Fig. 9(d) in red and rearranged Fig. 9(f) measurement in green. It is evident that the correction algorithm performs very well for the center part of the image, where almost every bead appears yellow, which means that the images are close to equal. The outer part is more reddish, which indicates a lower intensity in the corrected image. This is also visible in Fig. 9(c). A few completely red or green beads are visible. This is due to the cross sections not being perfectly set at the same heights for the matched and mismatched measurements.

The averaged cross sections shown in Fig. 9(a) were used as a phantom to cross-validate the results of the mismatched measurement. The result is shown in Fig. 10. The measurement and simulation match well. The same kind of artifacts appear in both images. The beads, which are not directly in the center of the sample, appear in a star-like shape. The drop-off in intensity toward the edge of the sample that was visible in Fig. 9(c) is also visible in this figure.

Fig. 10

Comparison of measured (a) and simulated (b) SLOT images with an RI mismatch of 0.09. The rearranged, summed cross sections shown in Fig. 9(a) were used as the simulation phantom for (b). The simulation and the real measurement match well and show the same kind of image artifacts, which is the star-like smearing of the supposedly spherical beads.

3.2.2.

Polydimethylsiloxan cylinder

A second validation experiment was performed on a PDMS cylinder with fluorescent nanoparticles. The purpose of this experiment was to show the inverse case of RI mismatch in which the RI of the sample is higher than the medium’s RI. The results of this experiment are shown in Fig. 11. Two different imaging planes of the same specimen are shown. The comparison of the matched and rearranged images show that the correction algorithm works fairly well for this case of RI mismatch. The contrast of the rearranged images is lower, and the nanoparticles appear less sharp. The drop-off in intensity when an RI mismatch is present, which was observable in the agarose cylinder measurements, is also visible in this case.

Fig. 11

(a)–(d) RI matched and mismatched measurements of a PDMS waveguide with evenly distributed fluorescent nanoparticles. The images of each row show the same part of the sample. The images are averages of multiple image planes. The general appearance is similar for matched and mismatched measurements. The contrast is lower for the mismatched images.

Not every fluorescent sphere appears in both measurements at the same position. This is because the specimen interacted with the silicone oil that was used as the immersion medium. The specimen grew in size, but the growth was not homogeneous. The geometry of the specimen changed, and the arrangement of the fluorescent spheres shifted unpredictably.