Significance: Surgery is often paramount in the management of many solid organ malignancies because optimal resection is a major factor in disease-specific survival. Cancer surgery has multiple challenges including localizing small lesions, ensuring negative surgical margins around a tumor, adequately staging patients by discriminating positive lymph nodes, and identifying potential synchronous cancers. Intraoperative molecular imaging (IMI) is an emerging potential tool proposed to address these issues. IMI is the process of injecting patients with fluorescent-targeted contrast agents that highlight cancer cells prior to surgery. Over the last 5 to 7 years, enormous progress has been achieved in tracer development, near-infrared camera approvals, and clinical trials. Therefore, a second biennial conference was organized at the University of Pennsylvania to gather surgical oncologists, scientists, and experts to discuss new investigative findings in the field. Our review summarizes the discussions from the conference and highlights findings in various clinical and scientific trials.

Aim: Recent advances in IMI were presented, and the importance of each clinical trial for surgical oncology was critically assessed. A major focus was to elaborate on the clinical endpoints that were being utilized in IMI trials to advance the respective surgical subspecialties.

Approach: Principal investigators presenting at the Perelman School of Medicine Abramson Cancer Center’s second clinical trials update on IMI were selected to discuss their clinical trials and endpoints.

Results: Multiple phase III, II, and I trials were discussed during the conference. Since the approval of 5-ALA for commercial use in neurosurgical malignancies, multiple tracers and devices have been developed to address common challenges faced by cancer surgeons across numerous specialties. Discussants also presented tracers that are being developed for delineation of normal anatomic structures that can serve as an adjunct during surgical procedures.

Conclusions: IMI is increasingly being recognized as an improvement to standard oncologic surgical resections and will likely advance the art of cancer surgery in the coming years. The endpoints in each individual surgical subspecialty are varied depending on how IMI helps each specialty solve their clinical challenges.

Guest editors Behrouz Shabestri, Mark Anastasio, Baowei Fei, and Frédéric Leblond provide an overview of the JBO Special Series on Artificial Intelligence Machine Learning in Biomedical Optics.

Significance: Physiological parameters extracted from diffuse reflectance spectroscopy (DRS) provide clinicians quantitative information about tissue that helps aid in diagnosis. There is a great need for an accurate and cost-effective method for extracting parameters from DRS measurements.

Aim: The aim is to explore the accuracy and speed of physiological parameter extraction using machine learning models compared to that of the widely used Monte Carlo lookup table (MCLUT) inverse model.

Approach: Diffuse reflectance spectra were simulated using a light transport model based on Monte Carlo simulations and weighted to six wavelengths. Deep learning (DL), random forest (RF), gradient boosting machine (GBM), and generalized linear model (GLM) machine learning models were built using a training set of 10,000 spectra from the simulated data. The MCLUT and machine learning models were used to predict physiological parameters from a separate test set of 30,000 simulated spectra. Mean absolute errors were calculated to evaluate the accuracy and compare it among MCLUT and machine learning models. In addition, the computational time to predict parameters from the test set was recorded to compare the speed among MCLUT and machine learning models.

Results: The DL, RF, GBM, and GLM models all had significantly lower errors than the MCLUT inverse method for six wavelengths. The DL model proved to have the lowest errors, with all absolute percent errors under 10%. The DL model had much faster runtimes than the MCLUT.

Conclusions: Machine learning is promising for extracting physiological parameters from six-wavelength DRS data, with both lower errors and a faster runtime than the widely used MCLUT model.

Significance: Automated understanding of human embryonic stem cell (hESC) videos is essential for the quantified analysis and classification of various states of hESCs and their health for diverse applications in regenerative medicine.

Aim: This paper aims to develop an ensemble method and bagging of deep learning classifiers as a model for hESC classification on a video dataset collected using a phase contrast microscope.

Approach: The paper describes a deep learning-based random network (RandNet) with an autoencoded feature extractor for the classification of hESCs into six different classes, namely, (1) cell clusters, (2) debris, (3) unattached cells, (4) attached cells, (5) dynamically blebbing cells, and (6) apoptotically blebbing cells. The approach uses unlabeled data to pre-train the autoencoder network and fine-tunes it using the available annotated data.

Results: The proposed approach achieves a classification accuracy of 97.23  ±  0.94  %   and outperforms the state-of-the-art methods. Additionally, the approach has a very low training cost compared with the other deep-learning-based approaches, and it can be used as a tool for annotating new videos, saving enormous hours of manual labor.

Conclusions: RandNet is an efficient and effective method that uses a combination of subnetworks trained using both labeled and unlabeled data to classify hESC images.

Significance: Optical scattering signals obtained from tissue constituents contain a wealth of structural information. Conventional intensity features, however, are mostly dictated by the overall morphology and mean refractive index of these constituents, making it very difficult to exclusively sense internal refractive index fluctuations.

Aim: We perform a systematic analysis to elucidate how changes in internal refractive index profile of cell nuclei can best be detected via optical scattering.

Approach: We construct stochastically inhomogeneous nuclear models and numerically simulate their azimuth-resolved scattering patterns. We then process these two-dimensional patterns with the goal of identifying features that directly point to subnuclear structure.

Results: Azimuth-dependent intensity variations over the side scattering range provide significant insights into subnuclear refractive index profile. A particular feature we refer to as contrast ratio is observed to be highly sensitive to the length scale and extent of refractive index fluctuations; further, this feature is not susceptible to changes in the overall size and mean refractive index of nuclei, thereby allowing for selective tracking of subnuclear structure that can be linked to chromatin distribution.

Conclusions: Our analysis will potentially pave the way for scattering-based assessment of chromatin reorganization that is considered to be a key hallmark of precancer progression.

Significance: Real-time information about oxygen delivery to the hepatic graft is important to direct care and diagnose vascular compromise in the immediate post-transplant period.

Aim: The current study was designed to determine the utility of visible diffuse reflectance spectroscopy (vis-DRS) for measuring liver tissue saturation in vivo.

Approach: A custom-built vis-DRS probe was calibrated using phantoms with hemoglobin (Hb) and polystyrene microspheres. Ex vivo (extracorporeal circulation) and in vivo protocols were used in a swine model (n  =  15) with validation via blood gas analysis.

Results: In vivo absorption and scattering measured by vis-DRS with and without biliverdin correction correlated closely between analyses. Lin’s concordance correlation coefficients are 0.991 for μ_a and 0.959 for μs^′. Hb measured by blood test and vis-DRS with (R²  =  0.81) and without (R²  =  0.85) biliverdin correction were compared. Vis-DRS data obtained from the ex vivo protocol plotted against the PO₂ derived from blood gas analysis showed a good fit for a Hill coefficient of 1.67 and P₅₀  =  34  mmHg (R²  =  0.81). A conversion formula was developed to account for the systematic deviation, which resulted in a goodness-of-fit R²  =  0.76 with the expected oxygen dissociation curve.

Conclusions: We show that vis-DRS allows for real-time measurement of liver tissue saturation, an indicator for liver perfusion and oxygen delivery.

Significance: High density diffuse optical tomography (HD-DOT) as applied in functional near-infrared spectroscopy (fNIRS) is largely limited to continuous wave (CW) data. Using a single modulation frequency, frequency domain (FD) HD-DOT has recently demonstrated better localization of focal activation as compared to CW data. We show that combining CW and FD measurements and multiple modulation frequencies increases imaging performance in fNIRS.

Aim: We evaluate the benefits of multiple modulation frequencies, combining different frequencies as well as CW data in fNIRS HD-DOT.

Approach: A layered model was used, with activation occurring within a cortex layer. CW and FD measurements were simulated at 78, 141, and 203 MHz with and without noise. The localization error, full width half maximum, and effective resolution were evaluated.

Results: Across the average of the three metrics, at 141 MHz, FD performed 8.4% better than CW, and the combination of CW and FD was 21.7% better than CW. FD measurements at 203 MHz performed 5% better than 78 MHz. Moreover, the three combined modulation frequencies of FD and CW performed up to 3.92% better than 141 MHz alone.

Conclusions: We show that combining CW and FD measurements offers better performance than FD alone, with higher modulation frequencies increasing accuracy. Combining CW and FD measurements at multiple modulation frequencies yields the best overall performance.

Significance: Imaging of the spinal cord is challenging due to the surrounding bony anatomy, physiologic motion, and the small diameter of the spinal cord. This precludes the use of non-invasive imaging techniques in assessing structural changes related to trauma and evaluating residual function.

Aim: The purpose of our research was to apply endovascular technology and techniques and construct a preclinical animal model of intrathecal spinal cord imaging using optical coherence tomography (OCT).

Approach: Five animals (2 Yorkshire Swine and 3 New Zealand Rabbits) were utilized. Intrathecal access was gained using a 16-guage Tuohy, and an OCT catheter was advanced under roadmap technique into the cervical canal. The OCT catheter has a motorized pullback, and a total length of 54 mm of the spinal canal is imaged.

Results: Image acquisition was successful for all animals. There were no instances of difficult catheter navigation, enabling OCT imaging rostrally to C2. The thecal sac provided excellent thoroughfare for the OCT catheter. The clear cerebrospinal fluid also provided an excellent medium for image acquisition, with no detectable artifact from the contents of the cerebrospinal fluid. The anatomical space of the spinal canal could be readily appreciated including: dural lining of the thecal sac, epidural veins, pial lining of the spinal cord, arachnoid bands, dentate ligaments, and nerve rootlets/roots.

Conclusion: Minimally invasive intrathecal imaging using endovascular OCT was feasible in this preclinical animal study. The repurposing of an endovascular device for spinal imaging comes with limitations, and a spine-specific device is necessary.

Significance: Despite its advantages in terms of safety, low cost, and portability, functional near-infrared spectroscopy applications can be challenging due to substantial signal contamination from hemodynamics in the extracerebral layer (ECL). Time-resolved near-infrared spectroscopy (tr NIRS) can improve sensitivity to brain activity but contamination from the ECL remains an issue. This study demonstrates how brain signal isolation can be further improved by applying regression analysis to tr data acquired at a single source–detector distance.

Aim: To investigate if regression analysis can be applied to single-channel trNIRS data to further isolate the brain and reduce signal contamination from the ECL.

Approach: Appropriate regressors for trNIRS were selected based on simulations, and performance was evaluated by applying the regression technique to oxygenation responses recording during hypercapnia and functional activation.

Results: Compared to current methods of enhancing depth sensitivity for trNIRS (i.e., higher statistical moments and late gates), incorporating regression analysis using a signal sensitive to the ECL significantly improved the extraction of cerebral oxygenation signals. In addition, this study demonstrated that regression could be applied to trNIRS data from a single detector using the early arriving photons to capture hemodynamic changes in the ECL.

Conclusion: Applying regression analysis to trNIRS metrics with different depth sensitivities improves the characterization of cerebral oxygenation signals.

Significance: Intraoperative parameters of renal cortical microperfusion (RCM) have been associated with postoperative ischemia/reperfusion injury. Laser speckle contrast imaging (LSCI) could provide valuable information in this regard with the advantage over the current standard of care of being a non-contact and full-field imaging technique.

Aim: Our study aims to validate the use of LSCI for the visualization of RCM on ex vivo perfused human-sized porcine kidneys in various models of hemodynamic changes.

Approach: A comparison was made between three renal perfusion measures: LSCI, the total arterial renal blood flow (RBF), and sidestream dark-field (SDF) imaging in different settings of ischemia/reperfusion.

Results: LSCI showed a good correlation with RBF for the reperfusion experiment (0.94  ±  0.02; p  <  0.0001) and short- and long-lasting local ischemia (0.90  ±  0.03; p  <  0.0001 and 0.81  ±  0.08; p  <  0.0001, respectively). The correlation decreased for low flow situations due to RBF redistribution. The correlation between LSCI and SDF (0.81  ±  0.10; p  <  0.0001) showed superiority over RBF (0.54  ±  0.22; p  <  0.0001).

Conclusions: LSCI is capable of imaging RCM with high spatial and temporal resolutions. It can instantaneously detect local perfusion deficits, which is not possible with the current standard of care. Further development of LSCI in transplant surgery could help with clinical decision making.

Significance: Longitudinal tracking of hemodynamic changes in the breast has shown potential for neoadjuvant chemotherapy (NAC) outcome prediction. Spatial frequency domain imaging (SFDI) could be suitable for frequent monitoring of shallow breast tumors, but strong sensitivity to superficial absorbers presents a challenge.

Aim: We investigated the efficacy of a two-layer SFDI inverse model that accounts for varying melanin concentration in the skin to improve discrimination of optical properties of deep tissue of the breast.

Approach: Hemodynamic changes in response to localized breast compression were measured in 13 healthy volunteers using a handheld SFDI device. Epidermis optical thickness was determined based on spectral fitting of the model output and used to calculate subcutaneous optical properties.

Results: Optical properties from a homogeneous model yielded physiologically unreasonable absorption and scattering coefficients for highly pigmented volunteers. The two-layer model compensated for the effect of melanin and yielded properties in the expected range for healthy breast. Extracted epidermal optical thickness was higher for higher Fitzpatrick types. Compression induced a decrease in total hemoglobin consistent with tissue blanching.

Conclusions: The handheld SFDI device and two-layer model show potential for imaging hemodynamic responses that potentially could help predict efficacy of NAC in patients of varying skin tones.

Significance: It is not sufficient to detect caries lesions on tooth surfaces; it is also necessary to measure the activity of the lesions to determine if intervention is needed. Changes in the reflectivity of lesion areas during dehydration with forced air at short wavelength infrared (SWIR) wavelengths can be used to assess lesion activity since these changes represent the evaporation dynamics of water from the lesion.

Aim: The aim of this study is to develop new optical methods for assessing lesion activity on tooth surfaces utilizing the strong water absorption band near 1950-nm.

Approach: The time-resolved reflectivity of 20 active and arrested caries lesions on the surfaces of human extracted teeth was monitored at 1300 to 2000 nm using broadband light sources and an extended range InGaAs camera during drying with air.

Results: Multiple parameters representing the rate of change of the lesion reflectivity correlated with the presence of a highly mineralized outer surface zone indicative of lesion arrest measured with x-ray microtomography (microCT). Performance at 1950-nm was higher than for other wavelengths.

Conclusions: This study demonstrates that SWIR imaging near 1950-nm has great potential for the assessment of lesion activity.

Significance: Histopathological analysis of tissues is an essential tool for grading, staging, diagnosing, and resecting cancers and other malignancies. Current histopathological imaging techniques require substantial sample processing, prior to staining with hematoxylin and eosin (H&E) dyes, to highlight nuclear and cellular morphology. Sample preparation and staining is resource intensive and introduces potential for variability during sample preparation.

Aim: We present a method for direct label-free histopathological assessment of formalin-fixed paraffin-embedded tissue blocks and thin tissue sections using a dual-contrast photoacoustic remote sensing (PARS) microscopy system.

Approach: To emulate the nuclear and cellular contrast of H&E staining, we leverage unique properties of the PARS system. Here, the ultraviolet excitation PARS microscope takes advantage of DNA’s unique optical absorption to provide nuclear contrast analogous to hematoxylin staining of cell nuclei. Concurrently, the optical scattering contrast of the PARS detection system is leveraged to provide bulk tissue contrast reminiscent of eosin staining of cell membranes.

Results: We demonstrate the efficacy of this technique by imaging human breast tissue and human skin tissues in formalin-fixed paraffin-embedded tissue blocks and frozen sections, respectively. Salient nuclear and extranuclear features such as cancerous cells, glands and ducts, adipocytes, and stromal structures such as collagen are captured.

Conclusions: The presented dual-contrast PARS microscope enables label-free visualization of tissues with contrast and quality comparable to the current gold standard for histopathological analysis. Thus, the proposed system is well positioned to augment existing histopathological workflows, providing histological imaging directly on unstained tissue blocks and sections.

Significance: Confocal laser scanning enables optical sectioning in fiber bundle endomicroscopy but limits the frame rate. To be able to better explore tissue morphology, it is useful to stitch sequentially acquired frames into a mosaic. However, low frame rates limit the maximum probe translation speed. Line-scanning (LS) confocal endomicroscopy provides higher frame rates, but residual out-of-focus light degrades images. Subtraction-based approaches can suppress this residue at the expense of introducing motion artifacts.

Aim: To generate high-frame-rate endomicroscopy images with improved optical sectioning, we develop a high-speed subtraction method that only requires the acquisition of a single camera frame.

Approach: The rolling shutter of a CMOS camera acts as both the aligned and offset detector slits required for subtraction-based sectioning enhancement. Two images of the bundle are formed on different regions of the camera, allowing both images to be acquired simultaneously.

Results: We confirm improved optical sectioning compared to conventional LS, particularly far from focus, and show that motion artifacts are not introduced. We demonstrate high-speed mosaicing at frame rates of up to 240 Hz.

Conclusion: High-speed acquisition of optically sectioned images using the new subtraction based-approach leads to improved mosaicing at high frame rates.

Significance: Autofluorescence measurements of the metabolic cofactors NADH and flavin adenine dinucleotide (FAD) provide a label-free method to quantify cellular metabolism. However, the effect of extracellular pH on flavin lifetimes is currently unknown.

Aim: To quantify the relationship between extracellular pH and the fluorescence lifetimes of FAD, flavin mononucleotide (FMN), and reduced nicotinamide adenine dinucleotide (phosphate) [NAD(P)H].

Approach: Human breast cancer (BT474) and HeLa cells were placed in pH-adjusted media. Images of an intracellular pH indicator or endogenous fluorescence were acquired using two-photon fluorescence lifetime imaging. Fluorescence lifetimes of FAD and FMN in solutions were quantified over the same pH range.

Results: The relationship between intracellular and extracellular pH was linear in both cell lines. Between extracellular pH 4 to 9, FAD mean lifetimes increased with increasing pH. NAD(P)H mean lifetimes decreased with increasing pH between extracellular pH 5 to 9. The relationship between NAD(P)H lifetime and extracellular pH differed between the two cell lines. Fluorescence lifetimes of FAD, FAD-cholesterol oxidase, and FMN solutions decreased, showed no trend, and showed no trend, respectively, with increasing pH.

Conclusions: Changes in endogenous fluorescence lifetimes with extracellular pH are mostly due to indirect changes within the cell rather than direct pH quenching of the endogenous molecules.

An error in the first author’s name is corrected.

An error in the first author’s name is corrected.