PurposeTo help radiologists examine the growing number of computed tomography (CT) scans, automatic anomaly detection is an ongoing focus of medical imaging research. Radiologists must analyze a CT scan by searching for any deviation from normal healthy anatomy. We propose an approach to detecting abnormalities in axial 2D CT slice images of the brain. Although much research has been done on detecting abnormalities in magnetic resonance images of the brain, there is little work on CT scans, where abnormalities are more difficult to detect due to the low image contrast that must be represented by the model used.ApproachWe use a generative adversarial network (GAN) to learn normal brain anatomy in the first step and compare two approaches to image reconstruction: training an encoder in the second step and using iterative optimization during inference. Then, we analyze the differences from the original scan to detect and localize anomalies in the brain.ResultsOur approach can reconstruct healthy anatomy with good image contrast for brain CT scans. We obtain median Dice scores of 0.71 on our hemorrhage test data and 0.43 on our test set with additional tumor images from publicly available data sources. We also compare our models to a state-of-the-art autoencoder and a diffusion model and obtain qualitatively more accurate reconstructions.ConclusionsWithout defining anomalies during training, a GAN-based network was used to learn healthy anatomy for brain CT scans. Notably, our approach is not limited to the localization of hemorrhages and tumors and could thus be used to detect structural anatomical changes and other lesions.
PurposeAnalyzing the anatomy of the aorta and left ventricular outflow tract (LVOT) is crucial for risk assessment and planning of transcatheter aortic valve implantation (TAVI). A comprehensive analysis of the aortic root and LVOT requires the extraction of the patient-individual anatomy via segmentation. Deep learning has shown good performance on various segmentation tasks. If this is formulated as a supervised problem, large amounts of annotated data are required for training. Therefore, minimizing the annotation complexity is desirable.ApproachWe propose two-dimensional (2D) cross-sectional annotation and point cloud-based surface reconstruction to train a fully automatic 3D segmentation network for the aortic root and the LVOT. Our sparse annotation scheme enables easy and fast training data generation for tubular structures such as the aortic root. From the segmentation results, we derive clinically relevant parameters for TAVI planning.ResultsThe proposed 2D cross-sectional annotation results in high inter-observer agreement [Dice similarity coefficient (DSC): 0.94]. The segmentation model achieves a DSC of 0.90 and an average surface distance of 0.96 mm. Our approach achieves an aortic annulus maximum diameter difference between prediction and annotation of 0.45 mm (inter-observer variance: 0.25 mm).ConclusionsThe presented approach facilitates reproducible annotations. The annotations allow for training accurate segmentation models of the aortic root and LVOT. The segmentation results facilitate reproducible and quantifiable measurements for TAVI planning.
Analyzing the anatomy of the aorta and left ventricular outflow tract (LVOT) is crucial for risk assessment and planning of transcatheter aortic valve implantation (TAVI). We propose 2D cross-sectional annotation and point cloud-based surface reconstruction to train a fully automatic 3D segmentation network for the aortic root and the LVOT. Our sparse annotation scheme enables easy and fast training data generation for tubular structures like the aortic root. Based on this annotation concept, we trained a 3D segmentation model that achieves a Dice similarity coefficient (DSC) of 0.9 and an average surface distance (ASD) of 0.96 mm. In addition, we show that our fully automatic segmentation approach facilitates reproducible and quantifiable measurements for TAVI planning. Our approach achieves an aortic annulus maximum diameter difference between prediction and annotation of 0.45 mm (inter-observer variance: 0.25 mm).
Previous work on methods for cross domain generalization in medical imaging found a simple but very effective method called ”global intensity non-linear” (GIN) augmentation. Our goal in this study is to use the GIN approach to train a model as powerful as TotalSegmentator for MRI data, despite having neither sufficient amounts of MRI data nor ground truth organ contours. Instead, we employ the GIN augmentation approach to show qualitatively and quantitatively that this is indeed feasible for a diverse set of anatomical structures including abdominal and thoracic organs as well as bones. The models are trained on the TotalSegmentator and AMOS22 datasets. For evaluation we apply them to whole body MRI scans from the German National Cohort (NAKO) study with a set of in-house reference masks. With GIN augmentation the mean Dice score of the model increases from 0.18 to 0.52 on Dixon water images, when using TotalSegmentator data for training. The improvements can be further split into 0.47 to 0.66 for abdominal organs, 0.55 to 0.79 for thoracic organs and 0.00 to 0.40 for bones.
SignificanceAlthough the registration of restained sections allows nucleus-level alignment that enables a direct analysis of interacting biomarkers, consecutive sections only allow the transfer of region-level annotations. The latter can be achieved at low computational cost using coarser image resolutions.PurposeIn digital histopathology, virtual multistaining is important for diagnosis and biomarker research. Additionally, it provides accurate ground truth for various deep-learning tasks. Virtual multistaining can be obtained using different stains for consecutive sections or by restaining the same section. Both approaches require image registration to compensate for tissue deformations, but little attention has been devoted to comparing their accuracy.ApproachWe compared affine and deformable variational image registration of consecutive and restained sections and analyzed the effect of the image resolution that influences accuracy and required computational resources. The registration was applied to the automatic nonrigid histological image registration (ANHIR) challenge data (230 consecutive slide pairs) and the hyperparameters were determined. Then without changing the parameters, the registration was applied to a newly published hybrid dataset of restained and consecutive sections (HyReCo, 86 slide pairs, 5404 landmarks).ResultsWe obtain a median landmark error after registration of 6.5 μm (HyReCo) and 24.1 μm (ANHIR) between consecutive sections. Between restained sections, the median registration error is 2.2 and 0.9 μm in the two subsets of the HyReCo dataset. We observe that deformable registration leads to lower landmark errors than affine registration in both cases (p < 0.001), though the effect is smaller in restained sections.ConclusionDeformable registration of consecutive and restained sections is a valuable tool for the joint analysis of different stains.
Automatic detection of abnormalities to assist radiologists in acute and screening scenarios has become a particular focus in medical imaging research. Various approaches have been proposed for the detection of anomalies in magnetic resonance (MR) data, but very little work has been done for computed tomography (CT). As far as we know, there is no satisfactory approach for anomaly detection in CT brain images. We present a novel unsupervised deep learning approach to generate a normal representation (without anomalies) of CT head scans that we use to discriminate between healthy and abnormal images. In the first step, we train a GAN with 1000 healthy CT scans to generate normal head images. Subsequently, we attach an encoder to the generator and train the auto encoder network to reconstruct healthy anatomy from new input images. The auto encoder is pre-trained with generated images using a perceptual loss function. When applied to abnormal scans, the reconstructed healthy output is then used to detect anomalies by computing the Mean Squared Error between input and output image. We evaluate our slice-wise anomaly detection on 250 test images including hemorrhages and tumors. Our approach achieves an area under receiver operating characteristic curve (AUC) of 0.90 with 85.8% sensitivity and 85.5% precision without requiring large training data sets or labeled anomaly data. Therefore, our method discriminates between normal and abnormal CT scans with good accuracy.
We present a novel approach for handling complex information of lesion segmentation in CT follow-up studies. The backbone of our approach is the computation of a longitudinal tumor tree. We perform deep learning based segmentation of all lesions for each time point in CT follow-up studies. Subsequently, follow-up images are registered to establish correspondence between the studies and trace tumors among time points, yielding tree-like relations. The tumor tree encodes the complexity of the individual disease progression. In addition, we present novel descriptive statistics and tools for correlating tumor volumes and RECIST diameters to analyze significance of various markers.
Inflammatory white matter brain lesions are a key pathological finding in patients suffering from multiple sclerosis (MS). Image based quantification of different characteristics of these lesions has become an elemental bio-marker in both diagnosis as well as therapy monitoring during treatment of these patients. Whilst it has been shown that the lesion load at a single point in time is only of limited value with respect to explaining clinical symptoms of the patients, a more robust estimate of disease activity can be observed by analyzing the evolution of lesions over time. Here, we propose a system for automated monitoring of temporal lesion evolution in MS. We describe an approach for analysis of lesion correspondence, along with a pipeline for fully automated computation of this model. The pipeline consists of a U-Net based lesion segmentation, a non-linear image registration between multiple studies, computation of temporal lesion correspondences, and finally an analysis module for extracting and visualizing quantitative parameters from the model.
While deep learning based methods for medical deformable image registration have recently shown significant advances in both speed and accuracy, methods for use in radio therapy are still rarely proposed due to several challenges such as low contrast and artifacts in cone beam CT (CBCT) images or extreme deformations. The aim of image registration in radio therapy is to align a baseline CT and low-dose CBCT images, which allows contours to be propagated and applied doses to be tracked over time. To this end, we present a novel deep learning method for multi-modal deformable CT-CBCT registration. We train a CNN in weakly supervised manner, aiming to optimize an edge-based image similarity and a deformation regularizer including a penalty for local changes of topology and foldings. Additionally, we measure the alignment of given segmentations, facing the problem of extreme deformations. Our method receives only CT and a CBCT images as input and uses groundtruth segmentations exclusively during training. Furthermore, our method is not dependent on the availability of difficult to access ground-truth deformation vector fields. We train and evaluate our method on follow-up image pairs of the pelvis and compare our results to conventional iterative registration algorithms. Our experiments show that the registration accuracy of our deep learning based approach is superior to iterative registration without additional guidance by segmentations and nearly as good as iterative structure guided registration that requires ground-truth segmentations. Furthermore, our deep learning based method runs approximately 100 times faster than the iterative methods.
Image registration is the process of aligning two or more images to achieve point-wise spatial correspondence. Typically, image registration is phrased as an optimization problem w.r.t. a spatial mapping that minimizes a suitable cost function and common approaches estimate solutions by applying iterative optimization schemes such as gradient descent or Newton-type methods. This optimization is performed independently for each pair of images, which can be time consuming. In this paper we present an unsupervised learning-based approach for deformable image registration of thoracic CT scans. Our experiments show that our method performs comparable to conventional image registration methods and in particular is able to deal with large motions. Registration of a new unseen pair of images only requires a single forward pass through the network yielding the desired deformation field in less than 0.2 seconds. Furthermore, as a novelty in the context of deep-learning-based registration, we use the edge-based normalized gradient fields distance measure together with the curvature regularization as a loss function of the registration network.
Since the first clinical interventions in the late 1980s, Deep Brain Stimulation (DBS) of the subthalamic nucleus has evolved into a very effective treatment option for patients with severe Parkinson's disease. DBS entails the implantation of an electrode that performs high frequency stimulations to a target area deep inside the brain. A very accurate placement of the electrode is a prerequisite for positive therapy outcome. The assessment of the intervention result is of central importance in DBS treatment and involves the registration of pre- and postinterventional scans.
In this paper, we present an image processing pipeline for highly accurate registration of postoperative CT to preoperative MR. Our method consists of two steps: a fully automatic pre-alignment using a detection of the skull tip in the CT based on fuzzy connectedness, and an intensity-based rigid registration. The registration uses the Normalized Gradient Fields distance measure in a multilevel Gauss-Newton optimization framework and focuses on a region around the subthalamic nucleus in the MR.
The accuracy of our method was extensively evaluated on 20 DBS datasets from clinical routine and compared with manual expert registrations. For each dataset, three independent registrations were available, thus allowing to relate algorithmic with expert performance. Our method achieved an average registration error of 0.95mm in the target region around the subthalamic nucleus as compared to an inter-observer variability of 1.12 mm. Together with the short registration time of about five seconds on average, our method forms a very attractive package that can be considered ready for clinical use.
Johannes Lotz, Judith Berger, Benedikt Müller, Kai Breuhahn, Niels Grabe, Stefan Heldmann, André Homeyer, Bernd Lahrmann, Hendrik Laue, Janine Olesch, Michael Schwier, Oliver Sedlaczek, Arne Warth
Much insight into metabolic interactions, tissue growth, and tissue organization can be gained by analyzing differently stained histological serial sections. One opportunity unavailable to classic histology is three-dimensional (3D) examination and computer aided analysis of tissue samples. In this case, registration is needed to reestablish spatial correspondence between adjacent slides that is lost during the sectioning process. Furthermore, the sectioning introduces various distortions like cuts, folding, tearing, and local deformations to the tissue, which need to be corrected in order to exploit the additional information arising from the analysis of neighboring slide images. In this paper we present a novel image registration based method for reconstructing a 3D tissue block implementing a zooming strategy around a user-defined point of interest. We efficiently align consecutive slides at increasingly fine resolution up to cell level. We use a two-step approach, where after a macroscopic, coarse alignment of the slides as preprocessing, a nonlinear, elastic registration is performed to correct local, non-uniform deformations. Being driven by the optimization of the normalized gradient field (NGF) distance measure, our method is suitable for differently stained and thus multi-modal slides. We applied our method to ultra thin serial sections (2 μm) of a human lung tumor. In total 170 slides, stained alternately with four different stains, have been registered. Thorough visual inspection of virtual cuts through the reconstructed block perpendicular to the cutting plane shows accurate alignment of vessels and other tissue structures. This observation is confirmed by a quantitative analysis. Using nonlinear image registration, our method is able to correct locally varying deformations in tissue structures and exceeds the limitations of globally linear transformations.
Lung registration in thoracic CT scans has received much attention in the medical imaging community. Possible applications range from follow-up analysis, motion correction for radiation therapy, monitoring of air flow and pulmonary function to lung elasticity analysis. In a clinical environment, runtime is always a critical issue, ruling out quite a few excellent registration approaches. In this paper, a highly efficient variational lung registration method based on minimizing the normalized gradient fields distance measure with curvature regularization is presented. The method ensures diffeomorphic deformations by an additional volume regularization. Supplemental user knowledge, like a segmentation of the lungs, may be incorporated as well. The accuracy of our method was evaluated on 40 test cases from clinical routine. In the EMPIRE10 lung registration challenge, our scheme ranks third, with respect to various validation criteria, out of 28 algorithms with an average landmark distance of 0.72 mm. The average runtime is about 1:50 min on a standard PC, making it by far the fastest approach of the top-ranking algorithms. Additionally, the ten publicly available DIR-Lab inhale-exhale scan pairs were registered to subvoxel accuracy at computation times of only 20 seconds. Our method thus combines very attractive runtimes with state-of-the-art accuracy in a unique way.
In navigated liver surgery it is an important task to align intra-operative data to pre-operative planning data.
This work describes a method to register pre-operative 3D-CT-data to tracked intra-operative 2D US-slices.
Instead of reconstructing a 3D-volume out of the two-dimensional US-slice sequence we directly apply the registration
scheme to the 2D-slices. The advantage of this approach is manyfold. We circumvent the time consuming
compounding process, we use only known information, and the complexity of the scheme reduces drastically. As
the liver is a non-rigid organ, we apply non-linear techniques to take care of deformations occurring during the
intervention. During the surgery, computing time is a crucial issue. As the complexity of the scheme is proportional
to the number of acquired slices, we devise a scheme which starts out by selecting a few "key-slices" to
be used in the non-linear registration scheme. This step is followed by multi-level/multi-scale strategies and fast
optimization techniques. In this abstract we briefly describe the new method and show first convincing results.
In this work we evaluate a novel method for multi-modal image registration of MR images. The key feature of our approach is a new distance measure that allows for comparing modalities that are related by an arbitrary gray-value mapping. The novel measure is formulated as least square problem for minimizing the sum of squared
differences of two images with respect to changing gray-values of one of the images. It turns out that the novel measure can be computed explicitly and allows for very simple and efficient implementation. We compare our new approach to rigid registration with cross-correlation, mutual information, and normalized gradient fields as distance measure.
The resection of a tumor is one of the most common tasks in liver surgery. Here, it is of particular importance to
resect the tumor and a safety margin on the one hand and on the other hand to preserve as much healthy liver
tissue as possible. To this end, a preoperative CT scan is taken in order to come up with a sound resection strategy.
It is the purpose of this paper to compare the preoperative planning with the actual resection result. Obviously
the pre- and postoperative data is not straightforward comparable, a meaningful registration is required. In the
literature one may find a rigid and a landmark-based approach for this task. Whereas the rigid registration does
not compensate for nonlinear deformation the landmark approach may lead to an unwanted overregistration.
Here we propose a fully automatic nonlinear registration with volume constraints which seems to overcome both
aforementioned problems and does lead to satisfactory results in our test cases.
In this work we present a novel approach for elastic image registration of multi-phase contrast enhanced CT
images of liver. A problem in registration of multiphase CT is that the images contain similar but complementary
structures. In our application each image shows a different part of the vessel system, e.g., portal/hepatic
venous/arterial, or biliary vessels. Portal, arterial and biliary vessels run in parallel and abut on each other
forming the so called portal triad, while hepatic veins run independent. Naive registration will tend to align
complementary vessel.
Our new approach is based on minimizing a cost function consisting of a distance measure and a regularizer.
For the distance we use the recently proposed normalized gradient field measure that focuses on the alignment
of edges. For the regularizer we use the linear elastic potential. The key feature of our approach is an additional
penalty term using segmentations of the different vessel systems in the images to avoid overlaps of complementary
structures. We successfully demonstrate our new method by real data examples.
Image registration is an important and active area of medical image processing. Given two images, the idea is
to compute a reasonable displacement field which deforms one image such that it becomes similar to the other
image. The design of an automatic registration scheme is a tricky task and often the computed displacement
field has to be discarded, when the outcome is not satisfactory. On the other hand, however, any displacement
field does contain useful information on the underlying images.
It is the idea of this note, to utilize this information and to benefit from an even unsuccessful attempt for the
subsequent treatment of the images. Here, we make use of typical vector analysis operators like the divergence
and curl operator to identify meaningful portions of the displacement field to be used in a follow-up run. The
idea is illustrated with the help of academic as well as a real life medical example. It is demonstrated on how the
novel methodology may be used to substantially improve a registration result and to solve a difficult segmentation
problem.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.