Many applications of MRI are facilitated by segmenting the volume spanned by the imagery into the various tissue types that are present. Intensity-based classification of MR images has proven to be problematic, even when advanced techniques such as non- parametric multi-channel methods are used. A persistent difficulty has been accommodating the spatial intensity inhomogeneities that are due to the equipment. This paper describes a statistical method that uses knowledge of tissue properties and intensity inhomogeneities to correct for these intensity inhomogeneities. Use of the Expectation-Maximization algorithm leads to a method (EM segmentation) for simultaneously estimating tissue class and the correcting gain field. The algorithm iterates two components to convergence: tissue classification, and gain field estimation. The result is a powerful new technique for segmenting and correcting MR images. An implementation of the method is discussed, and results are reported for segmentation of white matter and gray matter in gradient-echo and spin-echo images. Examples are shown for axial, coronal and sagittal (surface coil) images. For a given type of acquisition, intensity variations across patients, scans, and equipment have been accommodated without manual intervention in the segmentation. In this sense, the method is fully automatic for segmenting healthy brain tissue. An accuracy assessment was made in which the method was compared to manual segmentation, and to a method based on supervised multi-variate classification, in segmenting white matter and gray matter. The method was found to be consistent with manual segmentation, and closer to manual segmentation than the supervised method.

The methods presented here have been developed to perform the automated segmentation of coronary arterial structure from cine sequences of biplanar x-ray angiograms. We introduce a methodology to impose an integrated set of constraints based on knowledge concerning the anatomical structure of the vascular system, temporal changes in position due to motion, and spatial coherence. Results are shown for data sets generated from both porcine and human studies.

We propose a novel method for obtaining the maximum a posteriori (MAP) probabilistic segmentation of speckle-laden ultrasound images. Our technique is multiple-resolution based, and relies on the conversion of speckle images with Rayleigh statistics to subsampled images with Gaussian statistics. This conversion reduces computation time, as well as allowing accurate parameter estimation. Results appear to provide improvements over previous techniques, in terms of both low-resolution detail and accuracy.

Measured data are inherently inaccurate. Operations done on data for defining, visualizing, manipulating and analyzing object information should attempt to retain these inaccuracies as accurately as possible. The theory of fuzzy sets is a proper mathematical vehicle for this purpose. In this attempt, topological notions such as adjacency, connectedness, and boundary need to be developed starting from fuzzy sets. We develop such a framework in this paper and present algorithms for finding fuzzy connected components and boundaries in digital imagery. We demonstrate the usefulness of these algorithms in medical applications, particularly in separating objects that come close together which are otherwise difficult to segment using hard (non-fuzzy) criteria.

We describe a GUI-based system called INTERSEG that can define 3D radiological image-segmentation processes. Using INTERSEG's GUI interface, the user first defines interactively some problem cues, which specify a segmentation task. Then, INTERSEG uses the cues to automatically construct a process for the task. The process consists of automatic image-processing operations, such as image enhancement, image segmentation, shape analysis, and measurement. With other systems the user must resort to trial- and-error experimentation to construct such processes.

This paper describes an image segmentation technique in which an arbitrarily shaped contour was deformed stochastically until it fitted around an object of interest. The evolution of the contour was controlled by a simulated annealing process which caused the contour to settle into the global minimum of an image-derived 'energy' function. The non-parametric energy function was derived from the statistical properties of previously-segmented images, thereby incorporating prior experience. Since the method was based on a state space search for the contour with the best global properties, it was stable in the presence of image errors which confound segmentation techniques based on local criteria such as connectivity. However, unlike 'snakes' and other active contour approaches, the new method could handle arbitrarily irregular contours in which each inter-pixel crack represented an independent degree of freedom. The method was illustrated by using it to find the brain surface in magnetic resonance head images, to identify the epicardial surface in magnetic resonance cardiac images, and to track blood vessels in angiograms.

An improved segmentation algorithm based on the active contour model is developed. A distinguishing element of our algorithm is the incorporation of region-based image features which improves signifcantly the reliability of the algorithm, and allows multi- region segmentation of an image. We use simulated annealing in the energy minimization in order to locate the globally optimal solution and enhance flexibility in the construction of energy functional.

A globally optimal region growing algorithm for 3D segmentation of anatomical objects is developed. The notion of simple 3D connected component labelling is extended to enable the combination of arbitrary features in the segmentation process. This algorithm uses a hybrid octree-btree structure to segment an object of interest in an ordered fashion. This tree structure overcomes the computational complexity of global optimality in three dimensions. The segmentation process is controlled by a set of active features, which work in concert to extract the object of interest. The cost function used to enforce the order is based on the combination of active features. The characteristics of the data throughout the volume dynamically influences which features are active. A foundation for applying user interaction with the object directly to the feature set is established. The result is a system which analyzes user input and neighborhood data and optimizes the tools used in the segmentation process accordingly.

Shape-based interpolation (SBI) is used for interpolation between binary serial slice images. Although SBI approximates the interslice geometry more accurately than traditional techniques such as linear (L) or cubic spline (CS) interpolation, SBI produces only a binary result. This paper extends SBI to interpolation of grayscale images (SBIG) using simulated 3D distance maps to produce a grayscale image volume. Results of SBIG are superior visually (sharper detail, no artificial intensities) and quantitatively to L or CS. This is particularly evident in sagittal and coronal reconstructions. Clipping artifacts due to nonoverlapping structures or rapid changes in image brightness are minimized using simulated 3D maps. However, when objects between slices do not overlap, shape-based interpolation results in compressed or nonexistent geometry in some or all of the interpolated slices. The nonoverlapping problem is described and quantified. A new interpolation algorithm, directed distance morphing, is introduced and used to address the nonoverlapping problem.

Current modeling and segmentation techniques are not adequate for visualizing certain kinds of 3D medical data. This paper introduces volumetric deformable models, or active blobs, as a means of visualizing and segmenting 3D data. It presents an implicit framework that provides a means of generalizing deformable models to higher dimensions. It presents a new second- order smoothing function that offers some advantages over previous methods for smoothing surfaces. It describes some of the numerical techniques that are necessary for solving the resulting evolution equations. Finally it shows some examples of how active blobs are useful for the segmentation and visualization of 3D ultrasound and MRI data. Active blobs have a number of advantages over other types of modeling techniques. Because of these advantages, active blobs are able to produce models directly from the 3D data, rather than presegmented datasets, edge maps, or sets of landmarks.

Edgewarp, a program in C and XWindows, is a wholly graphical interface for managing the complicated algebra by which thin- plate splines are applied in contemporary biomedical image analysis. This flexible warping package permits the free specification of deformations by arbitrary combinations of landmark point correspondences and constraints. Its parameters may be forwarded for rigorous multivariate statistical analysis at the same time that the associated images can be averaged or correlated after unwarping. This paper describes the kinematics of the interface by which clinical users can exploit these graphics and biometrics. Even though we show an actual clinical finding arrived at by these means, our main purpose is to inveigle the reader into playing with our shareware.

This paper describes efforts aimed at more objectively and accurately quantifying the local, regional and global function of the left ventricle (LV) of the heart from 4D image data. Using our shape-based image analysis methods, point-wise myocardial motion vector fields between successive image frames through the entire cardiac cycle will be computed. Quantitative LV motion, thickening, and strain measurements will then be established from the point correspondence maps. In the paper, we will also briefly describe an in vivo experimental model which uses implanted imaging-opaque markers to validate the results of our image analysis methods. Finally, initial experimental results using image sequences from two different modalities will be presented.

An iterative algorithm is presented for simultaneous deformation of multiple curves and surfaces to an MRI, with inter-surface constraints and self-intersection avoidance. The resulting robust segmentation, combined with local curvature matching, automatically creates surfaces of MRI datasets with a common mapping to surface parametric space.

Three dimensional ultrasound imaging with a freehand probe allows a flexible approach to medical visualization and diagnosis. Given the imperfect accuracy of proprioceptive devices used to log the position and tilt of the probe, it is important to utilize the position constraints provided by image evidence. This is also important if we wish to consider the visualization of structures which move significantly during acquisition, such as a heart of fetus. We present here an initial approach to more robust segmentation and shape recovery in a particularly noisy modality. We consider 2D segmentation based on edge evidence, using first an active contour, then finding an optimal segmentation using simulated annealing. Correspondence between contours in adjacent frames can only be solved in general cases by use of a 3D prior model. Dynamic physics-based mesh models as used by Pentland [20] and Nastar [17], allow for shape modelling, then over-constrained 3D shape recovery can be performed using the intrinsic vibration modes of the model.

We describe an automated method to register MRI volumetric datasets to a digital human brain model. The technique employs 3D non-linear warping based on the estimation of local deformation fields using cross-correlation of invariant intensity features derived from image data. Results of the non-linear registration on a simple phantom, a complex brain phantom and real MRI data are presented. Anatomical variability is expressed with respect to the Talairach-like standardized brain-based coordinate system of the model. We show that the automated non-linear registration reduces the inter-subject variability of homologous points in standardized space by 15% over linear registration methods. A 3D variability map is shown.

The efficacy of using intensity edges, curvature of iso-intensity contours, and tissue classified data for image matching are examined. The image matching problem is formulated in such a way that the different features are handled uniformly, allowing the same code to be used in each instance. The results using both simulated and real brain images indicate that each feature affected and improvement in the correspondence after matching with it.

We present the concept of the feature space sequence: 2D distributions of voxel features of two images generated at registration and a sequence of misregistrations. We provide an explanation of the structure seen in these images. Feature space sequences have been generated for a pair of MR image volumes identical apart from the addition of Gaussian noise to one, MR image volumes with and without Gadolinium enhancement, MR and PET-FDG image volumes and MR and CT image volumes, all of the head. The structure seen in the feature space sequences was used to devise two new measures of similarity which in turn were used to produce plots of cost versus misregistration for the 6 degrees of freedom of rigid body motion. One of these, the third order moment of the feature space histogram, was used to register the MR image volumes with and without Gadolinium enhancement. These techniques have the potential for registration accuracy to within a small fraction of a voxel or resolution element and therefore interpolation errors in image transformation can be the dominant source of error in subtracted images. We present a method for removing these errors using sinc interpolation and show how interpolation errors can be reduced by over two orders of magnitude.

The core is a multiscale object description extracted directly from an image in which the significance of object features depends on the scale at which the object is currently being considered. Core descriptions extracted from the same object using different imaging modalities display remarkable similarities despite noise and variations imposed by the imaging devices. Core based registration methods have been successfully employed in examples of 3D registration that would have caused boundary or contour based methods to fail. In this paper, we present an overview of the core, and describe its properties that make it a promising approach toward registration. Our current core-based method of registration is also presented. Results of the registration of 3D volumes from both a single imaging modality and from two different modalities are given.

Grey value correlation is generally considered not to be applicable to matching of images of different modalities. In this paper we will demonstrate that, with a simple preprocessing step for the Computed Tomography (CT) images, grey value correlation can be used for matching of Magnetic Resonance Imaging (MRI) with CT images. Two simple schemes are presented for automated 3D matching of MRI and CT neuroradiological images. Both schemes involve grey value correlation of the images in order to determine the matching transformation. In both schemes the preprocessing consists of a simple intensity mapping of the original CT image only. It will be shown that the results are insensitive to considerable changes in the parameters that determine the intensity mapping. Whichever preprocessing step is chosen, the correlation method is robust and accurate. Results, compared with a skin marker-based matching technique, are shown for brain images. Additionally, results are shown for an entirely new application: matching of the cervical spine.

Composite or 'fused' images generated using multimodality/multispectral images from magnetic resonance images (MRI) and computed tomography (CT) are increasingly being used in various diagnostic and therapeutic procedures such as neurosurgery and radiotherapy treatment planning. To be able to make quantitative measurements from these images it is important to determine the error introduced by the image registration algorithm. We have quantified the errors introduced by one such algorithm, the surface matching algorithm available in the ANALYZE software package. Image data from a head phantom and from actual patient data are used in the analysis. Three indicators are evaluated - translation error, residual gray scale error and the error in external marker distances. Results are presented for out-of-plane, in-plane and multiaxis rotation over a range of angles. The registration error increases with an increase in the out-of-plane rotation. Errors introduced by the interpolation and truncation of data in the registration procedure is also evaluated and remains fairly constant. A high degree of correlation is seen among the three measures of error and these should prove useful in evaluating other algorithms as well.

Nowadays, several studies on 3D medical image registration are being investigated. The main purpose of these studies is to integrate complimentary information provided by different modalities. This paper describes a newly developed registration method of ultrasound echography to 3D medical image. The registration process consists of two stages. In the first stage, a registration process of the ultrasound echography to the pre- operative 3D images is performed using a position sensor. This is a rigid transformation, which can't provide a perfect match because the intra-operative organ is deformed or moved. The second step is a local matching process, where the current position of the ultrasound echography is approximated with a search-based surface matching algorithm. We used the 3D chamfer matching for this approximation. In this method, a goodness-of- match function, i.e., generalized distance, that rates the geometrical transformation is computed and minimized. In a preliminary experiment, the search-based matching process was examined. The algorithm and its accuracy were evaluated with an artificial transformed set of coordinates given as the initial guess. We also performed a clinical application and results confirmed the suitability of this method to clinical use.

High resolution volumetric scanning with modern slipring technology CT enables 3D subtraction angiography. The advantage over conventional CTA is the contrast improvement enabling automatic segmentation and the elimination of high density data (bone), a prerequisite for automatic generation of MIP renderings. Aspects of matching and quality are discussed.

Medical image analysis has to support the clinicians ability to identify, manipulate and quantify anatomical structures. On scalar 2D image data, a human observer is often superior to computer assisted analysis, but the interpretation of vector- valued data or data combined from different modalities, especially in 3D, can benefit from computer assistance. The problem of how to convey the complex information to the clinician is often tackled by providing colored multimodality renderings. We propose to go a step beyond by supplying a suitable modelling of anatomical and functional structures encoding important shape features and physical properties. The multiple attributes regarding geometry, topology and function are carried by the symbolic description and can be interactively queried and edited. Integrated 3D rendering of object surfaces and symbolic representation acts as a visual interface to allow interactive communication between the observer and the complex data, providing new possibilities for quantification and therapy planning. The discussion is guided by the prototypical example of investigating the cerebral vasculature in MRA volume data. Geometric, topological and flow-related information can be assessed by interactive analysis on a computer workstation, providing otherwise hidden qualitative and quantitative information. Several case studies demonstrate the potential usage for structure identification, definition of landmarks, assessment of topology for catheterization, and local simulation of blood flow.

We present multi-modality visualization strategies to convey information contained in registered Single Photon Emission Photography (SPECT) and Magnetic Resonance (MR) images of the brain. Multi-modality visualization provides a means to retrieve valuable information from the data which might otherwise remain obscured. Here we use MRI as an anatomical framework for functional information acquired with SPECT. This is part of clinical research studying the change of functionality caused by a frontal lobe damaged region. A number of known and newly developed techniques for the integrated visualization of SPECT and MR images will be discussed.

Simulation and 3D visualization of object motion is a prerequisite for any surgical planning system. In order to provide feedback to show whether a realistic motion has been simulated, it is necessary to detect, quantify and visualize interpenetrating volumes. This cannot be achieved by common surface based methods. Therefore we developed a voxel-based approach, providing the full information of the tomographic volume data. We present an extended ray-casting algorithm which allows visualization of object motion using ray compositing, thus avoiding explicit manipulation of the image volume. Possible volumetric intersections may be visualized and quantified and interior properties of scenes with any displace objects may be explored using volume cuts.

This work presents a head-motion-parallax visualization system for arbitrarily complex computer graphics databases (such as biomedical data sets) using precomputed images. It features a large rear-projection screen and a blind restricting the range of possible vantage points. We describe implementation details for the high-resolution low-lag prototype system and relate first reactions from users.

CT-Angiography acquires a volume of data defining vasculature in three dimensions. Prior to 3D visualization, bones have to be removed from the volume. We describe methods that reduce the workload for manual outlining of contours or remove bone structures in the abdomen, chest or head and neck region automatically. Volume rendering has been used to provide information not displayed with Maximum Intensity Projection or Surface Shaded Display. Tools have been developed to ease the setting of parameters for volume rendering of CT data.

Recent advances in image processing and visualization are of increasing use in the investigation of violent crime. The Digital Image Processing Laboratory at the Armed Forces Institute of Pathology in collaboration with groups at the University of North Carolina at Chapel Hill are actively exploring visualization applications including image processing of trauma images, 3D visualization, forensic database management and telemedicine. Examples of recent applications are presented. Future directions of effort include interactive consultation and image manipulation tools for forensic data exploration.

An optical surface scanner was used to digitize and model the torso of an adult female. This 3D surface scanner employs structured light and has an acquisition time of less than one second for a 0.4 X 0.4 X 0.4 meter sample volume. A female volunteer was digitized in three parts using this surface scanner: head, upper torso, and lower torso. External fiducials were used to aid in registration of the three data sets to create a complete body surface model. The fiducial point loci were sampled and entered in a least squares optimization scheme to rigidly transform (rotate and translate) the three data sets into alignment. The digitized data of each scan was converted into spline surfaces and imported into a computer graphics surface modeling package (Studio, Alias Research, Inc. Toronto, Canada). The results demonstrate whole body surface modeling with an optical surface scanner to achieve rapid complex 3D surface coverage.

The effective display of multi-parameter medical image data sets is assuming increasing importance as more distinct imaging modalities are becoming available. For medical pruposes, one desirable goal is to fuse such data sets into a single most informative gray-scale image without making rigid classification decisions. A visualization technique based on a non-linear projection onto a 1D self-organizing map is described and examples are shown. The SOM visualization technique is fast, theoretically attractive, and has properties which compliment those of projection-pursuit or other linear techniques. It may be of particular value in calling attention to specific regions in a multi-parameter image where the component images should be examined in detail.

Inhomogeneities in the fields of magnetic resonance (MR) systems cause the statistical characteristics of tissue classes to vary within the resulting MR images. These inhomogeneities must be taken into consideration when designing an algorithm for automated tissue classification. The traditional approach in image processing would be to apply a gain field correction technique to remove the inhomogeneities from the images. Statistical solutions would most likely focus on including spatial information in the feature space of the classifier so that it can be trained to model and adjust for the inhomogeneities. This paper will prove that neither of these general approaches offer a complete and viable solution. This paper will prove that neither of these general approaches offers a complete and viable solution. This paper will in fact show that not only do the inhomogeneities modify the local mean and variance of a tissue class as is commonly accepted, but the inhomogeneities also induce a rotation of the covariance matrices. As a result, gain field correction techniques cannot compensate for all of the artifacts associated with inhomogeneities. Additionally, it will be demonstrated that while statistical methods can capture all of the anomalies, the across patient and across time variations of the inhomogeneities necessitate frequent and time consuming retraining of any Bayesian classifier. This paper introduces a two stage process for MR tissue classification which addresses both of these issues by utilizing techniques from both image processing and statistics. First, a band-pass mean field corrector is used to alleviate the mean and variance deformations in each image. Then, using a kernel mixture model classifier couple to an interactive data augmentation tool, the user can selectively refine and explore the class representations for localized regions of the image and thereby capture the rotation of the covariance matrices. This approach is shown to outperform Gaussian classifiers and 4D mixture modeling techniques when both the final accuracy and user time requirements are considered.

The high resolution and excellent soft tissue contrast of Magnetic Resonance Imaging (MRI) have enabled direct, non- invasive visualization of Multiple Sclerosis (MS) lesions in vivo. This has allowed quantification of changes in the appearance of lesions in MR exams to be used as a measure of disease state. Nevertheless, accurate quantification techniques are subject to inter- and intra-operator variability, which may hinder monitoring of disease progression. We have developed a computer program to aid an experienced operator in the quantification of MS lesions in spin-echo MR exams. Assisted and manual quantification were used to study inter-operator and intra-operator variability under known conditions in exams of a test phantom, and under clinical conditions in 1.5T and 0.5T exams of an MS patient. Results of the phantom study show that accuracy was improved by assisted quantification. The patient exam results indicate that assisted quantification reduced both inter-operator and intra-operator variability, while quantification in 0.5T exams reduced the variability of manual quantification, but had no significant effect upon assisted quantification. Application of assisted quantification to the analysis and visualization of two large periventricular lesions reveals subtle changes over time in the internal composition of these apparently static lesions.

The paper addresses 3D neuroimaging from MRI data by using a dual probabilistic classifier. The goals are: to enable to see thru the scalp and skull in order to observe the cortical surface and brain deep structures, to achieve a correct appearance of gyration, and to provide tools easy to use by the medical professional. MRI head data is automatically segmented into two regions: the brain (along with some subarachnoid structures and some pare of the outer CSF filling the sulci and fissures) and the outer structures (including the scalp, skull marrow, dura mater). The brain and the outer structures are classified separately using a probabilistic classifier. A new volume is created so as to eliminate the density overlap between the brain and the outer structures. Color and opacity transfer functions suitable to render the volume are generated automatically based on the density probability plots for both regions. Preliminary results are discussed.

An earlier study investigated a technique to reconstruct in three-dimensions, the path of a bullet through a skull, using the post-mortem X-rays of the victim and stock computed tomography (CT) data. This paper describes the addition of image deformation methods in order to improve the accuracy of the reconstruction technique. The skull X-ray images of the victim were warped to match the shape of the skull in the CT. The warping was done by a thin-plate spline deformation algorithm. The bullet path was again reconstructed using the warped X-ray images as the source of information for the bullet entry and exit wound. The difference in the angle of the bullet paths in the two reconstructions was 6.9 degrees.

An earlier study investigated a technique to reconstruct in three- dimensions, the path of a bullet through a skull, using the post-mortem X-rays of the victim and stock computed tomography (CT) data. This paper describes the addition of image deformation methods in order to improve the accuracy of the reconstruction technique. The skull X-ray images of the victim were warped to match the shape of the skull in the CT. The warping was done by a thin-plate spline deformation algorithm. The bullet path was again reconstructed using the warped X-ray images as the source of information for the bullet entry and exit wound. The difference in the angle of the bullet paths in the two reconstructions was 6.9 degrees.

An algorithm is presented for the automatic detection of arachnoid contours in MR images of the human head. The primary motivation behind the present work has been to serve as a pre- processing step in automatic segmentation of brain tissue and CSF. A second objective was to use the algorithm in a fully automatic PET-MR registration algorithm. The method is primarily designed for, and requires, dual-echo (T1- and T2-weighted) MR images with transaxial orientations. The algorithm consists of three main stages. First, the head contour is detected using a series of low-level image processing techniques. In the second stage, the pixels inside the head contour are clustered into a number of connected components using the K-means algorithm. Finally, the extra-arachnoid connected components are eliminated based on a number of heuristics. Test results are presented for 10 MR image sets. As a quantitative measure of accuracy, manual segmentations were performed by radiologists on a number of slices and compared with the results obtained automatically. Visual inspection and quantitative validation of the results indicate that the algorithm accurately detects the arachnoid contours in MR images. This is an important step in fully automatic segmentation and registration of MR images.

Visualization of human anatomy in a 3D atlas requires both spatial and more abstract symbolic knowledge. Within our 'intelligent volume' model which integrates these two levels, we developed and implemented a semantic network model for describing human anatomy. Concepts for structuring (abstraction levels, domains, views, generic and case-specific modeling, inheritance) are introduced. Model, tools for generation and exploration and applications in our 3D anatomical atlas are presented and discussed.

This paper reports about first results obtained in a project aiming at developing a computerized system to manage knowledge about brain anatomy. The emphasis is put on the design of a knowledge base which includes a symbolic model of cerebral anatomical structures (grey nuclei, cortical structures such as gyri and sulci, verntricles, vessels, etc.) and of hypermedia facilities allowing to retrieve and display information associated with the objects (texts, drawings, images). Atlas plates digitized from a stereotactic atlas are also used to provide natural and effective communication means between the user and the system.

This paper presents a general scheme for the building of anatomical atlases. We propose to use specific and stable features, the crest lines (or ridge lines) which are automatically extracted from 3D images by differential geometry operators. We have developed non-rigid registration techniques based on polynomial transformations to find correspondences between lines. We got encouraging results for the building of atlases of the crest lines of the skull and of the brain based on several CT-Scan and MRI images of different patients.

The implementation and use of a computerized whole-brain mapping system which can be used in conjunction with the major imaging modalities (MR, CT, angiography, PET, SPECT) is described. Three mapping systems (brainstem anatomical, electrophysiological, and proportional whole-brain) based upon common reference structures about the third and fourth ventricular core of the brain are used in conjunction with internationally recognized nomenclature to create a comprehensive normalized whole-brain mapping system. This mapping technique can be universally applied to aid in discerning the surgical and anatomico-physiological topography of the brain for medical imaging.

We describe a system that automates atlas look-up when viewing cross-sectional images at a viewing station. Using simple specification of landmarks a linear transformation to a volume based anatomical atlas is performed. As a result corresponding atlas pictures containing information about structures, function, or blood supply, or classical atlas pages (like Talairach) appear next to the patient data for any chosen slice. In addition the slices are visible in the 3D context of the VOXEL-MAN 3D atlas, providing all its functionality.

In this communication we present a working prototype of a distributed data base of volume data from different forms of 3D imaging. Examples of 3D information both from electron tomography of biological macromolecules and from medical imaging are already included in this development. This type of structural information is crosslinked, whenever applicable, to other sources of information, such as bibliographies. The solution we present here is sufficiently general to be applicable to data in a number of different fields of biomedical science.

An application for design and simulation of conformal radiation therapy is described. The goal is to design a beam configuration that delivers therapeutic dose to the disease and minimizes irradiation of surrounding healthy tissues. The application supports a wide variety of tasks including: creation of patient- specific anatomic models from multimodality medical image data; interactive manipulation of high-level beam constructs; evaluation of calculated dose; and simulation of treatment procedures. The application exploits standard visualization technology as well as custom techniques developed specifically for treatment planning. This application, coupled with new computer-controlled treatment technologies permits design and delivery of radiation treatments that conform high dose to the diseased tissue, while minimizing the dose to surrounding healthy tissue. As a result, larger doses of radiation can now be safely applied than previously possible.

VISTAnet, an experimental gigabit network test bed, ties together a CRAY Y-MP, the Pixel-Planes 5 graphics engine, and an SGI host machine to create a metacomputer capable of real-time radiation therapy dose calculation and display. We report on the methods used to manipulate and examine the 3D radiation dose distribution, with emphasis on the visualization, which uses a parallel, interactive, multimodal renderer implemented on Pixel- Places 5. The real-time display is designed to facilitate comprehension of spatial relationships among the geometrically complex anatomy and radiation dose structures that characterize a 3D radiation treatment scenario. The currently ongoing clinical evaluation of VISTAnet has already yielded encouraging results.

The design of lower limb prostheses requires definitive geometric data to customize socket shape. Optical surface imaging and spiral x-ray computed tomography were applied to geometric analysis of limb residua in below knee (BK) amputees. Residua (limb remnants after amputation) of BK amputees were digitized and measured. Surface (optical) and volumetric (CT) data of the residuum were used to generate solid models and specify socket shape in (SDRC I-DEAS) CAD software. Volume measurements on the solid models were found to correspond within 2% of surface models and direct determinations made using Archimedean weighing. Anatomic 3D reconstruction of the residuum by optical surface and spiral x-ray computed tomography imaging are feasible modalities for prosthesis design.

The design of a custom hip implant can be broken up into several distinct phases which can be carried out by separate toolkits which specialize in the service needed for each stage. We describe a client-server environment in which multiple experts can collaborate in the implant design process and shape optimization, while computations for modelling the femur, simulating loading conditions, and visualization of the results are distributed among a heterogeneous network of workstations.

The major aim of conformal therapy is to engulf the tumor volume by a high radiation dose while minimizing the exposure of the surrounding healthy tissue. Visualization of the tumor volume, dose, and surrounding tissue is an important step in determining if the proposed treatment plan is optimal or even acceptable. This paper investigates the use of interactive volume visualization for the dose volume as well as the anatomical data. A commercially available volumetric processor capable of computing 10 million transformed and trilinearly interpolated points per second is used to generate multiplanar reformatted (MPR) and 3D displays of patient anatomy and radiation dose. The dose is treated as a volume data set in it's own right, which allows for the arbitrary selection of any isodose surface with interactive control of dose level and relative beam weights for multi beam treatment plans. A coarse-to-fine strategy is employed for all viewing modes. MPR views are generated at the rate of approximately 10 frames per second with 3D displays taking approximately 10 seconds per view. Clinically useful images, indicating the variation of radiation dose levels combined with 3D anatomy, can be generated interactively, allowing for the rapid evaluation of complex treatment plans.

Two new interactive visualizations, currently named 'computed fluoroscopy' and 'computed Polaroids' have been added to our 3D radiation therapy treatment planning system. Our preliminary experience with these visualizations suggests that they are of real clinical value.

We demonstrate the use of integrated multi-modality data (MRI, MRA, DSA, PET and live video) and 3D stereoscopic imaging in the context of image-guided neurosurgery. We consider here the integration of anatomical data (MRI), vascular data (DSA and MRA) and functional data (PET) derived from the patient undergoing the surgical procedure. In addition live video images are merged with renderings of the data stored in the computer. The integration of multimodality data provides the surgeon with interactive and intuitive access to a comprehensive overview of the brain structures on which surgery is being performed. Ready access to this information enhances the surgeon's ability to avoid critical vessels and structures of functional significance.

We have combined the use of 3D computer-reconstructed images with a video patient-registration technique to facilitate the guidance of neurosurgery. In addition we have devised an automated image registration process utilizing laser scanning to create a surface model of the patient that can be matched to the 3D computer model.

Image guided neurosurgery is becoming more widely used as conventional stereotactic techniques are replaced by frameless systems which allow flexible but accurate positioning of surgical approach routes with reduced invasiveness. We have developed a surgical planning and guidance system (VISLAN) which uses stereo video to construct an Intra-Operative Patient Representation (IOPR) and to track a hand-help locator in real time. The IOPR is automatically registered with the Pre-Operative Patient Representation (POPR), which is constructed from MR and other modalities and also includes the surgical plan. Images combining POPR and IOPR objects guide the surgeon on his pre-planned path towards the target.

We have designed and implemented a computer-based system that permits rapid acquisition of digital medical images, multi- modality registration and segmentation, and 3D planning of minimally invasive neurosurgical procedures. The system, known as Netra, is optimized for real-time planning: imaging, pre- processing and planning are performed on the morning of surgery in clinically useful times. We have tested the system on procedures such as needle biopsies, depth electrode placements and craniectomies for arteriovenous malformations, aneurysms and tumors. We describe in this paper the core algorithms of our system, and discuss issues related to implementation, validation and user acceptance. We focus on techniques for physician interaction that encourage active participation by the surgeon as principal operator of the visualization and planning system.

A multi-modality image based system is presented allowing for comfortable pre-operative planning and simulation of surgical procedures as well as real-time intra-operative tracking and visualization of surgical instruments. For this purpose a standard surgical tool set is instrumented with light emitting diode (LED) markers to be registered by an opto-electronic space digitization system. As a challenging clinical application within the field of orthopaedic surgery the image guided insertion of spinal pedicular screws is studied. The adaptation to other applications in orthopaedic surgery is currently being developed.

The idea that neurosurgeons can do neurosurgery without being physically present in the operating room is no longer science fiction. With the development of computer—assisted neurological surgery techniques and computer networking capabilities, especially with the upcoming information highway system, tele—presence neurosurgery becomes increasingly realistic. By tele—presence we mean surgeons can remotely monitor, guide, or even do operations while far away from the operating room and the patient (Fig. 1) . Remote monitoring, instrument guiding, and operating are based on a virtual environment (virtual reality) , the environment present in the operating room. Therefore it is also necessary to construct a "virtual patient" for the surgeons at the remote workstation to monitor during surgical procedures. With a passive tele—presence system, neurosurgeons remotely monitor and guide surgery but cannot control the surgical instrument actively. The surgery is done by an assistant surgeon inside the operating room. With an active tele— presence system, a neurosurgeon at a remote site can actively control the movement of surgical instruments. Current medical imaging technology together with correlation strategies, are the foundation of building the virtual patient. Biornedical image visualization as well as surgery planning system offer great surgeon—machine interfaces. Tele—presence can be both passive and active. Tele—presence in neurological surgery has important applications also in education and training. And with the improvement of national information highway system, tele—presence neurosurgery definitely will play an very important role in the medical information highway system. The first part of this paper discusses in detail the fundamental theory involved in a tele—presence surgery system. The discussion centers around the issue of system architecture (software module, hardware setup) , and detailed functionalities for each module are also discussed. The second part of the paper discusses a prototype system in development at Wayne State Universtty Medical Center. The experimental results are also given in this part.

Honeywell conducted two display simulations and evaluations to define design requirements for a stereoscopic, head-mounted, surgical display. The use of stereoscopic imagery resulted in striking improvements in surgical performance, as well as lending a degree of novelty and excitement to the procedure. Surgeons indicated they anticipate many benefits of such a system.

We present a craniofacial surgery simulation testbed that makes extensive use of virtual reality techniques. The skull, skin and fat tissues are represented with simplex meshes, that are characterized with a constant vertex to vertex connectivity. Surfaces and volumes are respectively described as three and four connected meshes. This representation is well suited for the implementation of surface deformations such as those exerted on the face skin under the action of fat tissues. Furthermore, cutting surface regions may be easily achieved due to the local nature of simplex meshes. The user proceeds by cutting skull fragments and reorganizing them with the help of a virtual hand. Fat tissue attached to both skin and skull adjusts the face shape to the reconstructed skull.

A computer vision approach for the extraction of feature points on 3D images of dental imprints is presented. The position of feature points are needed for the measurement of a set of parameters for automatic diagnosis of malocclusion problems in orthodontics. The system for the acquisition of the 3D profile of the imprint, the procedure for the detection of the interstices between teeth, and the approach for the identification of the type of tooth are described, as well as the algorithm for the reconstruction of the surface of each type of tooth. A new approach for the detection of feature points, called the watershed algorithm, is described in detail. The algorithm is a two-stage procedure which tracks the position of local minima at four different scales and produces a final map of the position of the minima. Experimental results of the application of the watershed algorithm on actual 3D images of dental imprints are presented for molars, premolars and canines. The segmentation approach for the analysis of the shape of incisors is also described in detail.

Software to construct polygonal models of anatomical structures embedded as isosurfaces in 3D medical images has been available since the mid 1970s. Such models are used for visualization, simulation, measurements (single and multi-modality image registration), and statistics. When working with standard MR- or CT-scans, the surface obtained can contain several million triangles. These models contain data an order of magnitude larger than that which can be efficiently handled by current workstations or transmitted through networks. These algorithms generally ignore efficient combinations that would produce fewer, well shaped triangles. An efficient algorithm must not create a larger data structure than present in the raw data. Recently, much research has been done on the simplification and optimization of surfaces ([Moore and Warren, 1991]); [Schroeder et al., 1992]; [Turk, 1992]; [Hoppe et al., 1993]; [Kalvin and Taylor, 1994]). All of these algorithms satisfy two criteria, consistency and accuracy, to some degree. Consistent simplification occurs via predictable patterns. Accuracy is measured in terms of fidelity to the original surface, and is a prerequisite for collecting reliable measurements from the simplified surface. We describe the 'Wrapper' algorithm that simplifies triangulated surfaces while preserving the same topological characteristics. We employ the same simplification operation in all cases. However, simplification is restricted but not forbidden in high curvature areas. This hierarchy of operations results in homogeneous triangle aspect and size. Images undergoing compression ratios between 10 and 20:1 are visually identical to full resolution images. More importantly, the metric accuracy of the simplified surfaces appears to be unimpaired. Measurements based upon 'ridge curves; (sensu [Cutting et al., 1993]) extracted on polygonal models were recently introduced [Ayache et al., 1993]. We compared ridge curves digitized from full resolution, Wrapper, and volume subsampled CT-scan isosurfaces. [Dean, 1993] introduced a method for measuring distances between space curves. In the best case this method demonstrated that ridge curves digitized from the Wrapper simplified images were two orders of magnitude closer to the full resolution image than those taken from the volume subsampled images.

Algorithms that tile iso-valued surfaces should produce 'correctly' tiled orientable manifold surfaces. Rigorous evaluation of different algorithms or case tables has been impossible up to now because of the lack of a clear and comprehensive theoretical framework. We propose the extension of Morse theory, as developed in the continuous domain, to apply to discretely sampled continuous domains (sampled Morse functions). We call this the digital domain, and thus formulate a Digital Morse Theory (DMT). We show that a discretely sampled continuous volume in which we are tiling a surface can have various classes of Morse criticalities. When the isosurface comes close (within a voxel length) of a criticality, this is what gives rise to certain apparent ambiguities in tiling surfaces. DMT provides a heuristic to correctly disambiguate tiling decisions. In addition, DMT gives insight to correctly simplifying a volume data set so as to produce an isosurface with a reduced number of tiles, and yet maintain a topology. Additionally, one can establish a hierarchical relationship between each criticality and its associated regions.

Goal: To estimate hippocampal volumes from in vivo 3D magnetic resonance (MR) brain images and determine inter-rater and intra- rater repeatability. Objective: The precision and repeatability of hippocampal volume estimates using stereologic measurement methods is sought. Design: Five normal control and five schizophrenic subjects were MR scanned using a MPRAGE protocol. Fixed grid stereologic methods were used to estimate hippocampal volumes on a graphics workstation. The images were preprocessed using histogram analysis to standardize 3D MR image scaling from 16 to 8 bits and image volumes were interpolated to 0.5 mm³ isotropic voxels. The following variables were constant for the repeated stereologic measures: grid size, inter-slice distance (1.5 mm), voxel dimensions (0.5 mm³), number of hippocampi measured (10), total number of measurements per rater (40), and number of raters (5). Two grid sizes were tested to determine the coefficient of error associated with the number of sampled 'hits' (approximately 140 and 280) on the hippocampus. Starting slice and grid position were randomly varied to assure unbiased volume estimates. Raters were blind to subject identity, diagnosis, and side of the brain from which the image volumes were extracted and the order of subject presentation was randomized for each of the raters. Inter- and intra-rater intraclass correlation coefficients (ICC) were determined. Results: The data indicate excellent repeatability of fixed grid stereologic hippocampal volume measures when using an inter-slice distance of 1.5 mm and a 6.25 mm² grid (inter-rater ICCs equals 0.86 - 0.97, intra- rater ICCs equals 0.85 - 0.97). One major advantage of the current study was the use of 3D MR data which significantly improved visualization of hippocampal boundaries by providing the ability to access simultaneous orthogonal views while counting stereological marks within the hippocampus. Conclusion: Stereological estimates of 3D volumes from 2D MR sections provide an inexpensive, unbiased and efficient way of determining brain structural volumes. The high precision and repeatability demonstrated with stereological MR volumetry suggest that these methods may be efficiently used to measure small volume reductions associated with schizophrenia and other brain disorders.

A new technique has been developed for the kinematic analysis of joints. It is based upon 3D reconstructions of the bones acting at a joint, from magnetic resonance image data, as the joint moves through its range of motion. The technique has been applied to the subtalar and transverse tarsal joints of the foot, but it can be used in the analysis of any skeletal joint. This method provides a description of the rotations and translations which occur as the bones move from one position to another. It also provides related data, such as a characterization of the contact areas between the bones.

To date, 3D ultrasound imaging has been hampered by fractionated, job specific computer procedures, and the need for significant operator interaction. This paper presents our data processing algorithm for cardiac structure visualization from serial transesophageal echocardiographic (TEE) images. Major steps in the algorithm are: 1) image registration, 2) histogram operations for contrast enhancement, 3) noise and speckle filtering, 4) segmentation of composite color Doppler flow images, and 5) coordinate system conversion and interpolation. Three-dimensional reconstructions of clinical TEE examinations were compared to the corresponding serial 2D scans using receiver operator characteristics (ROC) analysis. The results demonstrated significantly better trade-off between diagnostic sensitivity and specificity in the 3D method when compared to the original 2D tomograms. We conclude that 3D echocaridography based on our algorithm is clinically feasible. Moreover, the ROC analysis on a limited group of patients indicated that 3D imaging facilitated comprehension of complex anatomic relationships and diagnostic capabilities of conventional 2D TEE echocardiography.

A formal representation of neuron morphology, adequate for the geometric modeling of manually-traced neurons, is presented. The concept of a stochastic L-system is then introduced and the critical distribution functions governing the stochastic generation of dendritic and axonal trees are defined. Experiments with various stochastic L-system models for pyramidal, motoneuron, and Purkinje cells are reported which generate synthetic neurons with promising proximity to neurons in the neurobiology literature. Work is in progress to improve this degree of proximity, but more importantly to validate the derived stochastic models against available databases of manually-traced neurons. To this end a neuron morphology modeler is described which provides a methodology for iterative refinement of the stochastic L-system model.

Transmission Electron Microscopy allows the visualisation of cell organelles and intracellular structures at high magnification. But this technique is physically limited to ultrathin sections of about 20 to 100 nm because the electrons have to get through the biological structures to image them. In order to reconstruct the third dimension, we have previously developed a method based on laser topographical references to correct the physical deformations linked with the sectioning process and to allow the fine registration of the images1. The volume information is reconstructed but the intracellular organelles which are limited by membranes, are still embedded in a matrix of opaque cytoplasm containing dense ribosomes and fibrils. The rendering of the surface, volume and the original 3D appearance of each organelle requires individual segmentation. This operation is presently the only conceivable way to visualise the internal organisation of cells. We expose here some methods and algorithms for extraction of organellescontours and their subsequent preliminary visualisation. The algorithms allow the representation of the internal cell structure and pave the way toward virtual immersion.

In this paper, we present a semi-automatic technique to quantize and to visualize myocardial perfusion defects volumetrically. This involves: (1) differentiating the myocardium from encroaching viscera, (2) determining the mass of myocardium, (3) locating and measuring the perfusion defects, and (4) graphically rendering them. First, to differentiate between myocardium and other thoracic structures, we introduce the concept of Maximum Intensity Surface Segmentation (MISS). By locating local maximums in our cardiac Single Photon Emission Computerized Tomography (SPECT) image and connecting ones sufficiently close together, we generate the basic shape of the myocardium and the encroaching ones sufficiently close together, we generate the basic shape of the myocardium and the encroaching structures. Second, with the shape of the myocardium determined, we then measure the thickness at each point of the heart wall. Fitting bicubic splines to the measured myocardial thicknesses, we create smooth epicardial and endocardial boundaries for both normal and abnormal regions. Third, using a thresholding technique based on the maximal myocardial activity, we determine the abnormal regions. Finally, with the abnormal regions and the myocardial mass determined, we can calculate the abnormality's size as a percentage of the total myocardial mass. By providing quantitative measurements, our technique meets the physician's long standing need for a volumetric quantitation of the size of myocardial perfusion abnormalities.

Gastric motion after the ingestion of a radioactively labeled standard meal was visualized using a triple headed gamma camera and dynamic SPECT acquisitions consisting of 30 scans of 6 s duration each. After the ingestion of a radiolabeled standard meal tomographic reconstruction produced, after prefiltering with a Metz filter, images of reasonable quality, in spite of the short acquisition time per view. Oblique slices rectangular to the longitudinal axis of the antrum were positioned employing 3D rendering techniques. These slices were extracted from the sequential volumes to produce time activity curves (TACs) of antral contractions. From the TACs the amplitudes and the frequencies of the antral contraction curves obtained from planar acquisitions, were markedly higher for the curves from the tomographic slices. This was due to the removal of oblique components of motion in the oblique slices. The effect of the long sampling interval of 6 seconds was checked on simulations using data from planar acquisitions and found to permit sampling of the antral waves with acceptable accuracy. 3D display of the stomach contributed to the anatomical knowledge since it showed clearly differences to the positions that would have been expected from conventional x-ray views. Antral contraction strength was not necessarily related with the rate of gastric emptying, which emphasizes the role of other factors, mainly the tone of the fundus, for the emptying process.

Neurons in the myenteric plexus have been studied for many years but their 3D structure is largely unknown. We used a laser scanning confocal microscope (LSCM) to obtain serial, optical sections of these cells and ANALYZE software to process the images into 3D. The 3D reconstructions showed that the neurons had a variety of shapes in cross sections, unlike the previously presumed flat shape. The short processes of Dogiel type I cells that have been considered to be very flat appeared to have a nearly circular cross section. The processes of the neurons lay in a single narrow area and thus appeared not to make vertical connections in their parent ganglia. The surface areas and volumes were calculated for both the cell bodies and processes. These data have not been available from conventional microscopy and they demonstrate the power of this method for analyzing neuronal morphology.

This tutorial focused on the application ofnew visualization techniques to the practical problems confronting the neurosurgeon. In particular, the challenging issues associated with presurgical planning and with intra-operative navigation were presented in a tutorial format. The course began with a discussion of algorithms and hardware for intra-operative registration. Traditional frame-based stereotactic techniques were introduced, then several classes of non-invasive registration techniques, including fiducial based algorithms, surface and contour matching algorithms and video mixing approaches were described and compared. The "surgeon-computer interface" was discussed, including issues associated with time constraints, sterility problems, electrical safety, and software validation. Videotaped surgical procedures demonstrating these concepts were shown. The tutorial concluded with a review of various technologies for intraoperative registration and navigation. Mechanical, visible and infra-red camera systems, and ultrasonic techniques were compared. Attendees had the opportunity to gain hands-on experience with several intra-operative wands and with a prototype heads-up-display guidance system.

There are two basic principles associated with multiscale geometry: 1) geometry involves analysis that is invariant to certain spatial transformations, including translation, rotation, and zoom, and 2) the dimension of scale is as critical as the dimensions giving spatial position, corresponding intuitively to level of detail in image space. Considered together, image space and scale is called scale space. Three families of methods based on these principles are achieving impressive results in image analysis —particularlyin their insensitivity to irrelevant detail (including image noise) and intensity blurring, and in their ability to produce stable object descriptions and pixel classifications into objects. The three families are multiscale medial axis or core-based analysis (CBA), variable conductance difftision (VCD), and multiscale geometric statistical pattern recognition (MGSPR). This pair of tutorials (morning and afternoon) covered the basic mathematics of multiscale geometry as well as all three of these families of methods. It included algorithms for computation and also illustrative results of the applications of the methods to both 2D and 3D medical images of various modalities. The morning tutorial covered the mathematics of diffusion and scale space and the definition, effect on scale space geometry, and application of cores. The afternoon tutorial covered the mathematics, algorithms, and applications of variable conductance diffusion, including approaches involving MGSPR, and it covered algorithms for segmenting objects both via VCD and via CBA.

Based on the nature of the processing tools required to fulfill biomedical imaging objectives, three components may be identified in a complete image analysis system: visualization, manipulation, and analysis. Visualization refers to processes which enable us to see and comprehend structure information captured in the image data in its true form and shape so as to help understand the underlying physical phenomenon. Manipulation refers to processes which allow us to interactively alter structure information with a view to understanding how the functionality of the physical phenomenon may be modified. Analysis refers to processes which generate quantitative descriptions of the structures with a view to quantify the functionality of the phenomenon being studied. The phrase 3D imaging may be used to collectively denote these processes. This tutorial covered systematically the fundamental principles underlying 3D imaging, namely multidimensional, multimodal image data visualization, manipulation, and analysis. The tutorial included on-line demonstrations of the imaging operations on a workstation.

The essence of scientific visualization is captured in microscopy. Microscopy systems are a proven valuable tool for biomedical research. Most of our present knowledge of structural organization in living organisms from the cellular to the molecular level is imprinted by microscopical findings. This tutorial provided an introduction into modem forms of 3-D microscopy (e.g., confocal, NMR, X-ray), and covered image quality considerations, microscopy setup, and computational considerations. Deconvolution techniques, image preprocessing, segmentation methods and various forms of 3-D rendering were reviewed. The effects of specimen preparation and image contrast on visualization were discussed. The use of stereo-imaging and virtual reality applications in modem 3-D microscopy was presented.

Volume visualization encompasses an array of techniques which provide the mechanisms that make it possible to reveal and explore the inner or unseen structures of volumetric data and allow visual insight into opaque or complex datasets. Volume visualization is concerned with the tasks of representing, manipulating, and rendering volumetric data. This tutorial provided an overview of the technology, the nomenclature, and the techniques for these tasks, emphasizing algorithms and applications. The tutorial covered and compared different approaches in volume representation, volume synthesis, volume and surface viewing, volume shading, and biomedical applications of volume visualization.