Recently, after two decades of research in visual texture discrimination and in the discrimination of briefly presented patterns, a new structure of human vision has emerged. Accordingly, human vision is mediated by two separate visual systems: a preattentive and an attentive one. The parallel preattentive visual system cannot process complex forms, yet can, almost instantaneously, without effort or scrutiny, detect differences in a few local conspicuous features, regardless of the number of elements or where they occur. These "textons" (Julesz, 1981) are elongated blobs (e.g. rectangles, ellipses, or line segments) of specific color, angular orientation, width, length, binocular and movement disparity, and flicker rate. For line segments, their terminators (ends-of-lines) and crossings often behave as textons. The preattentive system can count only the number of textons and detect their locale, but cannot perceive the positional relations between them (Julesz, 1981, 1981a). However, the preattentive system can direct focal attention to the areas where differences in texton density occur in about 50 msec steps (four times faster than eye-movements). Only in this narrow aperture of focal attention (which can be as small as a few minutes of arc) can positional differences be perceived, permitting the prodigious feats of form recognition (Bergen and Julesz, 1981). Since disparity differences in random-dot stereograms can be perceived in a 50 msec presentation (followed by masking) provided the binocular disparity is beyond a critical limit, disparity is also a texton. It can be shown (for review, see Julesz and Schumer, 1981) that the largest disparity that can yield stereopsis in random-dot stereograms is a monotonic function of the cyclopean target area. Furthermore, the cyclopean modulation function (for corrugated sinusoidal depth gratings) is very "myopic" with a cit-off around 3 c/deg. Since stereopsis is important in quality inspection, an objective determination of stereoscopic abilities using evoked potential techniques to dynamic random-dot correlograms and stereograms is also discussed (Julesz, Kropfl and Petrig, 1980; Julesz and Kropfl, 1982). These techniques also permit the early diagnosis of stereo-blindness that afflicts 2% of the human population and might lead to its prevention.

We discuss computer techniques fordisplaying three-dimensional (3D) data using a two-dimensional (2D) display screen, with special emphasis on the representation of the data. Since our expertise is in medical imaging, we restrict our attention to techniques that have been found useful in that area. The first section gives a classification of methods that have appeared in the literature. The remaining two sections deal with specific topics which are of current interest to us: one with techniques of fast display with hidden surface removal and the other with a new approach to the representation and manipulation of three-dimensional scenes.

Holography's effectiveness as a visual medium is often constrained by the variety of scenes that it can render. Here we survey progress over the past fifteen years in the synthesis of quasi-holographic images from sequences of discrete perspective views, such as photographs and X-ray images. Paralleling the development of display holograms, holographic stereograms have evolved from laser illumination to white-light illumination, with improvements in the image quality and ease of viewing.

A survey of stereoscopic and autostereoscopic displays and their working principles is presented, including references to recent accomplishments in the various display fields.

A working prototype three-dimensional television display has been constructed and initial feasibility studies have been carried out. The system employs the time division technique which takes advantage of the interlace facility on the standard television display and incorporates electro-optic viewing spectacles for the observer. Improvements to the basic system have been carried out which include remotely triggered viewing spectacles, thus giving greater freedom of movement to the observer and also an increased switching rate to reduce the effects of flicker. It soon became apparent that a system containing an integral real time depth measurement facility would be attractive to potential users. The method used is the application of a photogrammetic technique to the display in order to provide a real time three dimensional viewing and depth measurement facility. Present available techniques involve processing a stereoscopic pair of photographs which introduces an obvious inherent time delay. For many applications a real time technique has advantages even though the depth resolution is unlikely to be as good as the standard photographic system.

Design and implementation of a viewpoint Dependent imaging system is described. The resultant display is an interactive, lifesize, stereoscopic image. that becomes a window into a three dimensional visual environment. As the user physically changes his viewpoint of the represented data in relation to the display surface, the image is continuously updated. The changing viewpoints are retrieved from a comprehensive, stereoscopic image array stored on computer controlled, optical videodisc and fluidly presented. in coordination with the viewer's, movements as detected by a body-tracking device. This imaging system is an attempt to more closely represent an observers interactive perceptual experience of the visual world by presenting sensory information cues not offered by traditional media technologies: binocular parallax, motion parallax, and motion perspective. Unlike holographic imaging, this display requires, relatively low bandwidth.

A laser range sensor directly produces 3-D information of a scene. Each pixel of a scanned scene is the range from the sensor to the nearest scene point at a particular azimuth and depression angle. This range information can be used to identify 3-D features in the scene which may be associated with buildings, roads, vehicles, etc. Such 3-D features can be identified easily and reliabily with range images, rather than using techniques based on 2-D images (such as stereo). This paper describes the approaches developed at the Lockheed Signal Processing Laboratory for identifying features in range images of buildings and vehicles. In the building scenario the features are matched to a model for either a terminal homing application or for a navigation update. In the vehicle scenario particular vehicles must be identified from a background of potentially rough terrain containing other vehicles as well as trees, shrubs, rocks, etc.

An n-dimensional Algebra computer code, NDIMEN, based on the Generalized Balanced Tern-ary (GBT) system is applied to the processing of "Battlefield Obscuration"--Dust, Smoke, Artillery Bursts. The obscurant data is obtained from the Atmospheric Sciences Laboratory's Multispectral Image Data Acquisition System (MIDAS), which video records multi-wavelength time-sequenced images of the obscurant cloud from a stereo-pair or more MIDAS camera stations. Examples of the GBT algebra and its application to 3-D objects plus the potential for modeling combat clouds will be given.

A theoretical discussion of a method of reducing redundancy from spatial image data is in-cluded. An approach to the reduction of bandwidth for multiple image three-dimensional television, is developed through the use of data gathered about subject's spatial properties, as well as conventional image properties. Types of apparatus suitable for gathering spatial and image data as they pertain to methods of reducing redundancy are discussed. A wave form derived from spatial image data is suggested and analyzed with reference to its usefulness in the transmission and storage of spatial image data. A brief description of the authors experimental work in the development of reproduction geometry for three-dimensional television displays that are viewed without special glasses of any kind and display parallax is provided. A discussion of theoretical advantages of two-dimensional image synthesis from three-dimen-sional spatial image data is provided. The use of such image synthesis in connection with the aforementioned transmission wave form allowes certain developments of existing television displays to be proposed as part of a practicable system.

Multidimensional (nD) signals can be replaced by 2D sequences of sectional images if they are bandlimited in all but two dimensions. Thus nD signal processing can be replaced by processing these 2D sequences, e.g. by optical computers. Examples are given for 3D spectrum analysis, interpolation and rearrangement of coordinates.

Recent developments in computer graphics and computer animation technology have provided researchers with sophisticated and powerful methods of displaying technical data. These methods are generally limited, however, to two-dimensional formats, and there is a growing need for three-dimensional display of information. The cylindrical holographic stereogram is an interesting medium which shows some promise for three-dimensional data display. This paper examines some of the possibilities and problems of this application of the holographic stereogram.

Since the mid 1930's crystallographers have studied the structures of the important biological polymers: proteins, nucleic acids, and polysaccharides. The first 3D structure, myoglobin, was solved by multiple isomorphous replacement (MIR) in 1957. Today we know the structure of over 200 proteins and the list is growing rapidly. Molecular biologists' efforts in obtaining and crystallising proteins have been much more fruitful than their efforts on nucleic acids, mostly because proteins are much more plentiful. There are currently less than twenty single-crystal DNA structures known and a similar number of RNA structures.

Seismic data for a three-dimensional (3-D) marine survey were simulated with a scaled physical modeling system and used to illustrate an interpretational procedure based on a vector refresh graphics system. The physical model included stratigraphic and structural features. A 3-D migration of the raw data formed the main data base for interpretation. Interpretation progressed by displaying seismic sections (vertical and horizontal) on a graphics screen in raster format and drawing interpretational lines using a data tablet controlled cursor in a vector format. The accumulated line drawings were displayed as a rotatable, 3-D drawing from which the three-dimensional aspects of the geologic features could be appreciated. Selected drawings and raster displays were produced in real time on an electrostatic plotter; color prints and high resolution sections were output through the host computer as batch jobs. An important element in the procedure was the number of grey levels used for the displays. Using two grey levels (sign-bit sampling) the whole data volume can be scanned rapidly. Using 64 grey levels each image is full of detail, but the cycle time to a new image is long.

This paper describes some current uses and needs of computer animation in meteorological research. Specific emphasis is placed upon practical experience gained from analysis and depiction of three-dimensional fields. Examples will be drawn from recent research by Dr. R. M. Chervin of the National Center for Atmospheric Research (NCAR) and me. All of the graphics were produced using the computing facility at NCAR.

The advent of the computer aided tomographic x-ray scanning system (CT scanner) has marked a new era of medical imaging modalities being developed for radiology departments of the future. Two fundamental elements common to all of these rapidly developing technologies are the ability of the technology to collect data over an entire three-dimensional volume of interest and the necessity for a digital computer to convert the acquired data into useful images. Our intention is to explore the advantages and limitations of a general digital holographic approach to the display of the reconstructed volume data. In this paper we: 1) describe a model of the three-dimensional data volume; 2) derive an algorithm to independently compute portions of the Fresnel diffraction pattern of the model; 3) discuss the com-putational and recorder requirements to produce the hologram; 4) show examples of holograms computed by our method and 5) discuss these results.

We are developing graphics systems, image preprocessing methods, and interactive manipulation techniques for a space-filling 3D display using a varifocal mirror principle. Our driving problem is a medical imaging need for presentation of three-dimensional intensity information. The major goal of both the image preprocessing and the interactive manipulation has been to overcome obscuration, which we feel is coming to be recognized as the central problem in any space-filling display. In our system, the preprocessing step highlights important image features such as surfaces. At display time, the object can be dynamically edited and rotated for convenient viewing from various directions. Our particular hardware design allows the 3D display to be constructed as an inexpensive add-on to a standard video graphics system. The interactive rotation and other manipulations are achieved by the standard built-in graphics processor.

This paper discusses digital perspective correction in the multiplex hologram,also known as Cross hologram or white light cylindrical holographic stereogram. It presents digital analogues of previously reported optical methods, and demonstrates the digital implementation of Benton's method. It introduces a non-linear remapping of the input views to compensate for a non-uniform color viewing position. Simulation results are included on the relative accuracy of different algorithms. Digital correction can be applied to both real and artificial objects such as computed tomography (CT) data.

The display of data in three dimensions overcomes the ambiguity often found in two dimensional displays. It allows a truly objective examination of the display data while two-dimensional displays require a subjective interpretation of what might exist in the Z direction. SpaceGraph allows the display of data in a volume filling manner. The data can occupy a volume of 20 x 25 x 30 centimeters. The display volume is generated by observing the reflection of a CRT in a circular mirror. This mirror is flexed about a rubber hinge located on a concentric circle several inches from the edge. By exciting this assembly with a hi-fi woofer, the mirror is caused to vibrate and takes on concave and convex optical shapes thus varying the focal length. This varying focal length causes the image of the CRT to sweep out apparent distance in Z of about 30 centimeters. By plotting points on the CRT in X and Y, these points permit us to draw vectors which can describe a wide variety of three-dimensional objects, such as mechanical parts, molecules, mechanical subassemblies or total assemblies such as aircraft and ships. In the vector mode, SpaceGraph provides 23 meters of vectors which can appear in as many segments as required by the object being displayed. The three-dimensional display can also be used in a second mode which we will call the image mode. In this mode, X and Y are controlled to generate a raster much like one generated in a conventional home TV. While the raster is being swept, brightness is varied to provide an image in gray shades. As this process takes place, the Z is continuously swept by the mirror as in the vector mode and a volume filling image is created. This mode appears to be of particular interest to seismologists and in computer aided tomography. Other applications or views of science which appear promising for 3-D display are computer-aided design, air traffic control, ultra sound analysis and anti-submarine warfare.

A verifocal mirror display system has been constructed at the University of Utah for medical imaging applications. The device has undergone trials in the Department of Radiology with X-ray CT, ultrasound, and NMR data. Potential applications for an improved portable display system are described.

Although the process of computed axial tomography (CAT) yields data from a sample volume, the possibility of fully three-dimensional imaging is not yet exploited by present tomographic systems, which produce a set of planar images. Data from contiguous cross-sectional scans can be combined to provide simultaneous direct three-dimensional visualization of the volume. CT data can be used as an input to several holographic display techniques. We present results of one such technique, the incoherent superposition hologram made from individual planar CT cross-sections. The holographic images are arranged in order corresponding to the method in which they were recorded by the x-ray scanner. Incoherent superposition holography is an additive three-dimensional display, the eye views a volume of data as the sum of light sources from each image. Structure in one plane is less obscured by structure in other planes as compared to coherent holography, of nonholographic displays where light reaching the eye has been attenuated in a multiplicative manner. This type of hologram provides fully three-dimensional depth cues and the ability to focus onto individual cross-sectional images. The recording medium for the incoherent super-position hologram is a silver halide emulsion. The experimental procedure for synthesizing large format holograms is discussed, in particular the method for superimposing holograms that reconstruct images of equal brightness. A method for optimizing this procedure is presented along with the actual incoherent superposition holograms of CT data.

Work is described which merges 3-D input and display technologies to implement a real time interactive computer graphics environment, generating a stereoscopic image that one creates by reaching directly into it. The display is an ordinary video monitor, reflected off a half-slivered mirror, and viewed through PLZT opto-electrical shutter glasses. For input, an electromagnetic six degree-of-freedom digitizer is mounted in a small wooden "magic wand", with a pushbutton mounted in the handle.

A common requirement in remote sensing and cartography applications is that of warping an image to register with a reference image. A second order polynomial coordinate transformation on two-dimensional images embodies translation, rotation, scaling, and several types of warping. To perform this operation efficiently and with minimal hardware, the transformation is separated into a line operation followed by a column operation. These two transformations are computed using a least square fit algorithm. Resampling with bilinear or cubic convolution interpolation is also separable into the two-pass technique. This algorithm is suitable for hardware implementation of current image processing systems.