There are significant international efforts underway to place mobile robots (`rovers') on the surface of Mars. This follows on the recent successful NASA Mars Pathfinder flight of summer 1997. In that mission, the 11+ Kg Sojourner rover explored a small 50 meter locale about its lander over a several week period. Future planned science missions of the Mars Surveyor Program are more aggressive, seeking to autonomously survey planetary climate, life and resources over multiple kilometers and many months duration. These missions will also retrieve collected sample materials back to a Mars Ascent Vehicle for more detailed analysis on Earth. In support of these future missions we are developing and field testing new rover technology concepts. We first overview the design and initial operations of SRR-1 (Sample Return Rover), a novel 10 kg-class four wheel, hybrid composite-metal vehicle for rapid (10 - 30 cm/sec) autonomous location, rendezvous, and retrieval of collected samples under integrated visual and beacon guidance. SRR is a light 88 X 55 X 36 (LWH) cm<sup>3</sup> vehicle collapsing to less than one third its deployed field volume, and carrying a powerful, visually-servoed all-composite manipulator. We then sketch development of the FIDO rover (Field Integrated Design and Operations), a new 50+ kg, six wheel, approximately 100 X 80 X 50 (LWH) cm<sup>3</sup>, high mobility, multi-km range science vehicle which includes mast-mounted multi-spectral stereo, bore-sighted IR point spectrometer, robot arm with attached microscope, and body- mounted rock sampling corer. Currently in integration phase, FIDO rover will first be tested in September, 1998, `MarsYard (JPL)' operations, followed by CY99 full-scale terrestrial field simulations of a planned Mars '03 multi- kilometer roving mission (Athena-based science rover payload), demonstrating remote science selection, autonomous navigation, in situ sample analysis, and robotic sample collection functions.

A new nonlinear feature extraction method is presented for classification and pose estimation of objects from single views. The feature extraction method is called the maximum representation and discrimination feature (MRDF) method. The nonlinear MRDF transformations to use are obtained in closed form, and offer significant advantages compared to nonlinear neural network implementations. The features extracted are useful for both object discrimination (classification) and object representation (pose estimation). We consider MRDFs on image data, provide a new 2-stage nonlinear MRDF solution, and show it specializes to well-known linear and nonlinear image processing transforms under certain conditions. We show the use of MRDF in estimating the class and pose of images of rendered solid CAD models of machine parts from single views using a feature-space trajectory neural network classifier. We show new results with better classification and pose estimation accuracy than are achieved by standard principal component analysis and Fukunaga-Koontz feature extraction methods.

An intelligent robot is a remarkably useful combination of a manipulator, sensors and controls. The use of these machines in factory automation can improve productivity, increase product quality and improve competitiveness. This paper presents a discussion of recent technical and economic trends. Technically, the machines are faster, cheaper, more repeatable, more reliable and safer. The knowledge base of inverse kinematic and dynamic solutions and intelligent controls is increasing. More attention is being given by industry to robots, vision and motion controls. New areas of usage are emerging for service robots, remote manipulators and automated guided vehicles. Economically, the robotics industry now has a 1.1 billion-dollar market in the U.S. and is growing. Feasibility studies results are presented which also show decreasing costs for robots and unaudited healthy rates of return for a variety of robotic applications. However, the road from inspiration to successful application can be long and difficult, often taking decades to achieve a new product. A greater emphasis on mechatronics is needed in our universities. Certainly, more cooperation between government, industry and universities is needed to speed the development of intelligent robots that will benefit industry and society.

Moments are widely used in pattern recognition, image processing and computer vision. To clarify and to guide the use of different types of moments, we present, in this paper, a comparison study of the behavior of different moments. After a brief introduction to geometric, Legendre, Hermite and Gaussian-Hermite moments, we analyze at first their behavior in spatial domain. Our analysis shows orthogonal moment base functions of different orders have different number of zero crossings and very different shapes, therefore they can better separate image features based in different nodes, which is very interesting for pattern analysis and shape classification. Moreover, Gaussian-Hermite moment base functions are much more smoothed, they are thus less sensitive to noise and avoid the artifacts introduced by window function discontinuity. We then analyze the spectral behavior of moments in frequency domain. Theoretical and numerical analyses show that Legendre and Gaussian-Hermite moments of different orders separate different frequency bands more effectively. It is also shown that Gaussian-Hermite moments give an approach to construct orthogonal features from wavelet analysis results. The orthogonality equivalence theorem is also presented. Our analysis is confirmed by numerical results, which are reported also.

In this paper, a generalization of Shen-Castan and Canny- Deriche filters in a family of edge detector is presented. This family is built on the choice of an exponential model, now well known in edge detection. It is shown that the best detector is theoretically the filter of index 2 of the family obtained. However, each filter presents interesting performances and the choice of one of them depends on the characteristics of the image and the aim pursue. To help this choice the resolution power of the detector called as the separating power is measured. In this way, the compromise between theoretical performances (isolated edge) and practical performances is discussed.

We present results that compare the performance of Bayesian learning methods for neural networks on the task of classifying forest scenes into trees and background. Classification task is demanding due to the texture richness of the trees, occlusions of the forest scene objects and diverse lighting conditions under operation. This makes it difficult to determine which are optimal image features for the classification. A natural way to proceed is to extract many different types of potentially suitable features, and to evaluate their usefulness in later processing stages. One approach to cope with large number of features is to use Bayesian methods to control the model complexity. Bayesian learning uses a prior on model parameters, combines this with evidence from a training data, and the integrates over the resulting posterior to make predictions. With this method, we can use large networks and many features without fear of overfitting. For this classification task we compare two Bayesian learning methods for multi-layer perceptron (MLP) neural networks: (1) The evidence framework of MacKay uses a Gaussian approximation to the posterior weight distribution and maximizes with respect to hyperparameters. (2) In a Markov Chain Monte Carlo (MCMC) method due to Neal, the posterior distribution of the network parameters is numerically integrated using the MCMC method. As baseline classifiers for comparison we use (3) MLP early stop committee, (4) K-nearest-neighbor and (5) Classification And Regression Tree.

A truly autonomous robot must sense its environment and react appropriately. These issues attain greater importance in an outdoor, variable environment. Previous mobile robot perception systems have relied on hand-coded algorithms for processing sensor information. Recent techniques involve the use of artificial neural networks to process sensor data for mobile robot guidance. A comparison of a fuzzy logic control for an AGV and a neural network perception is described in this work. A mobile robot test bed has been constructed using a golf cart base. The test bed has a fuzzy logic controller which uses both vision and obstacle information and provides the steering and speed controls to the robot. A feed-forward neural network is described to guide the robot using vision and range data. Suitable criteria for comparison will be formulated and the hand-coded system compared with a connectionist model. A comparison of the two systems, with performance, efficiency and reliability as the criteria, will be achieved. The significance of this work is that it provides comparative tradeoffs on two important robot guidance methods.

A new technique is discussed to estimate the pose (orientation) of unknown human faces from 2D gray-scale images and then transform the unknown face to a reference pose using a nonlinear feature extraction procedure. This feature extraction scheme is known as the maximum representation and discrimination feature (MRDF) method. The MRDF is shown to provide good features for discrimination between classes that is useful for pose estimation, and for object representation that is useful for pose transformation of an unknown face. Our approach is computationally efficient and has numerous potential applications, including post-invariant face recognition.

A surface inspection problem is divided into three parts, into an image acquisition part, into a defect detection part that is suitable for hardware implementation, and into a defect classification part that is done in a user's terminal. In the defect detection part extraction of texture features is done and potential defect areas are marked. The proposed scheme is taught only with examples of fault-free surface. In the defect classification part features describing the shape and internal structure of defects are extracted and defects are classified into different defect classes. Examples of defects are used to train the classification system. Use of the self-organizing map in defect detection and in defect classification makes the proposed method adaptable to different types of surfaces and to different types of defects. Only reselection of features may be necessary to cope with different surface and defect characteristics. The results of experiments with base paper samples are encouraging.

This research focuses on applications based on wireless monitoring and robot control, utilizing motion image and augmented reality. These applications include remote services and surveillance-related functions such as remote monitoring. A remote service can be, for example, a way to deliver products at a hospital or old people's home. Due to the mobile nature of the system, monitoring at places with privacy concerns is possible. On the other hand, mobility demands wireless communications. Suitable and present technologies for wireless video transfer are weighted. Identification of objects with the help of Radio Frequency Identifying (RFID) technology and facial recognition results in intelligent actions, for example, where the control of a robot does not require extensive workload from the user. In other words, tasks can be partially autonomous, RFID can be also used in augmentation of the video view with virtual objects. As a real-life experiment, a prototype environment is being constructed that consists of a robot equipped with a video camera and wireless links to the network and multimedia computer.

In this paper we propose a digital camera based method for estimating the stem diameters of growing trees for forest inventory purposes. The imaging system consists of a single camera, a laser distance measurement device and a calibration stick placed beside the tree to be measured. To carry out the task camera geometry parameters are first determined using linear pinhole camera and nonlinear lens distortion models. In addition of the accurate camera calibration, the viewing geometry has to be determined for 3D measurement purpose with the help of the calibration stick. The estimation of the stem diameters is carried out by combining the stem curve information from the image with a priori stem form model. Experimental results reported with Scots pines indicate that this approach provides feasible solutions for the measurement task.

Quality control in industrial application has greatly benefited from the development of tools like artificial vision. In order to obtain a good quality image of the object under investigation, the first step is to use a good lighting system. This paper presents a reliable method which allows to compare several lighting with respect to their capabilities of bringing out defects. This study has been led on textured industrial parts on which four types of defects have to be detected: smooth surfaces, bumps, lacks of material and hollows knocked surfaces. The aim is to determine the best lighting among various experimental sets. This method has two stages: the first one is a definition, according to the knowledge and the shape of the defects, of a pertinent attribute vector which components are defect sensitive. In the second step, discrimination power property of this vector is computed and compared under various illumination using Parzen's kernel. This method insures a well-suited illumination in numerous applications in defects detection and leads to an efficient set of lighting system and segmentation's parameters. Work is under way to generalize this method to multidimensional cases in order to allow interactions between components of the attribute vector.

Parallel approaches for packing arbitrary 3D objects into fixed volumes are characterized by rearranging all of the parts simultaneously and evaluating the results. The practical application of each proposed approach to real world problems has been hindered by the computational time required to find a solution or over simplifications made to reduce the time required. A serial approach is proposed in this paper that reduces the complexity of the problem domain by packing each object one a time as `best as possible', thus more closely emulating the way a human might arrange items in the trunk of a car. This technique has enabled the implementation of an efficient packing algorithm that is not limited by working with the object's bounding boxes and or by a restricted set of permissible orientations. Preliminary tests demonstrate that the technique reduces computational times, on average, by a factor of 19 or more compared to an existing technique. Furthermore, the new approach is guaranteed to produce a viable packing arrangement for a subset of the parts even if every part cannot possibly be accommodated in the available volume, a typical situation found in rapid prototyping service bureaus. The same cannot be said for existing parallel packing algorithm implementations.

In a satellite image the background is always regarded as the noise with high frequency. If the background can be removed, the phase only match filter (POMF), which is applied to registration of the template, can get an obvious pulse at the correct position where the template object locates. This article proposes an algorithm consisting of an artificial neural network and POMF method to tracking an object in a series of satellite images with variant intensity level. The notable adaptive resonance theory (ART) neural network is used as a high frequency noise filter for the POMF. In the proposed method, a template image is divided into several sub-images, which are used as the primitive texture blocks of the template. Since the ART has self-categorizing capability, it can learn the primitive patterns contained in the template. Once the source image is presented to the ART for registration, the network can recognize the texture blocks not contained in the template image. If these blocks are masked in the source image, the refined image contains only the blocks similar to the template image. Furthermore, an image that contains the texture blocks of the template image is obtained using the template as a mask to restore the blocks previously moved. The phase match filter is then applied to detecting the exact location of the template object on the source image. The proposed algorithm is examined by means of several satellite images. From the experimental results, the proposed method demonstrates that the performance of POMF can be successfully improved by reducing the high frequent noise of the background in the satellite images.

We present new test results for our active object recognition algorithms. The algorithms are used to classify and estimate the pose of objects in different stable rest positions and automatically re-position the camera if the class or pose of an object is ambiguous in a given image. Multiple object views are now used in determining both the final object class and pose estimate; previously, multiple views were used for classification only. A feature space trajectory (FST) in eigenspace is used to represent 3D distorted views of an object. FSTs are constructed using images rendered from solid models. We discuss lighting and material settings for photorealism in the rendering process. The FSTs are analyzed to determine the camera positions that best resolve ambiguities. Real objects are recognized from intensity images using the FST representation derived from rendered imagery.

A strategy for identifying, measuring, and duplicating machined parts using an EOL (endpoint open loop), position based, visual servo robot is presented. The procedure is demonstrated using the representative, and commercially important, problem of duplicating keys for automobiles. The tasks to be completed for this application are: (1) identify the blank type of the original, (2) measure the unique cut pattern or `code' that activates the tumblers, (3) correct deviations, possibly due to wear, from manufacturer specified acceptable cut profiles, and (4) grind the duplicate. There are over three hundred blank types in use in the U.S. alone, each varying in length, width, groove pattern on each side, and outline of the handle. These variations are often subtle, making identification difficult, even for a human.

A model of an object is an image consisting of features of an object. The input is a gray scale image from which features are computed. In his doctoral thesis, J. L. Chen used a model based approach to object recognition. His method is based on Rosin's work for extraction of parts. Both model and scene features are contour based properties. The scene features are matched to the model features by indexing. The following features are computed: convexity, compactness, roundedness, skewness, and the first moment invariant (all using Rosin's algorithm). In this paper, we extend the feature description to include internal and external parts of an object. We use the interpretation tree approach for matching the scene to model, where constraints such as distances and angles between parts are used to prune the interpretation tree. We compare the efficiency of the interpretation tree approach with the indexing method on the data that was used in previous experiments by J. L. Chen.

Rather than design an optimal filter over a large window, which may be computationally impossible or require unacceptable computation time, one can design an iterative filter, each stage of which is designed over a small window with acceptable design time. Theoretically, a two-stage iterative filter with each stage optimally designed over a window is suboptimal in comparison to the optimal filter over the larger window formed as the dilation of the small window with itself. In practice, however, filters are designed from realizations and lack of precision for design over a large window can result in a directly estimate optimal filter over a large window that performs worse than an iteratively designed filter. Using image-noise models, this paper considers there cases: (1) the designed filters are good estimates of the theoretically optimal filters, the two-stage iterative filter is close to optimal, and as further iterations are considered for both the small and large windows, the performance difference becomes small; (2) the designed filter over the large window is a poor estimate of the theoretically optimal filter and the iteratively designed filter outperforms the directly designed filter; (3) iteration cannot do well because the iteration window is too small for the image-noise model. We will see that, while in terms of logic there may be a significant difference between a noniterative and an approximating iterative filter, their probabilistic difference as operators on random processes can be negligible.

This paper addresses a method for implementation of some basic geometric transformation, namely rotation and scale, on 1D parallel SIMD computers. Solutions to solve many implementation details such as 2D image scanning with 1D machine, the floating point arithmetic simulation and an approximation of the transformed pixel spatial co-ordinates are proposed. The proposed approach tolerates up to +/- 15 degree(s) rotation and up to +/- 50% continue scale changes between images. Those transformations can be used for construction of robust matching primitives useful for robot applications. The evaluation of the proposed method, on SYMPATI2, a French 1D parallel computer (designed by CEA,LETI) is reported.

A computer vision algorithm was developed to detect moving aircraft located in video images. Using a gradient-based approach, the algorithm computes optical flow vectors in each frame of the sequence. Vectors with similar characteristics (location, magnitude, and direction) are clustered together using a spatial consistency test. Vectors that pass the spatial consistency test are extended temporally to make predictions about the optical flow locations, magnitudes, and directions in subsequent frames. The actual optical flow vectors that are consistent with the predicted vectors are labeled as vectors associated with a moving target. The algorithm was tested on images obtained with a video camera mounted below the nose of a Boeing 737. The algorithm correctly detected an aircraft from a distance of one mile in over 80% of the frames with zero false alarms.

Digital halftoning is used both in low and high resolution high quality printing technologies. Our method is designed to be mainly used for low resolution ink jet marking machines to produce both gray tone and color images. The main problem with digital halftoning is pink noise caused by the human eye's visual transfer function. To compensate for this the random dot patterns used are optimized to contain more blue than pink noise. Several such dot pattern generator threshold matrices have been created automatically by using genetic algorithm optimization, a non-deterministic global optimization method imitating natural evolution and genetics. A hybrid of genetic algorithm with a search method based on local backtracking was developed together with several fitness functions evaluating dot patterns for rectangular grids. By modifying the fitness function, a family of dot generators results, each with its particular statistical features. Several versions of genetic algorithms, backtracking and fitness functions were tested to find a reasonable combination. The generated threshold matrices have been tested by simulating a set of test images using the Khoros image processing system. Even though the work was focused on developing low resolution marking technology, the resulting family of dot generators can be applied also in other halftoning application areas including high resolution printing technology.

We present a restoration algorithm using a stable deconvolution for the real images with which we are confronted in our laboratory. In this images, the edge smoothness due to the lack of focusing is not homogeneous. The parameters of the algorithm are determined by the merging of multiscale edge detection. The edge detection is based on a generalization of the Canny-Deriche filter. This filter is used to generate a tight frame of wavelets. We propose a new criterium, the separating power, which allows to choose in a deterministic way the intrinsic parameters of the filters. This criterium introduces the concept of detection resolution which is a new concept in edge detection. Also, we evaluate the maximal delocalization obtained at the limit of the separating power. The merging algorithm is based on human vision which zooms back and forth on the image in order to identify global structures or details in the image. The merging method can be followed by a segmentation method selected from existing ones thus enabling a great adaptability of the complete system in the edge sense. The binary edge representation obtained selects the zones of restoration. Examples of result are presented.

In this paper an efficient method is presented to cope with the need of phase linear filters in orthonormal wavelet transform for image processing. Phase linear filtering can be obtained in two dimensions by using Cohen/Daubechies biorthogonal wavelets. But as orthogonal analysis is preferable, a new method to construct orthonormal bidimensional wavelet base in the quincunx scheme is proposed. These filters are designed by applying the McClellan Transform on 1D B-spline filters in order to get 2D orthonormal quincunx non-separable ones. This method takes advantage of the orthogonality of the analysis and of the quincunx scheme, indeed these filters lead to only one approximation image and only one detail image. The interscale resolution given by this analysis is twice more accurate than in the case of a separable analysis and the wavelet functions have better isotropic and frequency properties than those previously proposed by Feauveau.

A new concept of a self-learning, self-broadening knowledge base that may be used as the long-term memory for a completely calibration-free robot vision to manipulate objects is presented. With this concept the robot automatically acquires during its normal operation the necessary knowledge which can be saved afterwards in the knowledge base and allowing the robot to adapt itself to changing conditions. Thus, the robot presents self-learning characteristics. The robot control using the knowledge base is then based on the human way of solving problems, i.e. new, additional facts (in our case control words) are developed from available facts. Such a control enables improving skills of the robot. The concept has been successfully realized and tested in real-world experiments with an uncalibrated vision-guided manipulator involving the grasping of various objects with nearly any shape and in an arbitrary orientation in horizontal plane.

Robotic systems based on visual recognition are increasingly more difficult to implement in real time as the complexity and the number of target objects increases. Flexible robotic implementations require a 3D scene understanding. To obtain robust 3D information in active sensor based on depth from defocusing technique is implemented. The algorithm involves calculating distance to points in a scene using the relative blurring between two images detected with two different focal settings. These images are acquired by splitting the image of the scene and capturing them with two CCD sensors with a physical known distance between sensor planes. The recognition algorithm uses a variant of eigenimage analysis. This approach computes the eigenspace determined by processing the eigenvalues and eigenvectors of the each image set. Each image set is obtained by varying the post of each object in space. Also, details of the local image processing used for localization and the centering of the object are presented and discussed. The presented system has been successfully implemented and tested on several real world scenes.

The purpose of this paper is to present a new approach for path planing of a mobile robot in static outdoor environments. A simple sensor model is developed for fast acquisition of environment information. The obstacle avoidance system is based on a micro-controller interfaced with multiple ultrasonic transducers with a rotating motor. Using sonar readings and environment knowledge, a local map based on weight evaluation function is built for the robot path planing. The path planner finds the local optimal path using the A* search algorithm. The robot is trained to learn a goal-directed task under adequate supervision. The simulation experiments show that a robot, utilizing our neural network scheme, can learn tasks of obstacle avoidance in the work space of a certain geometrical complexity. The result shows that the proposed algorithm can be efficiently implemented in an outdoor environment.

Robots operating in an unstructured environment need high level of modeling of their operational space in order to plan a suitable path from an initial position to a desired goal. From this perspective, operational space modeling seems to be crucial to ensure a sufficient level of autonomy. In order to compile the information from various sources, we propose a fuzzy approach to evaluate each unit region on a grid map by a certain value of transition cost. This value expresses the cost of movement over the unit region: the higher the value, the more expensive the movement through the region in terms of energy, time, danger, etc. The approach for modeling, proposed in this paper, employs fuzzy granulation of information on various terrain features and their combination based on a fuzzy neural network. In order to adapt to the changing environmental conditions, and to improve the validity of constructed cost maps on-line, the system can be endowed with learning abilities. The learning subsystem would change parameters of the fuzzy neural network based decision system by reinforcements derived from comparisons of the actual cost of transition with the cost obtained from the model.

Collision freeness is an essential feature in off-line programming of robots. It relates to safe and efficient operation, and becomes even more critical, when two or more robots are sharing the same work space, which is typical for so called holonic robot systems. Here we are reporting a simple and fast path planning algorithm based on configuration space method, suitable for holonic robot applications. The basic version of the algorithm can be found from the literature, and we have extended and modified it to improve its feasibility. Simulation and experimental tests have been carried out successfully.

A sensor fusion method for navigation of an Autonomous Guided Vehicle robot using Artificial Neural Network is described. Robot navigation is defined as the guiding of a mobile robot to a desired destination or along a desired path in an environment characterized by a terrain and a set of distinct objects, such as obstacles and landmarks. The low-level sensor fusion technique is used for direct integration of sensor data, resulting in parameter and state estimates. The multi-layered perceptron, with a single hidden layer in neural network structure, and the back- propagation algorithm are employed for the mobile robot's navigation and for obstacle avoidance. The significance of this work lies in the development of a new estimation method for mobile robot obstacle avoidance and guidance.

Exploratory research on the design of a modular autonomous mobile robot controller for steering and speed control is described. The high level control of the robot incorporates a fuzzy logic approach. Steering and speed control are achieved using a three-axis Galil motion controller. The steering mechanism and the speed control involve the parallel control of the robot's two front wheels based on decisions made by the upper level fuzzy logic to guide the robot in a desired direction to follow a specific path and avoid obstacles in that path. The steering motors are Electrocraft brush DC motor. The BDC amplifiers are run in the current loop mode. The overall control is supervised by a personal computer through the multi-axis controller. The system has been simulated on Matlab and Simulink and optimal values for the digital gains were achieved for desired control. Testing of these systems has been done in a laboratory setting as well as on an outside track with positive results.

This paper describes the use of fuzzy logic control for the high level control systems of a mobile robot. The advantages of the fuzzy logic system are that multiple types of input such as that from vision and sonar sensors as well as stored map information can be used to guide the robot. Sensor fusion can be accomplished between real time sensed information and stored information in a manner similar to a human decision maker. Vision guidance is accomplished with a CCD camera with a zoom lens. The data is collected through a commercial tracking device, communicating to the computer the X,Y coordinates of a lane marker. Testing of these systems yielded positive results by showing that at five miles per hour, the vehicle can follow a line and avoid obstacles. The obstacle detection uses information from Polaroid sonar detection system. The motor control system uses a programmable Galil motion control system. This design, in its modularity, creates a portable autonomous controller that could be used for any mobile vehicle with only minor adaptations.

We address the problem of sensor-based motion planning for a robot arm manipulator operating amongst obstacles of arbitrary shapes. Collisions might occur at any point on the arm body. The input information is assumed to be coming from a sensitive skin that covers the arm body. The skin is covered by densely distributed proximity sensors. In prior work, step calculation was based on the average of normal vectors to the points on the obstacles sensed by the robot. There are cases when this algorithm can lead to collisions. In this paper an algorithm is proposed for local step planning whereby the direction for the next step motion is done along the intersection of planes tangential to the points of obstacles sensed. The tangential planes themselves are approximated via the least square fit of the sensor readings involved. An expansion of the scheme to robot arms with arbitrary number of links is also discussed.

Developing elementary behavior is the starting point for the realization of complex systems. We present a learning algorithm that realizes a simple goal-reaching behavior for an autonomous vehicle when no a-priori knowledge of the environment is provided. Information coming from a visual sensor is used to detect a general state of the system. To each state an optimal action is associated using a Q- learning algorithm. As sets of states and actions are limited, a few training trials are sufficient in simulation to learn the optimal policy. During test trials (both in simulated and real environment) fuzzy sets with membership functions are introduced to compute the state of the system and the proper action at the extent of tackling errors in state estimation due to noise in vision measures. Experimental results both in simulated and real environment are shown.

3D multilayer phase gratings (nuclear layers in the human retina) of oscillating cells positioned in the image plane of a monocular optical imaging system transform light double cones by diffraction and pulsation (quantization) into the reciprocal space behind the grating (Fourier-space in the Fresnel-nearfield). This effect we call OPTORETINA not only guarantees the transformation of the physical stimuli parameters (intensity/wavelength) in the visible spectrum into three chromatically tuned adaptive RGB-color channels (diffraction orders with Brightness-Hue-Saturation aspects) and of object distances in 3D-space into spatial frequencies or temporal phase differences in reciprocal space (the MULTIDIST grating optical sensors developed at CORRSYS).

A major problem in the creation of multi-sensor interfaces for tele-robotics is the development of method for combining various data sources into one coherent world model for examination by the operator. However, if the images acquired from different sensing modalities can be registered with LADAR range images, texture mapping can be employed for sensor integration. Presented in this paper is a novel stereo-based method for registering color and LIDAR images acquired from externally uncalibrated sensors for use in a visualization system as described above. In our approach, corresponding features are first extracted from LADAR intensity/color image pairs. Since the computation of Euclidean range using stereoscopic techniques depends on knowledge of the relative sensor orientation, a downhill simplex algorithm is iteratively applied to minimize the sum-squared-error between the stereo and LADAR range estimates of the matched points with respect to the system's relative rotation and translation parameters. Once the system's epipolar geometry has been estimated in this manner, the LADAR range data is employed to re-project the color image to the viewpoint of the scanner for use as a texture map.

This work studies visual quality control in ceramics industry. In tile manufacturing, it is important that in each set of tiles, every single tile looks similar. For example, the tiles should have similar color and texture. Our goal is to design a machine vision system that can estimate the sufficient similarity or same appearance to the human eye. Currently, the estimation is usually done by human vision. Differing from other approaches our aim is to use accurate spectral representation of color, and we are comparing spectral features to the RGB color features. A laboratory system for color measurement is built. Experimentations with five classes of brown tiles are presented. We use chromaticity RGB features and several spectral features for classification with the k-NN classifier and with a neural network, called Self-Organizing Map. We can classify many of the tiles but there are several problems that need further investigations: larger training and test sets are needed, illuminations effects must be studied further, and more suitable spectral features are needed with more sophisticated classifiers. It is also interesting to develop further the neural approach.

Road/Traffic Color Image Segmentation is an important task for a variety of traffic applications. In the past few decades, many color image segmentation approaches have been developed based on different segmentation techniques, such as edge detection, multiple thresholds, region growing, space clustering, and spectral classification etc. However, due to the lack of adaptability for color fluctuation, current segmentation techniques have difficulty to overcome the additional effects by variable illuminations on color objects. Furthermore, learning skill for color features is not included in most color segmentation approaches. Hence, they have limited applications to road/traffic color image segmentation under a dynamic outdoor environment. In this paper, a new color image segmentation approach based on an improved Restricted Coulomb Energy (RCE) neural network is presented. It attempts to solve the problem of color classification in which color classes are represented by both disjoint class distributions and non-separable classes whose distributions overlap in color space. The color features of each color object are extracted by using the method of `The Estimation of Color Prototype Density' in RCE training procedure, they are stored in the prototype layer as color prototypes of a particulate color class. With the various prototypes established at the learning stage, RCE neural network is able to generate an optimal segmentation output in either the fast response mode or the output probability mode during color image segmentation.

In this paper, we propose a set of effective algorithms to automatically detect the lung cancer cells in the cytological color image of examines' sputum smears. To increase the stability and efficiency of the detection of the cancer cells, a hierarchical processing architecture is adopted for the segmentation and recognition. For segmentation, RGB space and Lab space are combined to segment cell. By this method, both the nucleus and cytoplasm of cancer cells can be separated from background. Then, the candidate cancer cells are selected using some morphological features of nuclei, the purpose of this step is to pick out most of non cancer cells and leave a few doubtful cells for further verification, therefore improve the efficiency of the whole recognition process. As the last step, all the candidate cell, some statistic parameters in different color space are calculated, which are used as features for recognition. Experiment results are given.

The purpose of our works is to provide fast and reliable face localization techniques in real time an in real-life scenes. Person localization is included in this problem. The end application sought is the ability of mobile robots to navigate in human populated environments, and to start visual interaction with them. Known methods are computationally intensive, far from real time implementation at near future processing power of off-the-shelf processors. Our technique is based in motion segmentation, signature analysis and color processing. Signature analysis provides fast hints of the person and face localization. Color processing is used to confirm the face hypothesis, and it is based on our works on adaptive color quantization of image sequences. The technique can be implemented in real time and combined with other approaches to enhance the recognition results.

In this paper, we present a method for performing a multi- scale analysis by using the B-Spline wavelet transform applied on 3D objects containing relevant information at different scales. An accurate non-contact optical system based upon laser triangulation has been used to determine depth information of objects. The resulting range image is treated as a gray-level image by using a multi-resolution approach based on the generalization of the cascade algorithm. This method allows reconstruction of non- subsampled images that correspond to the decomposition previously done at a chosen scale. This new approach provides efficient tool for analyzing multi-scale information contained in range images. Thus, this approach may be applied on any 3D objects for extracting 3D features such as straight or smooth ridges, or object envelope. For example, we applied this method to the characterization of clam shells in order to reveal growth increment features.

Wide-Angle lenses are not often used for 3D reconstruction tasks, in spite of the potential advantages offered by their increased field-of-view, because existing algorithms for high-distortion lens compensation perform poorly at the image extremities. In this paper, a two-stage projection model based on quadric surfaces is developed which accurately characterizes the effect of wide-angle lenses across the entire image by incorporating the nonlinearity of the lens in the image projection transform. A comparison of this quadric projection model with an existing lens distortion correction technique based on spherical projection is presented to illustrate the increased accuracy of the pinhole images obtained using the new model.

A single rotating sonar element is used with a restricted angle of sweep to obtain readings to develop a range map for the unobstructed path of an autonomous guided vehicle (AGV). A Polaroid ultrasound transducer element is mounted on a micromotor with an encoder feedback. The motion of this motor is controlled using a Galil DMC 1000 motion control board. The encoder is interfaced with the DMC 1000 board using an intermediate IMC 1100 break-out board. By adjusting the parameters of the Polaroid element, it is possible to obtain range readings at known angles with respect to the center of the robot. The readings are mapped to obtain a range map of the unobstructed path in front of the robot. The idea can be extended to a 360 degree mapping by changing the assembly level programming on the Galil Motion control board. Such a system would be compact and reliable over a range of environments and AGV applications.

For faithful 3D reconstruction of objects with smooth surfaces, stereo vision techniques rely heavily on the surface texture. However, most of the real life objects lack this feature. To `sharpen' the textural content of visual surfaces, a structured-light sensing configuration has been used. This technique can be used to enhance the features used in solving the correspondence problem in computational stereo vision. A light pattern is projected to encode the smooth surface of the object. The observed light pattern is then used to compute surface properties. We present a simple design for a trinocular vision system with structured light using off-the-shelf components. The processing pipeline of the system consists of four stages. First, the light pattern is detected in the captured images. Second, the pattern is skeletonized using connected component labeling. Third, a maximum-weighted bipartite technique is used to do the matching. Finally, a global surface fitting technique is used to integrate the reconstruction from different views in one frame and fit a mesh of triangles to the integrated data. Results using real images are promising. The advantages of this system lie in its design simplicity, low cost and the potential for fast and parallel implementation.

One approach to stereo matching is to use different local features to find correspondences. The selection of an optimum feature set is the content of this paper. An operational software tool based on the principle of comparing feature vectors is used for stereo matching. A relatively large set of different local features is sought for optimum combinations of 6 - 10 of them. This is done by a genetic process that uses an intrinsic quality criterion that evaluates the correctness of each individual match. The convergence of the genetic feature selection process is demonstrated on a real stereo pair of a tunnel surface. Four areas were used for individual optimization. After several hundred generations for each of the areas, it is shown that the identified feature sets result in a considerably better stereo matching result than the currently used features, which were the result of an initial manual choice. The experiments described in this paper use a `super-set' of 145 features for every pixel, which are created by filtering the image with convolution kernels (averaging, Gaussian filters, bandpass, highpass), median filters and Gabor kernels. From these 145 filters, the genetic feature selection process selects an optimal set of operators. Using the selected filters results in a 15% improvement of the matching accuracy and robustness.

The purpose of this paper is to describe an expert system to design the base of an automated guided vehicle. The components of the expert system include: (1) A user-friendly graphic user interface, where the user can enter specifications--like the environment used, application of the robot, etc.; (2) An engine that converts the managerial requirements into technical parameters and designs the robot--initially assuming some parameters and confirming its assumptions during the course of the design; when unable to do so, it iterates with different assumptions until they are met; the code also selects various materials to be used from a corresponding database; (3) A database of various materials from their manufacturers/suppliers; (4) The output data is interfaced with a CAD engine, which generates a 3D solid model of the vehicle; and (5) A `Bill of Materials' file is generated as the output and suggestions for how to assemble them are given. The method has been tested by designing a small mobile robot. The software provides an excellent tool to develop a mobile robot based on performance specifications. Modeling helps the user understand the constraints on the design of the robot and the bill of materials--along with the vendor address, helps the user buy the components needed to assemble the robot.

Linear models are a popular tool to characterize the performance of active vision systems. They cannot, however, capture the nonlinear limits of the system performance that arise out of the physical limitations of the real system. We examine how the finite field of view and the limited motor torque impose bounds on the linear operation of an active vision system in a fixation task. We derive the bounds by analyzing a real active vision system and measuring its response to repeatable and controllable target motions generated by a robot arm. The knowledge of the operational bounds can guide us in deciding which aspects of the active vision system to improve for better overall performance.

This paper describes an Active Vision System whose design assumes a distinction between fast or reactive and slow or background processes. Fast processes need to operate in cycles with critical timeouts that may affect system stability. While slow processes, though necessary, do not compromise system stability if its execution is delayed. Based on this simple taxonomy, a control architecture has been proposed and a prototype implemented that is able to track people in real-time with a robotic head while trying to identify the target. In this system, the tracking mobile is considered as the reactive part of the system while person identification is considered a background task. This demonstrator has been developed using a new generation DSP (TMS320C80) as a specialized coprocessor to deal with fast processes, and a commercial robotic head with a dedicated DSP-based motor controller. These subsystems are hosted by a standard Pentium-Pro PC running Windows NT where slow processes are executed. The flexibility achieved in the design phase and the preliminary results obtained so far seem to validate the approach followed to integrate time- critical and slow tasks on a heterogeneous hardware platform.

This paper describes a practical approach for environment perception using fusion of active ranging with a low-power laser and passive, stereo based ranging. A low-cost, coarse pitch linear detector is used, but still sufficient distance accuracy for mobile robotics is achieved by using a new method of sub-pixel calibration. The device is intended for mobile robot or vehicle perception duties in obstacle detection and target position measurement.

In this paper a special type of image segmentation, a two- class segmentation, is considered. Defect detection in quality control applications is a typical two-class problem. The main idea in this paper is to train the two-class classifier with fault-free samples that is an unexpected approach. The reason is that defects are rare and expensive. The proposed defect detection is based on the following idea: an unknown sample is classified as a defect if it differs enough from the estimated prototypes of fault-free samples. The self-organizing map is used to estimate these prototypes. Surface images are used to demonstrate the proposed image segmentation procedure.

Scene segmentation is a pre-processing step for many vision systems. We are concerned with segmentation as a precursor to 3D scene modeling. Segmentation of a scene for this purpose usually involves dividing an image into areas that are relatively uniform in some value (e.g. intensity, range, or curvature). This single segmented image represents the analogous segmented scene. This paper presents a segmentation method that uses features to indicate boundaries or edges between regions. We incorporate features from multiple images types to obtain an more accurate segmentation of objects or object parts in the scene. Multiple features are not only combined directly to improve segmentation results, but they are also used to guide a smoothing operation. This smoothing technique preserves features representing edges while smoothing noise in the images.

Automatic object recognition methodologies are being used increasingly in the automation of many industrial processes. However, the specific nature of each system usually implies an intensive development effort. Aiming to control and constrain such effort, this paper presents a processing architecture that promotes the reutilization of common system modules. The approach described covers the whole processing chain by connecting low-level pre-processing to the final classification stage, specially emphasizing the segmentation and feature extraction levels. The resulting core is a search engine based on the data structure of a graph. This structure is initially fed with raw edge information extracted from the original raster image representation. The edge segments used at this preliminary version of the graph are the result of a multistage process of edge following and edge linking. The use of image properties at high levels of abstraction allows the dimensions of the search space to be significantly reduced. Modularity and reutilization allows the refinement of an initial low-level representation of information into successive higher levels of description, until a final set of features that may directly feed a classifier is achieved. Parallel contours, constant curvature edge segments and closed contours of a given shape are a few examples of the features easily defined within this architecture and that can be extracted by the search engine implemented.

Diffraction of light by 3D phase grating layers could be effectively used for color image processing in robotic vision. Gratings with hexagonal close-packed structures have the maximum amount of cells per volume unit, which leads to an advantage for color image processing. Using the 4D spectral method, we solve the wave equation for diffraction of light by a 3D hexagonal phase grating layer of spherical particles. Both ABCA and ABAB structures are considered. Distribution of diffracted light intensity is calculated in the Fraunhofer and Fresnel diffraction zones. For particular grating distances, the incident white light diffracts in three spatially separated maximums with the central wavelengths corresponding to the three primary colors. The wavelength dependence of diffracted light intensity, for incident white light, is calculated for the three maximums. In general case, by using these three primary curves one can reconstruct the color of incident light from corresponding values of light intensities measured in the three diffracted maximums. The conditions for self-imaging of 3D grating layers are formulated and investigated. Intensity distributions for diffracted light in planes of positive and negative self-imaging, and in a plane of lowest contrast are computed.

The process of recognition, for instance, understanding the text, written by different fonts, consists in the depriving of the individual attributes of the letters in the particular font. It is shown that such process, in Nature and technique, can be provided by the narrowing the spatial frequency of the object's image by its defocusing. In defocusing images remain only areas, so-called Informative Fragments (IFs), which all together form the generalized (stylized) image of many identical objects. It is shown that the variety of shapes of IFs is restricted and can be presented by `Geometrical alphabet'. The `letters' for this alphabet can be created using two basic `genetic' figures: a stripe and round spot. It is known from physiology that the special cells of visual cortex response to these particular figures. The prototype of such `genetic' alphabet has been made using Boolean algebra (Venn's diagrams). The algorithm for drawing the letter's (`genlet's') shape in this alphabet and generalized images of objects (for example, `sleeping cat'), are given. A scheme of an anthropomorphic robot is shown together with results of model computer experiment of the robot's action--`drawing' the generalized image.