KEYWORDS: Detection and tracking algorithms, Target detection, Sensors, Visualization, Video, Algorithm development, Signal to noise ratio, Cameras, Acoustics, Visual system
Algorithms for synergistically fusing acoustic and optical sensory inputs, thereby mimicking biological attentional
processes are described. Manual existing perimeter defense surveillance systems using more than one sensory modality
combine different sensors' information to corroborate findings by other sensors and to add data from a second modality.
In contrast to how conventional systems work, animals use information from multiple sensory inputs in a way that
improves each sensory system's performance. We demonstrated that performance is enhanced when information in one
modality is used to focus processing in the other modality (a form of attention). This synergistic bi-modal operation
improves surveillance efficacy by focusing auditory and visual "attention" on a particular target or location.
Algorithms for focusing auditory and visual sensors using detection information were developed. These
combination algorithms perform "zoom-with-enhanced-acuity" in both the visual and auditory domains, triggered by
detection in either domain. Sensory-input processing algorithms focus on specific locations, indicated by at least one of
the modalities. This spatially focused processing emulates biological attention-driven focusing. We showed that given
information about the target, the acoustic algorithms were able to achieve over 80% correct target detection at signal-tonoise
ratios (SNRs) of -20 dB and above, as compared with similar performance at SNRs of -10 db and above without
target information from another modality. Similarly, the visual algorithm achieved performance of over 80% detection
with added noise variance of 0.001 without target indication, but maintained 100% detection at added noise variance of
0.05 when acoustic target information was taken into account.
Increasing battlefield awareness can improve both the effectiveness and timeliness of response in hostile military
situations. A system that processes acoustic data is proposed to handle a variety of possible applications. The front-end
of the existing biomimetic acoustic direction finding system, a mammalian peripheral auditory system model, provides
the back-end system with what amounts to spike trains. The back-end system consists of individual algorithms tailored to
extract specific information. The back-end algorithms are transportable to FPGA platforms and other general-purpose
computers. The algorithms can be modified for use with both fixed and mobile, existing sensor platforms.
Currently, gunfire classification and localization algorithms based on both neural networks and pitch are being developed
and tested. The neural network model is trained under supervised learning to differentiate and trace various gunfire
acoustic signatures and reduce the effect of different frequency responses of microphones on different hardware
platforms. The model is being tested against impact and launch acoustic signals of various mortars, supersonic and
muzzle-blast of rifle shots, and other weapons. It outperforms the cross-correlation algorithm with regard to
computational efficiency, memory requirements, and noise robustness. The spike-based pitch model uses the times
between successive spike events to calculate the periodicity of the signal. Differences in the periodicity signatures and
comparisons of the overall spike activity are used to classify mortar size and event type. The localization of the gunfire
acoustic signals is further computed based on the classification result and the location of microphones and other
parameters of the existing hardware platform implementation.
KEYWORDS: Digital signal processing, Analog electronics, Nerve, Neurons, Signal processing, Field programmable gate arrays, Acoustics, Mirrors, Computing systems, Electronics
We are developing low-power microcircuitry that implements classification and direction finding systems of very small
size and small acoustic aperture. Our approach was inspired by the fact that small mammals are able to localize sounds
despite their ears may be separated by as little as a centimeter. Gerbils, in particular are good low-frequency localizers,
which is a particularly difficult task, since a wavelength at 500 Hz is on the order of two feet. Given such signals, crosscorrelation-
based methods to determine direction fail badly in the presence of a small amount of noise, e.g. wind noise
and noise clutter common to almost any realistic environment. Circuits are being developed using both analog and
digital techniques, each of which process signals in fundamentally the same way the peripheral auditory system of
mammals processes sound. A filter bank represents filtering done by the cochlea. The auditory nerve is implemented
using a combination of an envelope detector, an automatic gain stage, and a unique one-bit A/D, which creates what
amounts to a neural impulse. These impulses are used to extract pitch characteristics, which we use to classify sounds
such as vehicles, small and large weaponry from AK-47s to 155mm cannon, including mortar launches and impacts. In
addition to the pitchograms, we also use neural nets for classification.
KEYWORDS: Digital signal processing, Analog electronics, Neurons, Transistors, Biomimetics, Field programmable gate arrays, Nerve, Algorithm development, Signal processing, Signal to noise ratio
Biomimetic signal processing that is functionally similar to that performed by the mammalian peripheral auditory system
consists of several stages. The concatenated stages of the system each favor differing types of hardware
implementations. Ideally, the front-end would be an implementation of the mammalian cochlea, which is a tapered
nonlinear, traveling-wave amplifier. It is not a good candidate for standard digital implementations. The AM
demodulator can be implemented using digital or analog designs. The Automatic Gain Control (AGC) stage is highly
unusual. It requires filtering and multiplication in a closed-loop configuration, with bias added at each of two
concatenated stages. Its implementation is problematic in DSP, FPGA, full custom digital VLSI, and analog VLSI. The
one-bit A/D (also called the "spiking neuron"), while simple at face value, involves a complicated triggering mechanism,
which is amenable to DSP, FPGA, and custom digital but computationally intense, and is suited to an analog VLSI
implementation.
Currently, we have several hardware embodiments of the biomimetic system. The RedOwl application occupies about
160 cubic inches in volume at the present time. A DSP approach can compute 15 channels for two ears for three A/D
categories using Analog Devices Tiger SHARC-201 DSP chips within a system size estimated to be on the order of 30
cubic inches. BioMimetic Systems, Inc., a Boston University startup company is developing an FPGA solution. Within
the university, we are also pursuing both a custom digital ASIC route and a current-mode analog ASIC.
This paper describes the flow of scientific and technological achievements beginning with a stationary "small, smart,
biomimetic acoustic processor" designed for DARPA that led to a program aimed at acoustic characterization and
direction finding for multiple, mobile platforms. ARL support and collaboration has allowed us to adapt the core
technology to multiple platforms including a Packbot robotic platform, a soldier worn platform, as well as a vehicle
platform. Each of these has varying size and power requirements, but miniaturization is an important component of the
program for creating practical systems which we address further in companion papers. We have configured the system to
detect and localize gunfire and tested system performance with live fire from numerous weapons such as the AK47, the
Dragunov, and the AR15. The ARL-sponsored work has led to connections with Natick Labs and the Future Force
Warrior program, and in addition, the work has many and obvious applications to homeland defense, police, and civilian needs.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.