2.1

Bayesian Soft-Hard Data Fusion

Probabilistic models and Bayesian reasoning provide a powerful general framework for augmenting robotic perception systems with ‘human sensors’, which can provide soft data to complement ‘hard data’ from conventional sensors such as lidar, cameras, sonars, etc. in partially observable environments. ‘Soft data’ is any set of observations that originates from human sources. ²² For instance, human pilots and payload specialists in wilderness search and rescue (WiSAR) missions can interpret video feeds and electro-optical/IR data streams provided by small fixed-wing UAVs, and can spot important clues that help narrow down probable lost victim locations and movements. ²³ Likewise, in large-scale surveillance for defense applications, dismounted soldiers can provide evidence on the whereabouts and behaviors of potential intruders moving across unsecure areas; it is desirable to directly fuse such soft data with hard data from UAV patrols to improve intruder detection and tracking performance. Combined hard-soft sensing can also help cope with design limitations in UXV systems, where autonomous vehicles are subject to hardware constraints that restrict onboard sensing, processing and communication abilities. Soft data integration also lets humans stay ‘in the loop’ without overloading them with cognitively demanding planning/navigation tasks. ²⁴

A key problem then is: how should soft sensor data be formally integrated with hard data to augment robotic estimation and perception algorithms? Figure 1 shows the corresponding PGM for the generic human-robot sensor fusion problem, where sensor model parameters Θ for both hard and soft sensors may also be unknown, and thus may have to be estimated along with the state of interest X. Soft data can be broadly related to either ‘abstract’ phenomena that cannot be measured by robotic sensors (e.g. labels for object categories and behaviors) or measurable dynamical physical states that must be monitored constantly (object position, velocity, attitude, temperature, size, mass, etc.)²² Our work focuses on the latter, under the key assumption that humans are not oracles: as with any other sensor data, human observations are subject to errors, limitations and ambiguities that must be modeled properly. We aim to adapt widely used statistical sensor fusion and robotic state estimation algorithms, e.g. Bayes filters and the like, so that soft data can be exploited with minimal effort on the part of the robot or the human sensor.

Figure 1.

Generic PGM for soft-hard sensor fusion problem in a human-autonomous robot team.

3.1

Bayesian Fusion of Semantic Data

Let X be a continuous random vector representing the dynamic state of interest (e.g. target location, velocity, heading) with prior pdf p(X) (which may already be conditioned on hard data), and D be a discrete random variable representing a human-generated semantic observation related to X (e.g. ‘The target is on the bridge', ‘The target is heading over the bridge and slowing down’, etc.). Given the likelihood function P(D|X), Bayes' rule gives the posterior pdf

where P(D|X) models the human's ‘semantic classification error’ as a function of X. If D = l corresponds to one of m exclusive semantic categories for a known dictionary, then a softmax function can be used to model P (D = l|X),

Fig. 2 (a) shows an example softmax model for semantic spatial range and bearing observations in 2D. An important feature of this likelihood model is that, for a given parameter set of class weights and biases Θ = , the state space X is divided into m convex ‘e-probability polytopes’, i.e. convex subsets in X where certain semantic categories occur with a probability P(D|X) ≥ ε.³¹ Label classes in a softmax model can also be internally grouped together as ‘subclasses’ within larger semantic classes. This yields the generalized multimodal softmax (MMS) likelihood model,³² which can represent arbitrary non-convex probabilistic polytopes in X via piecewise-convex subclasses. For instance, as shown in Fig. 2 (b), a range-only semantic MMS model can be obtained from Fig. 2 (a) by grouping together range labels. The parameters Θ of both softmax and MMS models can be learned from human-generated calibration data using standard parameter estimation techniques, as shown in Fig. 2 (c)-(d) for two different human sensors.

Figure 2.

(a) Probability surfaces for example softmax likelihood model, where class labels take on a discrete range in ‘Next To’,‘Nearby’,‘Far From’ and a canonical bearing ‘N’,‘NE’,‘E’,‘SE’,…,‘NW’; (b) MMS model for semantic range derived from softmax model in (a), using 1 subclass for ‘next to’, 8 subclasses for ‘nearby’ and 8 subclasses for ‘far’; (c)-(d) ‘Nearby’ label probabilities in estimated MMS range-only models for two different human sensors.

To perform recursive Bayesian data fusion with softmax or MMS likelihoods, eq. (1) must be approximated, since the exact posterior pdf p(X|D) cannot be obtained in closed-form for any prior p(X). Ref. 28 showed that, if P(D = i|X) is generally given by an MMS model with q subclasses for observation label i and the prior is given by a finite Gaussian mixture (GM) with m_p prior components,

(where w_p, μ_ρ ∈ ℝⁿ, and Σ_ρ ∈ ℝ^n×n are the weights, mean vector and covariance matrix for mixand p), then p(X|D = i) can be well-approximated by a q_im_p component GM,

The weights, means and covariances of posterior component q can be determined by fast numerical quadrature techniques such as likelihood weighted importance sampling (LWIS) or variational Bayesian importance sampling (VBIS).²⁸ These techniques exploit the fact that the exact product of an MMS likelihood function and GM prior is a mixture of non-Gaussian component pdfs, where each component is guaranteed to be unimodal and thus can be well-approximated by a moment-matched Gaussian. To manage the growth of mixture terms from m_p to q_im_p, mixture compression methods such as Runnalls' joining algorithm³³ can used to find a GM with m_f < q_im_p components that minimizes an information loss with respect to (3). This allows semantic human sensor data to plug seamlessly into existing GM Bayes filters for hard robot sensor data fusion.²⁵’³⁰

Figure 3 illustrates the semantic data fusion process for an indoor target localization application, discussed in refs. 28, 34 and 35. In this example, a ground robot and a remote human supervisor perform a time critical search for static targets (red traffic cones) using a Bayesian search algorithm with GM prior distributions on the target locations. The robot autonomously plans optimal search paths using negative information gathered from an onboard visual detector, which has a very limited range and field of view. The human supervisor can confirm possible target detections (confirming true detections or false alarms), but can also voluntarily provide semantic observations about the environment based on a live (grainy and delayed) video feed. In this application, the human's semantic observations are selected from a pre-defined dictionary, which filled in templated observations of the form ‘[Something/Nothing] is [preposition][reference object]’.

Figure 3.

Static target localization application from refs. 28,35. The surveillance area consist of mapped obstacles and landmarks, and targets (orange traffice cones) with unknown locations X given by a Gaussian mixture (GM) prior p(X). An autonomous UGV robot performs Bayesian fusion of prior beliefs with detection/no detection reports from a vision sensor and semantic observations provided by a human supervisor' producing updated GM posteriors for X. The updated GMs are then used by autonomous path planning algorithms to navigate towards likely target positions' i.e. the human only provides sensor observations' and never directly commands or steers the robot.

As shown in the bottom right of Fig. 3, soft semantic data fusion leads to a massive injection of information and hence a significant shift in the robot's beliefs about the target locations. This allows the robot to plan and execute more efficient search paths. The soft semantic data fusion process does not require the human to engage in cognitively demanding planning or control activities: the robot simply responds to new human sensor information by updating its beliefs about the target state and executing corresponding optimal search actions. Experiments with 16 different human participants acting as supervisors³⁵ showed that targets well outside the robot's field of view could be localized to sub-meter accuracy, using only semantic data fusion updates provided by the human (which included both positive and positive observations, i.e. ‘Something is in front of the door’ vs. ‘Nothing is in front of the door’). In these experiments, human supervisors were also allowed to view a live heat map representing the robot's beliefs about the target locations. As such, soft data fusion can be particularly useful as a ‘coarse sensing’ input for both priming and correcting robotic perception in complex dynamic settings. For example, many human supervisors were able to use soft data to correctively/preepmtively ‘nudge’ the robot's beliefs whenever its detector encountered false negatives, thus preventing the robot from wandering away from the true target locations. The fusion of negative semantic information (e.g. ‘There is nothing in this room’) also allowed the robot to quickly narrow down search regions, thus saving considerable time and energy.

While promising, these initial results were obtained with significant design constraints on the semantic data fusion system and human-machine interface. For instance, a highly restrictive semantic dictionary and set of softmax/MMS likelihoods were used to model human sensor semantics in order to avoid the up front difficulties of natural language processing. This approach also assumes that it is possible to construct the entire dictionary and set of semantics ahead of time, which is prohibitive for many applications (e.g. mapping and tracking targets in unknown environments). Furthermore, a strictly passive data fusion process was used: the human supervisor would only voluntarily provide inputs if he/she deemed them necessary. The following subsections describe recent advances that address these issues.

3.2

Semantic Likelihood Synthesis and Compression

Softmax and MMS likelihood models are theoretically convenient for recursive data fusion via (3), but it is not immediately obvious how to physically interpret or manipulate these sensor models in different settings. Refs. 36 and 37 established several key properties of general softmax and MMS model geometry to address this issue, and provide the basis for novel solutions to two related problems: semantic likelihood model synthesis, and batch semantic likelihood compression.

Likelihood model synthesis: How should softmax/MMS models be constructed ‘from scratch’ and/or dynamically modified in general state spaces X? For instance, we may wish to model likelihood functions in complex 2D/3D spatial domains and beyond to include velocities, angles, etc. Or we may wish to build likelihood models ‘on the fly’ to exploit spatial semantics for newly perceived objects or environments that are mapped in real time. It is thus practically necessary to translate known geometric constraints in X directly into prior specifications/constraints for Θ. This is especially important for learning with sparse human sensor calibration data, and for adapting Θ online to enforce expected geometric properties of likelihood function polytopes in X space. For instance, meaningful spatial invariances can be enforced at different scales and for different reference geometries, e.g. ‘near the table’ vs. ‘near the house’. In Fig. 2 (a), Θ was obtained via maximum liklihood optimization on manually generated ‘prototypical’ training data, which was tuned to produce the desired polytope geometries via trial and error. This brute force approach is computationally expensive: it requires non-convex optimization, and is not suitable for higher dimensional settings.

To solve these problems, we can recall and exploit the important fact that the softmax function (2) describes a set of probabilistic polytopes in X space. In particular, the set of log-odds functions between classes i and j define the linear hyperplane boundaries of their corresponding polytopes at different relative probability levels; for equal probabilities ε, we have

Thus, if we are given normal vectors that describe a desired set of polytopes that should exist between semantic classes (or constraints/invariants on those class boundaries), it is easy to show that eq. (4) leads to a system of linear difference equations for Θ,

where represents a vectorized version of Θ and M is a relative difference operator that encodes the appropriate differencing operations on via (4) to produce the neighboring class polytopes defined by Hence, the set of Θ that produce a desired convex semantic decomposition of X (as encoded by M and ) form the solution space of (5). Once established, these parameters Θ and their relationships can be further manipulated to alter the likelihood model's embedded probabilistic polytopes as needed. Note that (5) does not require training data, but also does not guarantee that a (unique) solution will be found. Even then, (5) allows imposes useful constraints that greatly improve the efficiency of learning Θ from (highly sparse) calibration data.

Fig. 4 shows a simple example of synthesizing a semantic MMS likelihood model that describes the space ‘inside’ and ‘outside’ an arbitrary irregular 2D polygon. This specification of 5 polytope face boundaries for m = 6 softmax subclasses (5 softmax subclasses describe ‘outside’, while one describes ‘inside’) leads to a system of N_S(n + 1) = 15 linear equations in m(n + 1) = 18 unknown softmax parameters. Since one set of subclass parameters can always be set the zero vector, assuming w_i = 0 and b_j = 0 for i =‘inside’, gives 15 unknown softmax parameters in 15 difference equations. Thus, in eq. (5), M = I (the identity matrix), and so , i.e. the softmax model parameters for the 5 ‘outside’ subclasses come directly from the corresponding polygon edge specifications in . In general, such simple solutions will not always be obtained, since can represent a more complex set of semantic constraints derived from given maps and natural language processing models that restrict the meaning of certain human observations. The key point here, however, is that such constraints can be embedded into the likelihood and exploited by the fusion process in a mathematically sound way.

Figure 4.

(a) Polytope specification; (b) resulting subclass regions; (c) subclass probability surfaces with unit normals n_ji; (d) desired normals magnifed by 80; (e) non-convex likelihood for ‘outside’.

Likelihood model compression: The GM fusion approximation assumes that P(D = i|X) captures all information contained within a semantic human observation D. However, D can contain mixed information about different parts of X, e.g. target position and speed, but not heading. P(D = i|X) could be decomposed into more basic likelihoods that model relevant semantic information after D is parsed (e.g. by a natural language processing front-end, as discussed later). For instance, if D^a = i and D^b = j correspond to the observations ‘target near building’ and ‘target moving quickly away from building’, then the likelihood for the joint observation D = ([D^a = i] Λ [D^b = j]) can be modeled as the product of the corresponding softmax/MMS models (assuming both are conditionally independent given X),

For N_o observations D^o, o ∈ {1,…, N_o}, sequential processing via repeated application of the GM fusion and merging approximations could be very expensive and inefficient. Instead, we seek to extend the GM fusion method of²⁸ to handle the general case of ‘batch’ semantic measurement updates for fast online data fusion,

where a single ‘compressed’ softmax/MMS likelihood L(D^1:N_o |X) captures all information from the product , so that GM fusion and merging methods only need to be applied once. We exploit the fact that a product of N_o softmax/MMS models can be expressed as another softmax/MMS model for N_o conditionally independent semantic measurements,

where I = {i₁, i₂,…, iN_o} is the set of N_o class observations taken from the N_o softmax models, where i_o ∈ {1,…,m_o} and m_o is the number of classes for the oth softmax model. Assuming that the class labels are ordered within each softmax model, the product model parameters are defined as and . Each measurement i_o ∈ I comes from a different consituent softmax model of the product. Thus, P(D^1:N_o |X) can be exactly described as a single softmax likelihood over m₁ × m₂ × … m_n = m ‘product classes’, which appear in the denominator. The w_t and b_t terms cover all m combinations for the sum of the weights from the product of N_o softmax models.

Eq. (7) is computationally expensive to compute for online fusion for large N_o and m. As with GM compression algorithms, this motivates approximation of (7) via parameter compression techniques,

where m_* ≪ m, and the parameters , are based on some approximation technique. Ref. 37 presents two approximation techniques, geometric compression and neighborhood compression, that are based on the softmax model synthesis approach presented earlier. Geometric compression attempts to extract the relevant information in according to minimal set of log-odds boundaries needed to specify the resulting ‘product class’ polytope that appears in the numerator of (7). Neighborhood compression extends geometric compression by retaining additional ‘2nd order’ information about the polytopes for the neighboring classes of each observed class i_o ∈ I (where the neighboring polytopes of a given class in any softmax model can be determined offline). Both compression methods use linear programming to identify which class polytope boundaries should be kept in (8), and trade speed for accuracy in online GM fusion.

Fig. 5 shows results for compressing a product of three MMS models corresponding to the observation: ‘The target is inside the front yard, near the garage and the front porch’. The likelihoods for the single ‘inside’ and two ‘near’ observations are shifted/scaled from a base MMS model describing the ‘inside’ and ‘outside’ of the gray rectangular region in the upper left of Fig. 5. These results show a great speedup in computation for geometric compression (0.3 secs) over either the exact product or neighborhood fusion method (20 mins and 10 mins for GM fusion, respectively), for a fairly small and acceptable sacrifice in fusion accuracy.

Figure 5.

Comparison of MMS likelihood compression methods for three measurements: ‘Near the porch’, ‘Near the garage’, and ‘Inside the front yard’, shown by the grey area in the first plot.

3.3

Natural Language Chat Interfaces

A human could report soft data by composing structured observations from a list of available statements, as shown upper left in Fig. 6. This ‘direct selection’ interface bypasses the difficulties of natural language processing (NLP), but leads to several major limitations. Firstly, it restricts the human to a rigid pre-defined dictionary and message structure, which may not provide an intuitive or sufficiently rich set of semantics to convey desired information. Secondly, it is very inconvenient to select items one-by-one from a list for structured messaging; especially as the dictionary size m grows, this quickly becomes infeasible and time-consuming enough to render data irrelevant in dynamic settings. Furthermore, this approach does not scale well with environment/problem complexity and lexical richness for soft data reporting. In particular, it is often desirable to provide observations that activate multiple semantic D terms, e.g. ‘Roy is by the table, heading slowly to the kitchen’ simultaneously provides location, orientation and velocity information.

Figure 6.

Soft data fusion PGMs for direct selection and natural language chat interfaces in target localization application.

We are developing an unstructured natural language chat interface to support fast and highly flexible ‘free-form’ soft data reporting. The chat interface should ideally support a wide range of semantics. However, it is highly non-trivial to deriving meaningful and contextually relevant dynamic state information from free-form chat observations O. Unlike structured messages, it is infeasible to explicitly construct likelihood functions p(O|X) in advance to find the Bayes posterior p(X|O). Many different chat messages can also convey similar kinds of information, leading to additional uncertainties in lexical meaning (i.e. possible translation errors) in addition to intrinsic semantic (state spatial meaning) uncertainty. For example, the phrases ‘That guy's moving past the books’, ‘Roy next to the bookcase going to kitchen’, and ‘He's nearby shelf heading left’ all overlap in the sense that they could all to essentially refer to a structured set of atomic phrases: ‘Target is near the bookcase; and Target is moving toward the kitchen’. Our approach to handle free-form chat inputs separately accounts for lexical/translation uncertainties (using off-the-shelf NLP components, e.g. probabilistic syntactic parsers and sense-matchers) and semantic/meaning uncertainties (via state filtering) in a statistically consistent way that avoids joint reasoning over a large set of latent variables. The main idea is to translate a given Ok at time k into a reasonable ‘on the fly’ estimate of the likelihood p(O_k|X_k) via a very large (possibly expandable) dictionary of latent semantic observations, which have known generalized softmax likelihoods . As illustrated in the bottom of Figure 6, given the expansion

(where O_k is conditionally independent of X_k given ), we can generally approximate as the second summation term on the RHS, where accounts for the lexical uncertainty and accounts for the semantic uncertainty. Since O_k may also point to multiple soft observations (i.e. target position and velocity), the likelihoods on the RHS generally could correspond to unique products of independent dictionary terms, e.g. as in eq. (7). The general problem, then, is to identify how Dⁱ likelihoods (or sets of soft observations) should be ‘activated' for a given O input by identifying the scalars p(O|Dⁱ = j). Since p(X)p(D|X) can generally be approximated as a GM, it follows then that the LHS p(X|O) generally leads to a ‘mixture of GMs’.

Ref. 38 details one approach for estimating from chat inputs using off-the-shelf NLP tools. We focus here on the problems of phrase parsing and sense matching, i.e. matching human-generated synonyms and phrases from input chat messages O_k to corresponding words that lead to valid sets of semantic observations D_k in a fixed dictionary. Phrase parsing classifies raw input chat messages O_k into recognizable clusters of word tokens according into a set of predefined Target Description Clauses (TDCs) T_k. TDCs are conceptually similar to Spatial Description Clauses (SDCs) used in³⁹ for natural language control of robots, and allow the fusion engine to construct base templates for acceptable message types in terms of expected parts of speech and topical content. This allows the fusion engine to reject unhelpful semantic observations, e.g. ‘It is a nice day outside’. Then, the key issue for word sense matching (for each part of a given TDC) becomes deriving the relationship between the estimated 13 million tokens in the English language and the comparatively minuscule number of tokens in a set of predefined template semantic soft data observations. Recent efforts, particularly by Mikolov et al.,⁴⁰ introduced a negative sampling-based approach to efficiently learn multi-dimensional vector representations of all tokens in a vocabulary. Using their word2vec tool, we can develop a mapping between a set of tokens contained within an unstructured utterance and a set of tokens contained within a structured semantic template from the dictionary. Conditional probabilities from the parsing and word sense matching tools can be combined to form a score s(D_k, T_k) for each possible latent template statement D_k and parsed TDC T_k.

Figure 7 shows initial proof-of-concept results, using arg maxD_k s(D_k, T_k) to select template sensor statements that are most similar to the input tokenization for four correctly tokenized test phrases. In this example, 2682 possible template statements are considered, which covers 79.81% of the 208 input sentences collected in a pilot experiment with 12 human participants. From left-to-right, the four columns demonstrate the effects of increasing dissimilarity with template statements: the first input sentence is exactly a sensor statement template; the second input sentences replaces a spatial relation template token, ‘near’, with a non-template token, ‘next to’; the third input sentence replaces the grounding and changes the positivity; and the fourth is an imprecise reformulation of the first sentence. The results are promising, as the top-scoring statements are all qualitatively similar to the original phrase and produce sensible fusion results.

Figure 7.

Labeled indoor map, target location GM prior, and resulting GM posteriors for sense matching, with unstructured input phrases and corresponding template dictionary scores.

3.4

Optimal Value of Information Querying for Active Soft Data Fusion

We have thus far only considered passive data fusion strategies, where human sensors voluntarily provide observations as they see fit. However, semantic data fusion can also be extended to incorporate active soft sensing, i.e. intelligent ‘closed-loop’ querying of human sensors to gather information that would be most beneficial for complex machine planning and perception tasks. Active sensing problems have a rich tradition in target tracking and controls communities, but have focused on hard data sources such as radar, lidar, cameras, etc. One particularly relevant issue is the sensor scheduling problem, which seeks optimal selection of sensing assets given constraints on how many can be tasked to deliver quality data at any given instant. This problem also applies to scheduling of interactions between human sensors and autonomous agents: what is the most valuable soft information to request from humans, and when/how should such soft information be obtained? These issues can be tackled within formal planning frameworks that seek to maximize the value of information (VOI) under uncertainty.²⁷

Considering the target localization problem, suppose , where denotes a specific kind of binary soft observation report at time k. For instance, could represent a detection/no detection event for camera j, or a binary true/false response to a semantic query j from a large list of possible queries generated by a fixed map and dictionary (e.g. ‘yes/no’ queries such as ‘Is something by the door?’, ‘Is something behind the table?’,…, etc.). Given a utility function U(D_k, X_k) representing the expected long-term benefit of taking some discrete action X_k while the target is in state X_k, the VOI for receiving a single noisy report in response to a soft data query is

where E [f]_(v) is the expected value of f over v. Assuming cost c() for requesting , the human sensor should be queried for dk if . Thus, (9) gives a formal way to assess whether asking for is worth the cost, regardless of the outcome. All n_s alternatives for can thus be compared to select the one with the highest VOI at each time k. Such myopic querying strategies do not consider all possible combinations of that could be taken together at time k, but are practically implementable and still capture the bulk of information to be gained from querying. For human sensors, can be related to the expected cognitive cost of re-tasking human sensors.²⁷ For now, is ignored for simplicity, so only the utility defined by expected information gain for sensing actions is considered. Here, A_k is related only to the choice of j ∈ {1,…, n_s} and we seek to minimize the entropy of , so that . Thus, the VOI for in (9) is the expected decrease in posterior entropy,

This means that soft data will be (myopically) requested to minimize the entropy of p(X_k|D_1:k). Entropy minimization is also widely used for tasking of hard sensors in target localization applications, ⁴¹ and so provides a useful common objective for combined hard-soft sensor scheduling. However, VOI calculations are computationally expensive and lead to NP-hard Bayesian inference calculations for marginal observation likelihoods . The comparison of VOI for various reports also becomes expensive when n_s is large for large semantic dictionaries. Hence, even with simplifications such as myopic reasoning, optimal soft data querying remains challenging.

To address this, ref. 42 describes a novel querying policy approximation based on deep learning. This approach produces a multi-layer convolutional neural net (CNN) classification model to select among the n_s possible semantic queries to maximize the VOI eq. (9) given p(X_k|D_1:k-1). The CNN learning process uses training data labels for pairs of p(X_k|D_1:k-1) and , which are generated from brute force VOI optimizations obtained on simulated runs of the active sensing problem. The key advantage of this approach is that the major computational expense for generating a query is moved offline and can be implemented very quickly online once the CNN is learned. This is very useful for in problems where n_s is large (i.e. large semantic dictionaries and sets of possible semantic observations). Furthermore, the CNN deep learning approach can exploit non-obvious features of p(X_k|D_1:k-1) that lead to highly accurate predictions of VOI-optimal in complex scenarios.

Figure 8 shows an application of the CNN query policy approximation approach to a dynamic version of the target localization problem. The target moves with random walk dynamics in a 2D grid world, which is incompletely covered by 6 static cameras that can be accessed serially by a human supervisor. In this case, n_s = 6, and the problem here is to determine which camera j the human should look through at each time k for the best binary measurement (‘detection’ or ‘no detection’, with known false alarm and missed detection rates). The right plots show the resulting localization errors based on maximum a posteriori (MAP) estimates of the target location obtained from the post-fusion Bayesian posteriors p(X_k|D_1:k) at each time. The first histogram shows results using an alternative baseline sensor querying policy based on a partially observable Markov decision process (POMDP) model,⁴³ solved using the feature-based infinite horizon augmented MDP (AMDP) approximation. ¹⁹ The results here show that, despite the highly limited coverage area of the cameras, the CNN VOI approximation does a much better job of using querying sensors than AMDP, since it leads to much better tracking of the true target location. However, the CNN policy clearly rests on having adequate training data and an accurate model of the search environment; it is more brittle than the AMDP policy approximation to changes in camera layout or subtle changes in target dynamics (both of which are easily encoded as explicit POMDP parameter variations). A promising research direction to overcome such issues is to combine the strengths of AMDP and CNN policy approximations in a structured manner. To overcome the curse of dimensionality, we are also developing GM adaptations of these policy approximations. ⁴⁴

Figure 8.

(a) Snapshot of grid world problem, showing p(X_k|D_1:k) (heat map) for target randomly walking through field of 6 human-accessible cameras with limited coverage (rectangles); (b) histogram of MAP target state estimation errors for AMDP and CNN querying.