This PDF file contains the front matter associated with SPIE Proceedings Volume 10802, including the Title Page, Copyright information, Table of Contents, Author and Conference Committee lists.

Energy dispersive X-ray diffraction (EDXRD) is proposed to be a promising technique for identifying dangerous materials. This paper focuses on the development of the EDXRD system. A high-energy-resolution system using a room temperature detector was established. The diffraction angle of the system was adjustable depending on the detected samples and some experiments were carried out to study the suitable diffraction angle for different samples. At a nominal diffraction angle of 5°, the energy resolution of the system was approximately 6-9%. The system could effectively inspect the concealed samples and obtain the distinguishable profiles through the packaging. These experimental results demonstrated that the EDXRD system could satisfy the requirement of security inspection.

Materials can produce scattering effects on incident photons and ultrasound waves, by collecting scattered signal, the objects can be imaged. Backscattering imaging is a unique technique whereby the photons or ultrasound from an object in the generally backward direction are detected. Backscatter technology has been widely used in astronomy^{1, 2}, medicine^{3, 4}, security inspection^5-9 and other fields. Especially in the field of security, X-ray transmission is also the main means of inspecting dangerous goods, such as gun, knife, explosive, drug and so on. This technique is effective for detecting metal objects but difficult for organics such as drug and plastic explosive, because the attenuation coefficient of most organic matter to X-ray radiation is very low, the material appears to be transparent. The main elements of drug and plastic explosive are carbon, hydrogen, oxygen and nitrogen, which can produce strong scattering for incident X-ray. And, by collecting backscattered X-ray, bright images of organics such as drug and plastic explosive can be obtained. Relative to X-ray transmission, the backscattering imaging has three advantages: First, the backscatter images highlight organic materials, which appear significantly brighter than most metallic or inorganic materials. As a result, organic threats or contraband materials such as explosives and drugs can be more easily detected in the backscatter images than in the corresponding transmission images. Second, by changing the energy of X-ray, backscattering imaging can be utilized for detecting dangerous matters hidden inside or on the surface of objects, such as container and human body. Third, X-ray detectors can be placed on the same side of the objects being inspected as the X-ray source, which allow inspection of moving car or container vehicles by mounting the detectors and X-ray source on a mobile platform that drives past the objects of interest¹⁰. In addition, the equipment based on this technology can be miniaturized, which enable to extend the devise into a narrow space (car, container, etc.) and scan internally. So, X-ray backscattering imaging has become a significant method for detecting dangerous organics in the field of security inspection. At present, several inspection technologies based on X-ray backscattering imaging have been developed, for example, “Flying Spot” scanner

5

, coded aperture imaging9 and lobster-eye imaging^6-8, etc.

We discuss an appearance of the false absorption frequencies which are caused by process of the frequency conversion at action of the broadband THz pulse. The physical reason of their occurrence is discussed. A way for definition of the false absorption frequencies is proposed. A few experiments, which suppose the existence of such frequencies, are presented. However, we propose using of the up-conversion of THz frequencies for the detection and identification of substances.

It is known that development of optical sensors for explosives detection is currently of great interest. Among others sensors based on the luminescence quenching of conjugated polymers caused by photoinduced electron transfer have attracted considerable attention. Embedding such polymers into porous silicon (pSi) microcavity (MC) allows modify its luminescence spectrum and increase specific surface area and sensitivity of sensor. At the same time optimization of pSi MC structure and its mode of operation are important aspects of sensors design. This study presents the results of the structure and temperature optimization of pSi MC with embedded PPV derivatives polymers. The pSi MCs were fabricated using a standard electrochemical etching process. The luminescence spectra of polymers were drastically narrowed after embedding in pSi MC. It was experimentally found that optimal thickness of the front mirror is from 4 to 5 pairs of low and high porosity layers. The optimal thickness of the rear mirror is about 15 pairs of low and high porosity layers. We also discovered that temperature of pSi MC strongly influences on the rate of the polymer luminescence quenching under exposure to TNT vapors. In particular, it was shown that a decrease of MC temperature to 5° C leads to more than three times drop of quenching time. The obtained results can be applied for the design of optical sensors of explosives based on pSi MC.

In 1998 the US Commission on National Security in the 21st Century stated biological agents were the most likely weapon of choice for disaffected states and groups. The 2001 Anthrax letters case highlighted this threat of bacterial biological warfare agents, resulting in 5 deaths and a clean up costing hundreds of millions of US government dollars. Biological warfare agents (BWAs) appeal to disaffected states and groups due their ease of relatively cheap manufacture, small doses required for maximum damage, and difficulty to trace. Being able to rapidly identify which bacterium was used in an incident would lead to faster, specific support to victims, as well as providing vital information to aid front line personnel. Within a healthcare environment knowing the specific identity of bacterium allows the correct antibiotics to be prescribed and gives important information on any necessary quarantine conditions required to prevent the spread of infection. Forensically, strain level identification of BWAs can help determine the original laboratory source of bacteria and medically aid precise diagnosis and treatment. Strain level identification is currently very challenging and time consuming. Methods such as MALDI-TOF struggle with strain level identification due to the samples being too similar. Vibrational spectroscopies fingerprint of information and handheld instruments provides an exciting, novel opportunity to provide more detailed bacterial identification. Bacillus anthracis presents itself as an effective bioweapon not only due to its mortality rates, but its ability to form spores. Spores protect the bacteria from harsher conditions and can remain dormant for vast lengths of time until effectively awoken when more desirable growth conditions. Gruinard Island is an examples of bacilli spores’ survival strength. ATR-FTIR is a well proven technique for investigating biological molecules. However, one significant issue with the technique is the use of expensive fixed internal reflective elements (IRE’s), e.g. diamond, and thus drying times. Water is a strong infra-red absorber, resulting in the need to dry samples to reveal the detailed information lying underneath H2O peaks. Here, ATR-FTIR has been developed to encompass a more high throughput approach, ditching expensive IREs for cheap, disposable silicon IREs SIREs). With these IREs samples can be batch dried and analysed much more rapidly. Their disposable nature removes the need to clean in-between samples, reducing the risk of contamination. This work aims to differentiate bacilli samples capable of forming spores, both in their spore and vegetative state using this novel SIRE approach. With the current turbulence seen amongst defect states and militant groups the threat of biowarfare being used is increasing and accurate, rapid identification is crucial. This paper reports the use of a novel method of high throughput vibrational spectroscopy to demonstrate the use of FTIR for analysis of bacteria. This work will present a bacteria study, involving biological warfare surrogates, with the aim of advancing FTIR into a higher throughput, rapid approach to bacterial identification which can identify down to the strain level.

The U.S. Army Research Development Engineering Command Chemical Biological Center (RDECOM C&B) continues to develop technologies for the forensic detection of energetic materials and illicit drugs of abuse due to their recent confluence in counter terrorism operations. One specific technology developed here is the use of Raman Chemical imaging to detect these substances located concomitant with residual latent fingerprints. This study demonstrates the ability to identify threat materials non-destructively so that the fingerprint remains intact for further biometric analysis. Utilizing Raman spectroscopy, the Generation I Chemical Fingerprint Identification System (CFIS) semi-autonomously locates and identifies particles of interest found on the friction ridge of a given recovered fingerprint with minimal input from the operator. This work presents results from a collaborative effort between the U.S. Defense Forensic Science Center (DFSC) and RDECOM C&B in which two prototype CFIS systems were assessed with a variety of samples and examines additional practical considerations leading toward the development of the next generation of expeditionary systems for military forensic analysis.

Border security challenges and immigration issues are increasing considerably in recent years. Counterfeiting and fraudulent use of identity and other travel documents are posing serious threats and safety concerns worldwide, ever since the advancement of computers, photocopiers, printers and scanners. Considering the current scenario of illegal migration and terrorism across the world, advanced technologies and improved security features are essential to enhance border security and to enable smooth transits. In this paper, we present a novel dual waveguide based invisible fluorescence security feature and a simple validation system to elevate and strengthen the security at border controls. The validation system consists of an LED (light emitting diode) as excitation source and an array photodetector which helps in the simultaneous detection of multiple features from the fluorescence waveguides. The fluorescence waveguides can be embedded into the identity document as micro-threads or tags which are invisible to the naked eye and are only machine readable. In order to improve the sensitivity, rare earth fluorescence materials are used which absorb only specific ultraviolet (UV) or visible (VIS) wavelengths to create corresponding fluorescent emission lines in the visible or infrared wavelengths. Herein, we present the preliminary results based on the fluorescence spectroscopic studies carried out on the fabricated rare earth doped waveguides. The effect of different rare earth concentrations and excitation wavelengths on the fluorescence intensity were investigated.

The possibility of optimizing the parameters of laser radiation for increasing the efficiency of ionization of vapors of explosives in a laser field asymmetric ion mobility spectrometer (FAIMS) was investigated. The dependences of TNT ion signal on intensity of laser radiation, pulse energy and repetition frequency of laser pulses were obtained. Optimum value of the intensity of laser radiation for ionization in air was experimentally obtained - 3•10⁷ W/cm². It is shown that defocusing of laser radiation and expansion of the ionization region increase the TNT signal if the radiation intensity remains at the optimum level. It is shown that repetition rate should be such that the interval between pulses is shorter than the time of diffusion of non-ionized molecules into the laser irradiation region.

A scheme for a multi-pass laser beam through the ion source of a spectrometer was proposed. The scheme will effectively use laser radiation and improve the ionization efficiency of nitro aromatic molecules.

One of the possible ways to increase sensitivity of ion mobility spectrometers for explosives detection is the use of dopants - substances enhancing ion formation. The specificity of laser ionization in ion mobility spectrometry with regard to possible dopants - ion-formation enhancers - remains still unclear when detecting nitro compounds. The purpose of this work was to study the opportunities to increase sensitivity of detection of explosives at the presence of dopants under laser ionization of sample vapors. Trinitrotoluene (TNT) was used as an explosive. Toluene, naphthalene and chloroform were chosen as dopants, as they differ in the magnitude of ionization potential and in the value of electron affinity. The laser radiation had a wavelength of λ=266 nm and a repetition rate of 10 Hz. It was found that the use of toluene at laser intensities (0.04÷3).10⁷ W/cm² leads to an increase of TNT ion signal by 50%. The use of naphthalene leads to an increase of TNT ion signal by 15%. It was found that chloroform reduces the efficiency of formation of negative ions of TNT due to the capture of free electrons. This is caused by its high electron affinity and the impossibility of two-step ionization mechanism of chloroform for the laser intensities used.

As forensic science technologies progress, digital photography in the crime scene documentation is being replaced in favor of high-precision 3D measurements. Three-dimensional documentation presents every object in the context of the entire crime scene and allows accurate measurements between potentially important traces, like bloodstains or weapons. These and other advanced 3D documentation and analysis tools have improved the possibilities of investigation to a previously unattainable level. We present a novel solution for detailed 3D documentation, which overcomes the limitations of commonly used 3D measurement techniques, e.g. highly accurate Structured Light Scanning or convenient to use Structure from Motion. Our solution, called active-SfM, involves the use of special projection devices to project a random pattern on the part of the scene under measurement. This modification makes the measurement process robust and reliable, even when measuring featureless surfaces. The reconstructed 3D model has better quality and surface uniformity than the result of standard Structure from Motion measurements. Moreover, the acquisition process remains as quick and easy to use as before the modifications.

We present newly developed equipment: wireless projection devices and controllers, that were designed especially to be used by forensic technicians on crime scenes and that are compatible with stock cameras used by them in everyday practice. We also present a full set of developed algorithms that transform input images into the final 3D model.

The proposed solution complements the hierarchical, three-dimensional measurement system developed in recent years by Polish Central Forensic Laboratory of the Police, CYBID Ltd., and Warsaw University of Technology, designed especially for crime scene documentation. The whole documentation process is supervised by a specialized CrimeView3D application, a software platform for measurement management and data visualization. We also present the outcome of measurement sessions that were conducted on both simulated and real crime scenes with the cooperation of Technicians from Central Forensic Laboratory of Police.

This paper discusses a modular system architecture for detection, classification and localisation of Unmanned Aerial System (UAS) targets, consisting of intelligent Autonomous Sensor Modules (ASMs), a High-Level Decision Making Module (HLDMM), a middleware integration layer and an end-user GUI, under the previously-reported SAPIENT framework. This enables plug and play sensor integration and autonomous fusion, including prediction of the trajectory of the vehicle for sensor cueing, multi-modal sensor fusion and target hand-off.

The SAPIENT Counter-UAS (C-UAS) system was successfully demonstrated in a live trial against a range of UAS targets flown in a variety of attack trajectories, using radar and Electro-Optic (EO) C-UAS ASMs. In addition, the trial also demonstrated the use of synthetic sensors, on their own and in combination with real sensors. Outputs of all the available sensors were tracked and fused by the Cubica SAPIENT HLDMM which then steered narrow field-of-view cameras onto the predicted 3D position of the UAS. The operator was provided with a map-based view showing alerts and tracks to provide situational awareness, together with snapshots from the EO sensor and video feeds from the steerable narrow field of view cameras.

This demonstrates an effective C-UAS system operating entirely autonomously, with autonomous detection, localisation, classification, tracking, fusion and sensor management, leading to “eyes on” the aerial threat, and all happening with zero operator intervention and in real-time.

Algorithms for the detection and tracking of (moving) objects can be combined into a system that automatically extracts relevant events from a large amount of video data. Such a system (data pipeline), can be particularly useful in video surveillance applications, notably to support analysts in retrieving information from hours of video while working under strict time constraints. Such data pipelines entail all sort of uncertainties, however, which can lead to erroneous detections being presented to the analyst. In this paper we present a novel method to attribute a confidence of correct detection to the output of a computer vision data pipeline. The method relies on a datadriven approach. A machine learning-based classifier is built to separate correct from erroneous detections. It is trained on features extracted from the pipeline; The features relate to both raw data properties, such as image quality, and to video content properties, such as detection characteristics. The validation of the results is done using two full motion video datasets from airborne platforms; the first being of the same type (same context) as the training set, the second being of a different type (new context). We conclude that the result of this classifier could be used to build a confidence of correct detection, separating the True Positives from the False Positives. This confidence can furthermore be used to prioritize the detections in order of reliability. This study concludes by identifying additional work measures needed to improve the robustness of the method.

Globalization has created complex economic and sociological dependencies. The nature of conflicts has changed and nations are confronted with a vast number of new threat scenarios. Information superiority is a question of being able to get the right information at the right time. Technology allows to disseminate information in near real-time and enables both aggressors and defenders to act remotely and network over time and space. Technologies in the areas of sensors and platforms as well as network technology and storage capability have evolved to a level where mass data can be easily shared and disseminated. To benefit fully from these new capabilities, there is a need for systems and services that can interact with each other in a well-defined interoperable way. On an organizational level it is necessary to define common processes to coordinate actors, their activities, the assets available and the data and information created. Security restrictions, (intellectual) property rights as well as data privacy regulations need to be fulfilled. The Coalition Shared Data (CSD) concept supports operational processes as defined by NATO within Joint ISR (Intelligence, Surveillance and Reconnaissance) and the Intelligence Cycle by defining standardized interfaces, data models, services and workflows. To support information provision additionally, techniques of data and information extraction, fusion and visual analysis can be added at the system level. Other available sources can be connected through the usage of semantic world models. To ensure data integrity multilevel security measures need to be combined with the existing concept.

The publication introduces the operational processes defined within NATO doctrines and process descriptions and maps the CSD concept to it. It describes the new Edition of STANAG (NATO Standardization Agreement) 4559 Edition 4 that implements the CSD concept and connects it to operational processes. Based on this it introduces a system architecture for ISR Analytics.

Remote sensing is one of the most dynamically developing fields of science that can be applicable in security and biometrics. Application of various biometric systems has been increased in past decades due to growing demands of security. There are a lot of human characteristics that might be used in a biometric system to identify an individual, i.e., hand palm veins, fingerprints, iris recognition, etc. Hand geometry biometric is based on the shape of hands. Hand palm veins based biometry that uses the vascular system patterns is one of the most complex and can be utilized for personal identification. Therefore, the hand palm veins based biometry includes some of the most useful technologies for person identification. This paper examines the utility of hyperspectral imagery for personal identification based on the fusion of hand geometry and hand palms veins biometrics. The hyperspectral data are used to develop the methodology of fast imagery post-processing and data matching, which could be used to develop a low-cost hand biometric recognition system intended to make a highly efficient identification and to have a fast response and easy usage. The research was conducted using a hyperspectral pushbroom sensor in controlled laboratory conditions. The hand vein detection process is based on a camera that takes a picture of the subject’s veins under a radiation source. The application of hyperspectral data can be time-consuming. Therefore biometrics systems should use faster solutions. Experiment with a hyperspectral camera allowed to select the best wavelength in which acquisition of palm veins and hand geometry was optimal. The developed methodology will be next used to build a low-cost system with one panchromatic frame camera and adequate interferometric filter, that would facilitate data acquisition, and identification process.

Reliable personal authentication is a critical and vital obligation to the security in all the real-world applications. Nevertheless, biometric features are effectively used for the personal authentication, in some cases the criminal impersonation is an easy task. The reproducible attack in fingerprint system and cost prohibitive nature of iris and facial based system limit the vast implementation of the same for personal authentication. Hence, considering the cases of increasing identity theft, there is more reason than ever to ensure the reliable and cost-effective personal authentication. The vein biometric identification system has been gaining increased attention in recent years. Anatomically, the shape of the vein pattern at the dorsal hand is unique for each even for identical twins and remains stable for a period. The objective of this work is to conduct a feasibility analysis of unique ID generation using dorsal hand vein pattern as the biometric authentication system for preventing identity theft of the valuable documents. Images of the vein pattern from the dorsal side of the hand are tried to capture using a regular type Smartphone camera. Vein pattern thus captured is used to extract the unique features. Numerical values generated from such unique feature are encrypted and used to create a unique ID. This paper discusses about the method for the dorsal hand vein pattern capturing, its feature extraction, conversion to the unique ID and its feasibility to integrate to the valuable documents.

In this work, we investigate biometrics applied on 2D faces in order to secure areas requiring high security level. Based on emerging deep learning methods (more precisely transfer learning) as well as two classical machine learning techniques (Support Vector Machines and Random Forest), different approaches have been used to perform person authentication. Preprocessing filtering steps of input images have been included before features extraction and selection. The goal has been to compare those in terms of processing time, storage size and authentication accuracy according to the number of input images (for the learning task) and preprocessing tasks. We focus on data-related aspects to store biometric information on a low storage capacity remote card (10Ko), not only in a high security context but also in terms of privacy control. The proposed solutions guarantee users the control of their own biometrics data. The study highlights the impact of preprocessing to perform real-time computation, preserving a relevant accuracy while reducing the amount of biometric data. Considering application constraints, this study concludes with a discussion dealing with the tradeoff of the available resources against the required performances to determine the most appropriate method.

Face recognition is a key task of computer vision research that has been employed in various security and surveillance applications. Recently, the importance of this task has risen with the improvements in the quality of sensors of cameras, as well as with the increasing coverage of camera networks setup everywhere in the cities. Moreover, biometry-based technologies have been developed for the last three decades and have been available on many devices such as the mobile phones. The goal is to identify people based on specific physiological landmarks. Faces are one of the most commonly utilized landmarks, due to the fact that facial recognition systems do not require any voluntary actions such as placing hands or fingers on a sensor, unlike the other bio-metric methods. In order to inhibit cyber-crimes and identity theft, the development of effective methods is necessary. In this paper, we address the face recognition problem by matching any face image visually with previously captured ones. Firstly, considering the challenges due to optical artifacts and environmental factors such as illumination changes and low resolution, in this paper, we deal with these problems by using convolutional neural networks (CNN) with state-of-the-art architecture, ResNet. Secondly, we make use of a large amount of data consisting of face images and train these networks with the help of our proposed loss function. Application of CNNs was proven to be effective in visual recognition compared to the traditional methods based on hand-crafted features. In this work, we further improve the performance by introducing a novel training policy, which utilizes quadruplet pairs. In order to ameliorate the learning process, we exploit several methods for generating quadruplet pairs from the dataset and define a new loss function corresponding to the generation policy. With the help of the proposed selection methods, we obtain improvement in classification accuracy, recall, and normalized mutual information. Finally, we report results for the end-to-end system for face recognition, performing both detection and classification.

Visual surveillance is of utmost importance for ensuring public safety and, detecting and preventing violent activities. The rapidly increasing number of surveillance cameras makes automated visual surveillance necessary, since monitoring a large of number of cameras by operators is not feasible, requiring a huge workload. In this paper, we propose a compact method for automated analysis of behaviours in crowds, specifically detecting the abnormal activities in crowd videos, which is one of the most critical applications of visual surveillance. The most intuitive way of abnormal activity detection is to consider common and typical activities in the scenes as normal and any unseen strange activities as anomalies that might be due to dangerous events. When tragic incidents such as accidents, disasters, shootings and violent behaviours happen, people tend to move in a very fast pace and in arbitrary directions. Thus, the proposed method consists of modelling the activities of crowds in the scenes during regular events, and analysing the spatial and temporal changes in their motion, which may be related to abnormal activities. For defining the crowd activities, first, crowd specific motion representations are computed. The computed representations utilize motion attributes such as speed, direction and acceleration of people in the crowds. Next, by employing these representations, typical activities in crowd videos, related to normal behaviours of people when no abnormal activities are present, are learned. Later, the distributions of motion representations are inspected; abrupt changes in the distributions of motion representations, occurring in several parts of the scenes, are labelled as anomalies. Experiments, conducted on a publicly available dataset, involving videos of crowds, reveal that the proposed method is effective in detecting abnormal activities. Additionally, quantitative performance of variants of the proposed method including the baseline approaches were measured for a comparison.

This paper presents a novel approach to sensing both suspicious, and task-specific behaviours through the use of advanced computational intelligence techniques. Locating suspicious activity in surveillance camera networks is an intensive task due to the volume of information and large numbers of camera sources to monitor. This results in countless hours of video data being streamed to disk without being screened by a human operator. To address this need, there are emerging video analytics solutions that have introduced new metrics such as people counting and route monitoring, alongside more traditional alerts such as motion detection. There are however few solutions that are sufficiently robust to reduce the need for human operators in these environments, and new approaches are needed to address the uncertainty in identifying and classifying human behaviours, autonomously, from a video stream. In this work we present an approach to address the autonomous identification of human behaviours derived from human pose analysis. Behavioural recognition is a significant challenge due to the complex subtleties that often make up an action; the large overlap in cues results in high levels of classification uncertainty. False alarms are significant impairments to autonomous detection and alerting systems, and over reporting can lead to systems being muted, disabled, or decommissioned. We present results on a Computational- Intelligence based Behaviour Recognition (CIBR) that utilises artificial intelligence to learn, optimise, and classify human activity. We achieve this through extraction of skeleton recognition of human forms within an image. A type-2 Fuzzy logic classifier then converts the human skeletal forms into a set of base atomic poses (standing, walking, etc.), after which a Markov-chain model is used to order a pose sequence. Through this method we are able to identify, with good accuracy, several classes of human behaviour that correlate with known suspicious, or anomalous, behaviours.

The paper presents a method for detection of abnormal behaviors related to violent events in urban environments. We have further developed a person detection method and introduced a person tracker as an intermediate stage between person detection and anomaly detection. The method is presented together with results from evaluations based on real sensor data that have been recorded in a realistic urban environment. Several scenarios have been recorded such as normal situations, a person loitering, a person shooting and people escaping from sudden smoke development. The method uses sensor data from a static video surveillance system, consisting of two visual and two thermal infrared cameras. The person detection method combines foreground segmentation with a trained machine learning algorithm which is based on boosting. The tracker associates incoming detections to objects (i.e. people), and generates tracks of world coordinates and velocities of objects. The anomaly detection method, which is based on the hidden Markov model (HMM), fuses information that is derived from the tracker. The information contains a number of persons, their positions and velocities. The results indicate that the person detection method in combination with the tracker can produce robust and accurate observations to the HMM, which in turn provide good conditions for accurate anomaly detection.

Surveillance systems are essential for the protection of military compounds and critical infrastructures. Automatic recognition of suspicious activities can augment human operators to find relevant threats. Common solutions for behavior analysis or action recognition require large amounts of training data. However, suspicious activities and threats are rare events, and the modus operandi of enemies may suddenly change, which makes it unrealistic to obtain sufficient training data in realistic situations. Therefore, we developed a demonstrator for the recognition of suspicious activity that allows users to easily define new alerts based on their expert knowledge. We developed basic modules for the computation of object detections, tracking and action recognition to generate features. The user is able to specify complex behavior with Symbols and Sentences. Symbols are low-level descriptions to analyze combinations of features. An example of a symbol is ‘approach_gate’, which consists of three conditions related to: the distance to the gate, the speed of a person and whether a person is on the compound. Sentences are high-level descriptions to analyze temporal ordering. An example of a sentence with temporal ordering is ‘entering the compound’, which consists of first approaching the gate, then entering the gate, and finally being on the compound. The demonstrator is compliant with the SAPIENT architecture, which uses multiple low-level Autonomous Sensor Modules (ASM) and a High-Level Decision Making Module (HLDMM). Features and symbols are computed in an ASM and sentences are computed in the HLDMM. Our demonstrator is tested on a multi-camera dataset to recognize suspicious behavior (e.g., digging for placement of improvised explosive devices, climbing over a fence, approaching the compound, and car standing on the road), allowing the user to interactively creates and modifies both symbols and sentences to mitigate new threats that were not implemented during design time.

This paper describes a framework for action recognition which aims to recognize the goals and activities of one or more human from a series of observations. We propose an approach for the human action recognition based on the 3D dense micro-block difference. The proposed algorithm is a two-stage procedure: (a) image preprocessing using a 3D Gabor filter and (b) a descriptor calculation using 3D dense micro-block difference with SVM classifier. At the first step, an efficient spatial computational scheme designed for the convolution with a bank of 3D Gabor filters is present. This filter intensifies motion using a convolution for a set of 3D patches and arbitrarily-oriented anisotropic Gaussian. For preprocessed frames, we calculate the local features such as 3D dense micro-block difference (3D DMD), which capture the local structure from the image patches at high scales. This approach is processing the small 3D blocks with different scales from frames which capture the microstructure from it. The proposed image representation is combined with fisher vector method and linear SVM classifier. We evaluate the proposed approach on the UCF50, HMDB51 and UCF101 databases. Experimental results demonstrate the effectiveness of the proposed approach on video with a stochastic textures background with comparisons of the state-of-the-art methods.

Entering confidential information is the classical application for optical skimming. For example entering a PIN or password to authenticate banking transactions is one of the most sensitive moments of a transaction. Up to now it was unclear how efficient optical skimming attacks can be mounted even on mobile devices. In this paper we show that filming the entering of a password with the camera of a standard mobile phone is enough for a fully automated recovering of the sensitive information. Our analysis method of the recorded video stream leads to a success rate of more than 90%. In our model a user enters his password into an Android or IOS driven touchscreen device while being filmed by the attacker’s smartphone. The goal of the attacker is to derive the password from the movie instantly, e.g. to use it in a real time man-in-the-middle attack. On first sight such an attack seems hard to mount due to many disturbing factors like movements of the device, bad light conditions etc. However, we show that many of these disturbing factors can be mitigated by smart video analytics. We implemented the whole attack in our lab simulating real conditions. Parts of the setup where a Samsung S7 for filming and an iPhone 6 as target. For real time processing we use the computer vision library OpenCV which supports most of the common image and video processing algorithms. Our goal is to cover most of the important cases while holding as little knowledge as possible about the video the algorithm has to work on. Starting from movable or instable position for camera and device we apply a discrete Fourier transformation to obtain a constant plan view as starting point for our algorithm. To obtain the necessary information for this the user has to select the four corners of the keyboard which is observed by hand. In real live situations the device is hold by hand of the user so we decided to use the MOSSE algorithm for stabilization to deal with the problem of a jiggling device. The keystroke detection is based on the “pop up” feature of common smartphone keyboards. This event is remarkable enough to get detected and located by simple gray value subtraction of following frames. Not only the magnified keys are the primary changes we observed and analyzed. Minor changes like noise can be removed by sufficiently blurring the frame. Subtracting areas where normal skin colors are detected yields information about the position of the keystrokes. The relative position obtains a probability for the pressed keys by analyzing the activity of the regions where the “pop ups” usually are. Our work showed that optical skimming of passwords are practical even on mobile devices. We show that it is possible to make a simple attack with basic computer vision techniques. Up to date mobile devices have enough computing power to provide for this kind of attacks. We give recommendations for entering passwords securely.

Fusion of thermal infrared and visual images is an important technique for real-time surveillance applications. Since image fusion is used in many real-time night vision applications such as target detection, recognition and tracking, it is important to understand the processing requirements and provide computationally efficient methods. In this paper, we present a real-time image fusion system designed for night vision supervisory and monitoring purposes. The system is equipped with two image sensors: TV and IR (thermal infrared). We implement a processing pipeline on the NVIDIA Tegra TX2. The TX2 platform is equipped with a many-core NVIDIA GPU and multi-core ARM CPU. Additionally, we present the system architecture as well as the design process of the efficient, real-time multi-spectral signal-processing algorithm. The algorithm is based on the second-generation wavelets also called lifting scheme. We show also a novel parallelization approach to perform the calculations in place, so no auxiliary memory is needed. This enables a fast parallel and pipelined processing flow. We achieve a considerable speedup compared to an optimized CPU implementation. The experimental results show that the fusion system can realize real-time image fusion processing for dual channels images at the rate of 30 frames per second for the Full-HD images.

Due to the increasing need for effective security measures and the integration of cameras in commercial products, a huge amount of visual data is created today. Law enforcement agencies (LEAs) are inspecting images and videos to find radicalization, propaganda for terrorist organizations and illegal products on darknet markets. This is time consuming. Instead of an undirected search, LEAs would like to adapt to new crimes and threats, and focus only on data from specific locations, persons or objects, which requires flexible interpretation of image content. Visual concept detection with deep convolutional neural networks (CNNs) is a crucial component to understand the image content. This paper has five contributions. The first contribution allows image-based geo-localization to estimate the origin of an image. CNNs and geotagged images are used to create a model that determines the location of an image by its pixel values. The second contribution enables analysis of fine-grained concepts to distinguish sub-categories in a generic concept. The proposed method encompasses data acquisition and cleaning and concept hierarchies. The third contribution is the recognition of person attributes (e.g., glasses or moustache) to enable query by textual description for a person. The person-attribute problem is treated as a specific sub-task of concept classification. The fourth contribution is an intuitive image annotation tool based on active learning. Active learning allows users to define novel concepts flexibly and train CNNs with minimal annotation effort. The fifth contribution increases the flexibility for LEAs in the query definition by using query expansion. Query expansion maps user queries to known and detectable concepts. Therefore, no prior knowledge of the detectable concepts is required for the users. The methods are validated on data with varying locations (popular and non-touristic locations), varying person attributes (CelebA dataset), and varying number of annotations.

In this article, the problem of the lack of robustness and reliability of surveillance systems through disturbing security irrelevant events such as tree shaking, birds flying, etc. is tackled. A novel scene analysis approach based on hypergraph-based trajectories is introduced for reducing the rate of false positives. The conception of hypergraph-based trajectories relaxes the notion of point-based trajectories by allowing multiple incidences between subsequent points in time. This allows a principled approach for the extraction of robust features based on bounding boxes resulting from existing 3rd party detection methods. The experimental part is based on data collected from single-view camera systems over a two-year non-stop recording in the frame of the Austrian KIRAS project SKIN1 on protecting critical infrastructure. The results show substantial reduction of irrelevant false alarms, hence improving the overall system’s performance.

Digital video forgery or manipulation is a modification of the digital video for fabrication, which includes frame sequence manipulations such as deleting, insertion and swapping. In this paper, we focus on the detection problem of deleted frames in videos. Frame dropping is a type of video manipulation where consecutive frames are deleted to skip content from the original video. The automatic detection of deleted frames is a challenging task in digital video forensics. This paper describes an approach using spatial-temporal analysis based on the convolution with a bank of 3D Gabor filters. Also, we use the 3D Convolutional Neural Network for frame drop detection for preprocessed frames. Experimental results demonstrate the effectiveness of the proposed approach on a test video database.

Tougher security requirements and an increasing number of terror attacks have led to rapid advances in the field of explosive detection in the past few years. Detection systems have to be tested and validated with realistic samples and amounts as early as possible during the development. However, at present, well defined and homogeneous test materials which are a crucial requirement for development and validation of trace detection systems are not available.

Existing preparation methods of reference samples, such as drop-and-dry (drop casting) methods, present a range of variability and reproducibility issues, including inhomogeneous sample coverage and material waste. Especially for the preparation of samples for optical stand-off technologies the drop-on-demand and inkjet printing technology could be a promising method of producing standardized chemically contaminated test materials with high accuracy, precision, scalability, and flexibility to allow for the inexpensive, high-throughput production.

Test samples with contaminations ranging from nanogrammes to microgrammes have been prepared and analysed over several weeks of storage. The influence of plot parameters on the morphology and durability of printed samples of various common explosives have been investigated on different substrates.

HPLC measurements were made to quantitatively evaluate the durability of printed samples. The morphology of the test samples was additionally characterized by optical microscopy and confocal Raman-microscopy. Besides the occurrence of polymorphic phase changes, described in literature for low concentrated drop-on-demand samples of RDX, we observed changes in sample distribution by recrystallization of some explosives in the printed samples.

This paper focuses on the optimization of methods for the preparation of test samples and on the analysis especially concerning the effect of polymorphic phase changes caused by inkjet printing of samples.

Explosive trace detection (ETD) systems need to be tested against the target substances on a nanogram scale applied to various different surfaces. To reach this goal a solvent-free method using inert particles coated with explosives was researched. A powder pipette was developed that is able to perform a dry transfer which can place a required number of particles on a target area to guarantee a reproducible generation of traces on different surfaces. A negative pressure is used to grasp a singular particle from a powder bed into the round pore of the pipette. The particle can be released by an electric stimulus of the pipette which is integrated into a platform solution enabling a fully automatic control. Combining the knowledge of the explosive amount on each particle and the possibility to perform the deposition of an exact number of particles it is possible to create reproducible and quantitative test specimen for explosive trace detection systems.

Explosive vapour detection (EVD) systems need to be tested with controlled concentrations of explosive vapour down to ppt level. A suited gas generator was developed taking into account measures to prevent unwished adsorption and to ensure a controlled gas concentration output. The developed gas generator is able to evaporate small amounts of dissolved explosives from a liquid circle into a gas stream. By controlling the gas stream, the concentration of explosive in the liquid circle and the numbers of pulses, it is possible to create defined explosive vapours down to ppt level.

This paper describes preliminary results of the interdisciplinary research and development venture lead by The Military University of Technology, Warsaw, Poland. The main task of the project is to develop remote solutions that support dogs guide during different special activities. According to the law-enforcement operational procedures, system has been designed basing on three different modules: special dogs electronic vest, mobile command center and supporting UAV (Unnamed Airborne Vehicle) platform. Project integrates such technologies as GNSS (Global Navigation Satellite System), IMU (Inertial Measurement Unit), remote sensing tools and UAV platforms as a support device. General idea of the project comes from boarder control and law-enforcement forces whose big scope of work is related with K-9 dogs activity. The main functionality of the system is to monitor and control the K-9 dog during high level risk operations. The biggest challenge of the project is to develop official training and operational procedures suitable to the applied technology.

This article presents general concept of the system and the results of validation process in scope of remote sensing (NIR , RGB) and navigation functionalities (GNSS). The set of tests has been taken with different methods and tools to establish efficient ergonomic solution. Possibilities of project application areas are also shown here in details.