Due to the current global political situation, threats to critical maritime infrastructures such as LNG terminals, ports, offshore wind farms, subsea pipelines and cables tend to increase. The protection of infrastructures is essential to prevent deterioration of societal structures and to maintain essential services and functions that impact the well-being and livelihood of citizens. To counter this threat, permanent protection is necessary, which is difficult to achieve through surveillance technology operated by people for reasons of cost and personnel. One solution is to deploy a cooperating team (swarm) of unmanned surface vehicles (USVs) that coordinate with each other and independently carry out security tasks such as patrolling or surveillance and react intelligently to suspicious events.
Within the Fraunhofer project HUGIN “Heterogeneous Unmanned Group of INtelligent surface vehicles” the use of heterogeneous autonomous USVs for monitoring critical infrastructure and inspecting ship hulls was developed and tested. Up to three vehicles (two different platform types and heterogeneous sensor technology) were used. The team was assigned a joint task, which had to be divided into subtasks.
The subtasks were forwarded to the vehicles with the appropriate capabilities to be completed independently. The autonomous mission processing and coordination of the three USVs was investigated. The algorithms were tested in dynamic cooperative operation with parallel execution of specific subtasks. To enable the efficient, effective and flexible operation of such systems and the integration of the operator into the overall system, we used the Management by Objective (MbO) concept. MbO for parallel operating autonomous USVs is separated into three main aspects: Vehicle autonomy, Product creation control cycle, and Mission management.
Both, the autonomous cooperative behavior of the vehicles which affect vehicle autonomy as well as mission management and the operator-independent sensor data evaluation (product generation) were further developed and tested in HUGIN.
New algorithms for appropriate planning and dynamic control of pan-tilt-zoom sensors (stationary or mounted on mobile systems) in in context of reconnaissance missions have been developed at Fraunhofer IOSB. These algorithms are based on the prior proposed solutions for an efficient control of robotic vehicles and groups of heterogeneous robotic vehicles.
In this paper two specific algorithms (deterministic and non-deterministic) are referenced, and it is exploited how these algorithms - originally developed for the control of unmanned vehicles of potential different domains – can be adapted to also support the intelligent autonomous sensor control. The aim is to maximize the effectiveness of these sensors when used in reconnaissance missions.
The deterministic algorithm is based on extensive pre-planning that considers all relevant aspects of the task at hand and the optical sensors to be used like target area, restricted zones, fields of view, resolutions, zoom, and uses approximations and assumptions to determine the best possible area coverage. The non-deterministic algorithm does not undertake preplanning but rather provides basic behaviors and mission relevant compiled information that is used by the autonomous control system to identify the most reasonable actions based on the current situation.
Both algorithm types are suitable for the autonomous control of (heterogeneous) cooperative sensors without any operator interaction. To provide effective and efficient reconnaissance, the usage of each sensor assigned to the operation must be optimized and depending on the task ensure best possible coverage of the mission-relevant area, focus on certain areas by increasing the scanning cadence, etc. To provide sufficient image resolution and quality, the footprint should cover each specified target for a defined time period with a suitable zoom level. Changing alignment angles and relative positions must be continuously taken into account. Therefore, regarding sensors mounted on mobile systems (flying, swimming or driving) planning and control need to be fast and reliable in order to take the movements of the carrier platform into account. The theoretical foundations and practical approaches of the algorithms are compared and discussed.
In the last years Unmanned Vehicles for different environments (UxVs) have been recognized as relevant game changers and key technologies for a wide range of military and civilian applications. Even parallel deployment of heterogeneous autonomous assets as swarms or cooperating teams is no longer science fiction, but a realistic operational scenario. Their effectiveness can be significantly increased by optimization of number, capabilities and application strategy of assigned assets, particularly in time-critical tasks (e.g. the search for missing persons, disaster relief and modern warfare operations). Especially, the movements of the vehicles must be carefully managed. However, the use of small UxVs is often accompanied by limitations like short mission time and limited sensor coverage, which in turn can be compensated by the intelligent assignment of vehicles with increased autonomy in cooperating groups. An important basis for efficient and effective autonomous reconnaissance, especially in swarming or teaming scenarios, is movement optimization considering the capabilities of the heterogeneous vehicles equipped with different sensor systems operating in a combined mission. For this purpose, algorithms have been developed at Fraunhofer IOSB that enable appropriate planning and dynamic processing in very different situations for heterogeneous groups of cooperating vehicles. In the following, two exploration methods for reconnaissance missions are described in more detail and their possibilities are discussed and compared.
Current capabilities and sales volume of present-day UAVs (unmanned aerial vehicles) strongly demand counterUAV systems in a lot of applications to protect facilities or areas from misused or threatening drones. In order to reach a maximum detection and information gathering performance such systems need to combine different detection subsystems, i.e. based on visual optical, radar, and radio sensors. But available systems on the market are very expensive, the price is typically far over half a million dollars. Therefore, a far more cost-efficient solution has been developed which is presented in this paper. Four high-resolution visual optical cameras offer full 360 degree observation at distances up to several hundred meters. As soon as UAVs are visible in an image as small dots, they are detected and tracked with a GPU-based point target detector. Radar and radio sensor subsystems detect UAVs at higher distances. A full HD camera on a pan and tilt unit successively focuses on each found object to enable a convolutional neural network (CNN) to classify it with a higher local image resolution to identify UAVs and discard false alarms, e.g. from birds. Furthermore, drone type and payload are determined with CNNs, too, and a laser rangefinder on the pan and tilt unit measures the object distance. All information is collected and visualized in a 2D or 3D environmental map or situation representation on the base of geo-coordinates that are computed based on a RTK GNSS sensor self-localization. All software and hardware components are described in detail. The overall system is powerful, modular, scalable, and cost-efficient.
Applications of drones have been rapidly changing during the last years. The driver in development of drone systems in the past was the military. This changed due to the fast technological progress of drone systems in the private sector as well as the industrial market. Sinking costs, progressive miniaturization, functional enlargement and increasing performance and usability are key enabler for practical realization of previously only theoretical civil and military exertions. RD is currently developing systems-of-systems, grouping drones into swarms to solve or execute mostly non-complex tasks cooperatively to demonstrate feasibility with respect to pre-defined scenarios. The used mission management and control systems are often rudimental, non-dynamic and designed to serve only the corresponding scenarios. For real world applications of drone systems operating in cooperative groups this is insufficient, as flexible control mechanisms with respect to changing environments or mission targets are missing. This work addresses mission management and control as the central executing and overarching system glue, rendering effective and efficient application of drone swarms possible in the first place. Requirements to the command and control station, the operator as human in the loop and the assigned assets are investigated and consolidated into a novel approach. The system centric view is neglected in favour of a paradigm shift to macro control by introducing the "management by objective" approach based on prior work. The focus of mission control by the operator is moved from system-oriented control to a goal-oriented control focusing on results provided by the executing assets.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.