DIVINA

DIstributed cooperative VIsual Navigation for multi-uAv systems

Principal investigator: Vincent Frémont (Heudiasyc)

Nowadays, robotic vehicles are being used in a wide range of applications and scenarios, with significant improvements over past solutions. Empowered by fast paced technological and theoretical advances, we see a trend shift towards the use of large-scale, distributed formations of robots. Indeed, multi-robot systems are now recognized as a superior answer to many problems, taking advantage of important unique features such as flexible distributed sensing and intervention abilities, complementary heterogeneity, robustness to individual failures through redundancy, scalability in time and space, smaller deployment times, reduced operational costs and improved performances.
A key capability in multi-robot autonomous systems is collaborative Simultaneous Localization and Mapping in totally or partially unknown and GPS-denied environments, which has recently attracted the interest of many researchers. By sharing information between the robots, the performance of individual agents can be significantly improved, allowing for cooperatively performing complicated tasks in different domains including surveillance, search and rescue, object manipulation, etc.

In this challenge team, we are studying a fleet of Unmanned Aerial Vehicles (UAVs) whose main mission is to efficiently explore a totally or partially unknown geographic area, using on-board vision-based sensing through distributed control under local communications constraints. The fleet will be composed of autonomous UAVs with heterogeneous capabilities of control, communication and visual perception, as well as limited embedded resources, in terms of processing and energy. Indeed, each UAV can perform its localization, navigation and mapping in a non-supervised and non-coordinated manner, while maintaining a consistent topology to realize the common objective of efficiently exploring the unknown environment. Furthermore, the interactions among heterogeneous devices provided with different capabilities contribute to creating emergent behaviors in the fleet as a whole. Such a fleet of UAVs can be considered as a TSoS in which the integrated systems are heterogeneous and independently operable on their own, and networked together for the common goal of efficient exploration. The main originality of the project is to propose new algorithms beyond the state-of-the-art, based on networked robots and visual SLAM paradigms using algebraic graph theory and swarm intelligence. The idea is to optimize the UAV fleet behavior, ensuring collision-free flocking and efficient exploration strategies through distributed control, on-board visual perception capabilities and local communication constraints within a TSoS framework.

Isabelle Fantoni, Control and Vision.
Vincent Frémont, Computer Vision, Principal Investigator.
Enrico Natalizio, Sensor Networks and Swarm Communications.
Ahmet Sekercioglu, Visual Sensor Networks.
Eliseo Ferrante, Swarm Robotics.
Guillaume Sanahuja, Research Engineer.
Gildas Bayard, Research Engineer.

Post-doctoral positions:

Nicola Roberto ZEMA, “Networking and Control in Robotic Multi-SLAM DIVINA Project Environment”.

PhD Students:

Nicolas CAMBIER (2016-2019), “Bio-inspired collective exploration and cognitive mapping”.
Ariane SPAENLEHAUER (2016-2019), “Multi-Robots Exploration Strategies using Active Visual SLAM and Distributed Control Architectures”.
Nesrine MAHDOUI (2015-2018), “Robust Multi-Robots Visual SLAM using Heterogeneous Mobile Cameras Network”.

Master student’s projects:

Hicham TOUMI (2017), “Depth Map Estimation in Monocular Visual SLAM”.
Ariane SPAENLEHAUER (2016), “Visual Navigation strategies for Multi-Robot SLAM”.
Sarah SAOU (2016), “Multi-UAV Exploration Strategies”.
Nesrine MADHOUI (2015), “Multi-UAVs Network Communication Study for Distributed Visual Simultaneous Localization and Mapping”.

Fig. 1: Multi-UAV Simulation Environment and Network Topology Mode

Concerning the communication and control aspects, we have developed a new simulation environment able to integrate, in one platform, detailed control and telecommunications aspects (see Fig. 1). The simulator grants the team with the ability to perform all the necessary simulations for devised case studies involving the UAV fleet [1]. A novel algorithm and protocol that leverages “corrective consensus” has been also proposed [2]. The devised system will allow us to grant distributed synchronization capabilities, as well as fine-grained formation control, to the implemented network. A study on controlled mobility considering multiple node placements to support high-bandwidth video streams is also under investigation. As a future work, we will study the possibility for DIVINA project’s nodes to reach high bandwidth and QoS constraints when sharing the SLAM’s video streams by optimized positioning and relaying.

Fig. 2: Active SLAM System and metric distances recovery by fusing inertial measurements with monocular pose estimation

From a perception and control viewpoint, we focused on multi-Robots exploration strategies using active visual SLAM and distributed control architectures (see Fig. 2). For that purpose, we are currently working on the design of a filter that will provide us with the metric scale we need to achieve for map merging, and better pose estimates for the control of the drones. This work has been submitted to IEEE MFI 2017. Future works will be dedicated to improving the metric distance estimation method by using factor graphs and to designing of a method to compute uncertainty in SLAM algorithm using Bundle Adjustment optimization method.

Fig. 3: Multi-UAV RGB-D SLAM Architecture

Concerning the communication and perception aspects, our main objective is to improve the localization and mapping process by means of enhanced local perception and inter-robots bio-inspired communication using collective schemes of management (see Fig. 3). In this case, each robot uses a RGB-D (color and depth information) camera to compute the relative motion between two consecutive poses [4][5]. This visual odometry is fused with IMU (Inertial Measurement Unit) data to achieve a more accurate robot state estimation and to avoid drift in time.

Using its trajectory within its local environment, each robot builds a local map that can even be enhanced by integrating other robots’ local maps [3]. The sub-maps are then exchanged among robots to create a consistent global mapping (see Fig. 4). Each robot will send its constructed local map to a robot proclaimed "Leader" within its communication range. Each robot may be a "Leader" if it satisfies certain conditions. Based on the fused local maps and obeying an exploration strategy (Frontiers-based Cooperative Navigation), the "Leader" assign to each robot a goal target to reach. In addition, members of the fleet exchange with their neighbor’s information about relative localization of other robots detected using visual fiducial system. This work has been submitted to IEEE MFI 2017. As future works, we plan to run real world environment experiments with the proposed exploration strategy on-board MAV and to increase the number of robots within the fleet. We also plan to extend our approach to use relative localization techniques within our exploration strategy. From the networking side, we plan to investigation new communication protocols to improve the data exchange robustness.

Fig. 4: Multi-UAV Local Maps Creation and Frontiers-based Cooperative Navigation

Finally, we started to work toward exploration strategies using swarm intelligence. We proposed evolutionary linguistic models such as the naming game to offer robotic swarms a way to make collective decisions when several unforeseen alternatives are available. So far, we concentrated on implementing such a model in an aggregation behaviour and we submitted a paper for the SWARM 20175 international symposium. Another submission, focusing on the new dynamics of the Naming Game (converging on several words instead of one) under collective aggregation, is currently in preparation for the Second International Workshop of Social Learning and Cultural Evolution6 (SLACE). Future works will be related to extend this work by performing experiments in more environmental conditions to fully characterize the dynamics and by introducing more variations in the parameters, both to confirm already visible trends and to study their scalability. Also, we will investigate the advantages of the Naming Game’s decision-making aspect for collaborative SLAM.

Publications

[1] Zema N.-R., Trotta A., Sanahuja G., Natalizio E., Di Felice M. and Bononi L. “CUSCUS: An integrated simulation architecture for Distributed Networked Control Systems», 14^th Annual IEEE Consumer Communications & Networking Conference (CCNC 2017) Las Vegas, USA.

[2] Sabato Manfredi, Claudio Pascariello, Nicola Roberto Zema, Isabelle Fantoni. A cooperative packet loss-tolerant algorithm for wireless networked robots rendezvous. IEEE 2017 International Conference on Computing, Networking and Communications (ICNC): Workshop (ICNC'17 WS), Silicon Valley (United States) 2017.

[3] Mahdoui N., Natalizio E. and Fremont V. “Multi-UAVs network communication study for distributed visual simultaneous localization and mapping”, International Conference on Computing, Networking and Communications (ICNC), 2016.

[4] Wang X., Sekercioglu Y. A., Drummond T., Natalizio E., Fantoni I. and Fremont V. "Fast Depth Video Compression for Mobile RGB-D Sensors", IEEE Transactions on Circuits and Systems for Video Technology, Vol. 26, No. 4, Pages 673-686, April 2016.

[5] Wang X., Sekercioglu Y. A., Drummond T., Natalizio E., Fantoni I. and Fremont V. “Collaborative Multi-Sensor Image Transmission and Data Fusion in Mobile Visual Sensor Networks Equipped with RGB-D Cameras”, IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI 2016), Baden-Baden, Germany, September 2016.