An open visual-inertial mapping framework: maplab

This repository contains maplab, an open, research-oriented visual-inertial mapping framework, written in C++, for creating, processing and manipulating multi-session maps. On the one hand, maplab can be considered as a ready-to-use visual-inertial mapping and localization system. On the other hand, maplab provides the research community with a collection of multi-session mapping tools that include map merging, visual-inertial batch optimization, and loop closure.

Furthermore, it includes an online frontend, ROVIOLI, that can create visual-inertial maps and also track a global drift-free pose within a localization map.

Overview on List of Publications

The UP-Drive project resulted in 45 scientific publications in journals, international conferences and workshops.

The blog entries below provide an overview of these publications including links to their open access versions.


Motion Prediction Influence on the Pedestrian Intention Estimation Near a Zebra Crossing

J. Škovierová, A. Vobecký, M. Uller, R. Škoviera, V. Hlaváč

4th International Conference on Vehicle Technology and Intelligent Transport Systems (VEHITS 2018)

The reported work contributes to the self-driving car efforts, more specifically to scenario understanding from the ego-car point of view. We focus on estimating the intentions of pedestrians near a zebra crossing. First,we predict the future motion of detected pedestrians in a three seconds time horizon. Second, we estimate the intention of each pedestrian to cross the street using a Bayesian network. Results indicate, that the dependence between the error rate of motion prediction and the intention estimation is sub-linear. Thus, despite the lower performance of motion prediction for the time scope larger than one second, the intention estimation remains relatively stable

Paper (.pdf)

Robust Maximum-likelihood On-Line LiDAR-to-Camera Calibration Monitoring and Refinement

J. Moravec, R. Sara

Proceedings of the 23rd Computer Vision Winter Workshop, Cesky Krumlov, Czech Republic, pp. 27-35. February 5-7, 2018.

We present a novel method for online LiDAR–Camera system calibration tracking and refinement.  The method is correspondence-free, formulated as a maximum-likelihood learning task. It is based on a consistency of projected LiDAR point cloud corners and optical image edges. The likelihood function is robustified using a model in which the inlier /outlier label for the image edge pixel is marginalized out. The learning is performed by a stochastic on-line algorithm that includes a delayed learning mechanism improving its stability. Ground-truth experimental results are shown on KITTI sequences with known reference calibration. Assuming motion-compensated LiDAR data the method is able to track synthetic rotation calibration drift with about 0.06 degree accuracy in yaw and roll angles and 0.1 degree accuracy in the pitch angle. The basin of attraction of the optimization is about plus minus 1.2 degree.  The method is able to track rotation calibration parameter drift of 0.02 degree per measurement mini-batch.  Full convergence occurs after about 50 mini-batches.  We conclude the method is suitable for real-scene driving scenarios.

Paper (.pdf)

Real-time Semantic Segmentation-based Depth Upsampling using Deep Learning

V. Miclea, S. Nedevschi

Proceedings of 2018 IEEE Intelligent Vehicles Symposium (IV), Changshu, 2018, pp. 300-306.

We propose a new real-time depth upsampling method based on convolutional neural networks (CNNs) that uses the local context provided by semantic information. Two solutions based on convolutional networks are introduced, modeled according to the level of sparsity given by the depth sensor. While first CNN upsamples data from a partial-dense input, the second one uses dilated convolutions as means to cope with sparse inputs from cost-effective depth sensors. Experiments over data extracted from Kitti dataset highlight the performance of our methods while running in real-time (11 ms for the first case and 17 ms for the second) on a regular GPU.


Multi-Object tracking of 3D cuboids using aggregated features

M.P. Muresan, S. Nedevschi

Proceedings of 2019 15th IEEE International Conference on Intelligent Computer Communication and Processing (ICCP), Cluj-Napoca, Romania, September 5-7, 2019, pp. 11-18.

The unknown correspondences of measurements and targets, referred to as data association, is one of the main challenges of multi-target tracking. Each new measurement received could be the continuation of some previously detected target, the first detection of a new target or a false alarm. Tracking 3D cuboids, is particularly difficult due to the high amount of data, which can include erroneous or noisy information coming from sensors, that can lead to false measurements, detections from an unknown number of objects which may not be consistent over frames or varying object properties like dimension and orientation. In the self-driving car context, the target tracking module holds an important role due to the fact that the ego vehicle has to make predictions regarding the position and velocity of the surrounding objects in the next time epoch, plan for actions and make the correct decisions. To tackle the above mentioned problems and other issues coming from the self-driving car processing pipeline we propose three original contributions: 1) designing a novel affinity measurement function to associate measurements and targets using multiple types of features coming from LIDAR and camera, 2) a context aware descriptor for 3D objects that improves the data association process, 3) a framework that includes a module for tracking dimensions and orientation of objects. The implemented solution runs in real time and experiments that were performed on real world urban scenarios prove that the presented method is effective and robust even in a highly dynamic environment.