Absolute Localization by Mono-camera for a Vehicle in Urban Area using Street View

Abstract : In a work made at Centre de Robotique and Institut VEDECOM, we studied robust visual urban localization systems for self-driving cars. Obtaining an exact pose from a monocular camera is difficult and cannot be applied to the current autonomous cars. We mainly focused on fully leveraging Geographical Information Systems (GIS) to achieve a low-cost, robust, accurate and global urban localization.The development of public GIS's has brought us a new horizon to address the localization problem but their tremendous amount of information, such as topological, semantic, metric maps, Street Views, depth maps, 3D cadastral maps and High Definition maps, has to be well analyzed and organized to extract relevant information for self-driving cars. Our first task was to design a robotic accessible offline database from a dense public GIS, namely Google Maps, which has the advantage to propose a worldwide coverage. We make a compact topometric representation for the dynamic urban environment by extracting four useful data from the GIS, including topologies, geo-coordinates, panoramic Street Views, and associated depth maps. At the same time, an online dataset was acquired with a low-cost camera equipped on VEDECOM vehicles. In order to make spheric Street Views compatible with the online imagery, an image warping and interpolation based transformation is introduced to render rectilinear images from Street Views.We proposed two localization methods: one is a handcrafted-features-based computer vision approach, the other is a convolutional neural network (convnet) based learning technique. In computer vision, extracting handcrafted features is a popular way to solve the image based positioning. We take advantages of the abundant sources from Google Maps and benefit from the topometric offline data structure to build a coarse-to-fine positioning, namely a topological place recognition process and then a metric pose estimation by a graph optimization. The method is tested on an urban environment and demonstrates both sub-meter accuracy and robustness to viewpoint changes, illumination and occlusion. Moreover, we demonstrate that sparse Street View locations produce a significant error in the metric pose estimation phase. Thus our former framework is refined by synthesizing more artificial Street Views to compensate the sparsity of original Street Views and improve the precision.The handcrafted feature based framework requires the image retrieval and graph optimization. It is hard to achieve in a real-time application. Since the GIS offers us a global scale geotagged database, it motivates us to regress global localizations from convnet features in an end-to-end manner. The previously constructed offline database is still insufficient for a convnet training. We hereby augment the originally constructed database by a thousand factor and take advantage of the transfer learning method to make our convnet regressor converge and have a good performance. In our test, the regressor can also give a global localization of an input camera image in real time.The results obtained by the two approaches provide us insights on the comparison and connection between handcrafted feature-based and convnet based methods. After analyzing and comparing the localization performances of both methods, we also talked about some perspectives to improve the localization robustness and precision towards the GIS-aided urban localization problem.
Document type :
Liste complète des métadonnées

Cited literature [106 references]  Display  Hide  Download

Contributor : Abes Star <>
Submitted on : Tuesday, August 28, 2018 - 12:01:11 PM
Last modification on : Wednesday, January 30, 2019 - 4:11:28 PM
Document(s) archivé(s) le : Thursday, November 29, 2018 - 2:36:53 PM


Version validated by the jury (STAR)


  • HAL Id : tel-01863297, version 1


Li Yu. Absolute Localization by Mono-camera for a Vehicle in Urban Area using Street View. Automatic. PSL Research University, 2018. English. 〈NNT : 2018PSLEM003〉. 〈tel-01863297〉



Record views


Files downloads