视觉深度估计与点云建图研究进展

陈苑锋

doi:10.37188/CJLCD.2021-0047

您当前的位置：

首页 >

文章列表页 >

视觉深度估计与点云建图研究进展

更新时间：2021-04-09

- 视觉深度估计与点云建图研究进展
- Progress of visual depth estimation and point cloud mapping
- 液晶与显示 2021年页码：1-18
- 作者机构：
  
  美的集团（上海）有限公司，上海 201799
- 作者简介：
  
  [ "陈苑锋(1979-)，男，福建安溪人，博士，工程师，2006年于复旦大学获得博士学位，主要从事机器人方向的研究。E-mail: yuanfeng_chen@hotmail.com" ]
- 基金信息：
- DOI：10.37188/CJLCD.2021-0047
  中图分类号： TP394.1;TH691.9
- 收稿日期：2021-02-24，
  
  修回日期：2021-03-20，
- 稿件说明：
移动端阅览
陈苑锋. 视觉深度估计与点云建图研究进展[J/OL]. 液晶与显示, 2021,1-18.

Yuan-feng CHEN. Progress of visual depth estimation and point cloud mapping[J/OL]. Chinese journal of liquid crystals and displays, 2021, 1-18.
陈苑锋. 视觉深度估计与点云建图研究进展[J/OL]. 液晶与显示, 2021,1-18. DOI： 10.37188/CJLCD.2021-0047.

Yuan-feng CHEN. Progress of visual depth estimation and point cloud mapping[J/OL]. Chinese journal of liquid crystals and displays, 2021, 1-18. DOI： 10.37188/CJLCD.2021-0047.

摘要

即时定位导航（SLAM）是无人驾驶和机器人实现自主移动的关键技术，而目前广泛应用于SLAM技术中的激光雷达传感器存在成本高昂、激光点云空间分辨率低及难以获得精确的语义信息等一系列问题。相比之下视觉传感器（摄像头等）可以有效避免以上问题，但是在深度预测和建图等方面需要更复杂的算法。近年来，随着处理器算力的提升、数据集的不断丰富，以及新的机器视觉算法的提出，视觉深度预测和建图算法的精度和效率都已经有了较大提升。本文对现有视觉深度预测与视觉建图方法进行了总结，从视觉数据的采集和算法设计等方面进行分类阐述，最后针对视觉深度估计和建图方案的应用场景和未来发展方向进行了分析。

Abstract

Simultaneous location and mapping （SLAM） is a key technology for autonomous driving vehicles and robots to realize autonomous movement. The lidar currently widely used in SLAM technology present a series of issues， including high cost， low spatial resolution of laser point clouds， and difficulty in obtaining accurate semantic information. In contrast， cameras can effectively avoid the above problems， but more complex algorithms are required in depth prediction and mapping. In recent years， with the increase in computing power， the continuous enrichment of data sets， as well as the introduction of new machine vision algorithms， the accuracy and efficiency of vision depth prediction and mapping algorithms have been greatly improved. This article summarizes the existing methods of visual depth prediction and point cloud mapping and classifies the methodology based on visual data collection approach and algorithm design， then analyzes the application scenarios and prospect of visual depth estimation and point cloud mapping.

关键词

Keywords

references

Schneider T . Maplab： An Open Framework for Research in Visual-inertial Mapping and Localization ［J］. IEEE Robotics & Automation Letters ， 2018 . 3 （ 3 ）： 1418 - 1425 .

Konolige K ， Agrawal M . FrameSLAM： From Bundle Adjustment to Real-Time Visual Mapping ［J］. IEEE Transactions on Robotics ， 2008 . 24 （ 5 ）： 1066 - 1077 .

Xavier R S ， da SilvaB M ， GoncalvesL M . Accuracy analysis of augmented reality markers for visual mapping and localization ［C］// 2017 Workshop of Computer Vision （WVC） . 2017 ， IEEE .

Ortiz-Fernandez L E . Smart Artificial Markers for Accurate Visual Mapping and Localization ［J］. Sensors ， 2021 . 21 （ 2 ）： 625 .

Chai Y ， Cao X . Stereo Matching Algorithm Based on Joint Matching Cost and Adaptive Window ［C］// 2018 IEEE 3rd Advanced Information Technology ， Electronic and Automation Control Conference （IAEAC） . 2018 .

Guo X . Deep Eyes： Binocular Depth-from-Focus on Focal Stack Pairs . 2019 .

Song X ， Zhao X ， Lin T . Face Spoofing Detection by Fusing Binocular Depth and Spatial Pyramid Coding Micro-Texture Features ［C］// 2017 IEEE International Conference on Image Processing （ICIP） . 2017 .

Pilzer A . Progressive fusion for unsupervised binocular depth estimation using cycled networks ［J］. IEEE Transactions on Pattern Analysis And Machine Intelligence ， 2019 . 42 （ 10 ）： 2380 - 2395 .

［J］ Xu K， Chen Z， Jia F， Unsupervised binocular depth prediction network for laparoscopic surgery ［J］. Computer Assisted Surgery ， 2019 ： 1 - 6 .

Sinha A . DELTAS： Depth Estimation by Learning Triangulation and densification of Sparse points . 2020 .

Yao Y . Mvsnet： Depth inference for unstructured multi-view stereo ［C］// Proceedings of the European Conference on Computer Vision （ECCV） . 2018 .

Kusupati U . Normal Assisted Stereo Depth Estimation . 2019 .

Wang K ， Shen S . Mvdepthnet： Real-time multiview depth estimation neural network ［C］// 2018 International conference on 3d vision （3DV） . 2018 . IEEE .

Hou Y ， Solin A ， Kannala J . Unstructured Multi-View Depth Estimation Using Mask-Based Multiplane Representation ［C］// 2019： Springer ， Cham .

Long X . Multi-view Depth Estimation using Epipolar Spatio-Temporal Networks . 2020 .

Godard C . Digging Into Self-Supervised Monocular Depth Estimation ［C］// 2019 IEEE/CVF International Conference on Computer Vision （ICCV） . 2020 .

Aleotti F . Generative Adversarial Networks for Unsupervised Monocular Depth Prediction ［C］// Proceedings， Part I. Munich， Germany ， 2019 .

Atapour-Abarghouei A . Real-time monocular depth estimation using synthetic data with domain adaptation ［C］// IEEE/CVF Conference on Computer Vision & Pattern Recognition . 2018 .

Madhu B V . A Deeper Insight into the UnDEMoN： Unsupervised Deep Network for Depth and Ego-Motion Estimation ［C］// 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems （IROS） . 2018 .

Casser V . Depth Prediction without the Sensors： Leveraging Structure for Unsupervised Learning from Monocular Videos ［C］// Thirty-Third AAAI Conference on Artificial Intelligence （AAAI'

2019 .

Fu H . Deep ordinal regression network for monocular depth estimation ［C］// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2018 .

Guo X . Learning monocular depth by distilling cross-domain stereo networks ［C］// Proceedings of the European Conference on Computer Vision （ECCV） . 2018 .

Jiang H . Self-supervised relative depth learning for urban scene understanding ［C］// Proceedings of the European Conference on Computer Vision （ECCV） . 2018 .

Kuznietsov Y ， Stückler J ， Leibe B . Semi-Supervised Deep Learning for Monocular Depth Map Prediction ［C］// IEEE Conference on Computer Vision & Pattern Recognition . 2017 .

Mahjourian R ， Wicke M ， Angelova A . Unsupervised learning of depth and ego-motion from monocular video using 3d geometric constraints ［C］// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2018 .

Li R . Deep attention-based classification network for robust depth prediction ［C］// Asian Conference on Computer Vision . 2018 . Springer .

Ranjan A . Competitive collaboration： Joint unsupervised learning of depth， camera motion， optical flow and motion segmentation ［C］// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 2019 .

Poggi M . Towards Real-Time Unsupervised Monocular Depth Estimation on CPU ［C］// 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems （IROS） . 2018 .

Zhan H . Unsupervised learning of monocular depth estimation and visual odometry with deep feature reconstruction ［C］// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2018 .

Pillai S ， Ambrus R ， Gaidon A . SuperDepth ： Self-Supervised， Super-Resolved Monocular Depth Estimation ［C］// 2019 International Conference on Robotics and Automation （ICRA） . 2019 .

程明明，王 . 贺，安. 平，基于特征点匹配的多视图像校正［J］. 液晶与显示， 2010 . 25 （ 4 ）： 593 - 597 .

Cheng Mingming ， Wang . He， An. Ping， Multi-view image correction based on feature point matching ［J］. Chin ， J . Liq . Cryst. Displays ， 2010 . 25 （ 4 ）： 593 - 597 . （in Chinese）

张建业，朴燕，基于改进稳态匹配概率的立体匹配算法研究［J］. 液晶与显示， 2018 . 33 （ 4 ）： 357 - 364 .

Zhang Jianye ， Park Yan ， Research on stereo matching algorithm based on improved steady-state matching probability ［J］. Chin， J. Liq. Cryst. Displays ， 2018 . 33 （ 4 ）： 357 - 364 . （in Chinese）

Huimin Yin . A High Spatial Resolution Depth Sensing Method Based on Binocular Structured Light ［J］. Sensors ， 2017 . 17 （ 4 ）： 805 - 811 .

Ende W . Stereo matching algorithm based on the combination of matching costs ［C］// 2017 IEEE 7th Annual International Conference on CYBER Technology in Automation ， Control， and Intelligent Systems （CYBER） . 2017 .

汪伟，基于深度学习的双目立体视觉研究［D］. 合肥：合肥工业大学， 2020 ，

Wang Wei， Research on Binocular Stereo Vision Based on Deep Learning ［D］ Hefei ： Hefei University of Technology ， 2020 ，（in Chinese）

Anantrasirichai N . Dynamic Programming for Multi-View Disparity/Depth Estimation ［C］// 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings . 2006 .

Montserrat T . Depth estimation based on multiview matching with depth/color segmentation and memory efficient belief propagation ［C］// Image Processing （ICIP）， 2009 16th IEEE International Conference on . 2009 .

Liu Y . Continuous depth estimation for multi-view stereo ［C］// IEEE Conference on Computer Vision & Pattern Recognition . 2009 .

Lee S B ， Ho Y S . View-consistent multi-view depth estimation for three-dimensional video generation ［C］// 3dtv-conference： the True Vision-capture . 2010 .

Mieloch D . Graph-based multiview depth estimation using segmentation ［C］// IEEE International Conference on Multimedia & Expo . 2017 .

Facil J M . Single-View and Multi-View Depth Fusion . IEEE Robotics & Automation Letters ， 2017 . 2 （ 4 ）： 1994 - 2001 .

Yang M ， Cao ， X ， Dai Q . Multiview video depth estimation with spatial-temporal consistency ［C］// British Machine Vision Conference ， BMVC 2010 ， Aberystwyth ， UK ， 2010 .

Ince S . Depth Estimation for View Synthesis in Multiview Video Coding ［C］// 3dtv Conference . 2007 .

Long X . Occlusion-Aware Depth Estimation with Adaptive Normal Constraints . 2020 .

Strecha C . Gool L V. PDE-based Multi-view Depth Estimation ［C］// International Symposium on 3d Data Processing Visualization & Transmission . 2002 .

Garg R . Unsupervised CNN for Single View Depth Estimation： Geometry to the Rescue ［C］// European Conference on Computer Vision . 2016 .

Godard C ， Mac Aodha O ， Brostow G J . Unsupervised Monocular Depth Estimation with Left-Right Consistency ［C］// Computer Vision & Pattern Recognition . 2017 .

Zhou T . Unsupervised Learning of Depth and Ego-Motion from Video . in 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR） . 2017 .

Eigen D ， Fergus R . Predicting Depth， Surface Normals and Semantic Labels with a Common Multi-scale Convolutional Architecture ［C］// IEEE International Conference on Computer Vision . 2015 .

Chen W . Single-image depth perception in the wild ［J］. arXiv preprint arXiv： 1604.03901 ， 2016 .

Kendall A . End-to-end learning of geometry and context for deep stereo regression ［C］// Proceedings of the IEEE International Conference on Computer Vision . 2017 .

Li Z ， Snavely N . Megadepth： Learning single-view depth prediction from internet photos ［C］// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2018 .

Liu M ， Salzmann M ， He X . Discrete-Continuous Depth Estimation from a Single Image ［C］// IEEE Conference on Computer Vision & Pattern Recognition . 2014 .

Mur-Artal R ， Montiel J M M ， Tardos J D . ORB-SLAM： a versatile and accurate monocular SLAM system ［J］. IEEE Transactions on Robotics ， 2015 . 31 （ 5 ）： 1147 - 1163 .

Wu Y ， Ying S ， Zheng L ， Size-to-depth ： a new perspective for single image depth estimation ［J］. arXiv preprint arXiv： 1801.04461 ， 2018 .

Yang Z . Unsupervised learning of geometry with edge-aware depth-normal consistency ［J］. arXiv preprint arXiv： 1711.03665 ， 2017 .

Williams O ， Blake A ， Cipolla R . Sparse and Semi-supervised Visual Mapping with the S^3GP ［C］// 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition （CVPR'

2006 .

Qin T . AVP-SLAM： Semantic Visual Mapping and Localization for Autonomous Vehicles in the Parking Lot ［J］. arXiv preprint arXiv： 2007.01813 ， 2020 .

Saeedi S . Characterizing visual localization and mapping datasets ［C］// 2019 International Conference on Robotics and Automation （ICRA） . 2019 . IEEE .

Usenko V . Visual-Inertial Mapping With Non-Linear Factor Recovery ［J］. IEEE Robotics and Automation Letters ， 2020 . 5 （ 2 ）： 422 - 429 .

Xiao L . Dynamic-SLAM： Semantic monocular visual localization and mapping based on deep learning in dynamic environment ［J］. Robotics & Autonomous Systems ， 2019 .

Hong S . Kim J， Three-Dimensional Visual Mapping of Underwater Ship Hull Surface using View-based Piecewise-Planar Measurements ［J］. IFAC-PapersOnLine ， 2019 . 52 （ 21 ）： 384 - 389 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

视觉深度估计与点云建图研究进展

无人机绝对视觉定位的研究进展

基于自适应遮挡恢复与拓扑姿态双向感知的三维手部重建方法

基于Mamba-2的视频快照压缩成像重构方法

基于光流预测的三维光场显示透视关系校正