Progress of visual depth estimation and point cloud mapping

Yuan-feng CHEN

doi:10.37188/CJLCD.2021-0047

您当前的位置：

首页 >

文章列表页 >

Progress of visual depth estimation and point cloud mapping

更新时间：2021-04-09

- Progress of visual depth estimation and point cloud mapping
- Chinese Journal of Liquid Crystals and Displays Pages: 1-18(2021)
- 作者机构：
  
  美的集团（上海）有限公司，上海 201799
- 作者简介：
- 基金信息：
- DOI：10.37188/CJLCD.2021-0047
  CLC： TP394.1;TH691.9
- Received：24 February 2021，
  
  Revised：20 March 2021，
- 稿件说明：
移动端阅览
Yuan-feng CHEN. Progress of visual depth estimation and point cloud mapping[J/OL]. Chinese journal of liquid crystals and displays, 2021, 1-18.
DOI：

Yuan-feng CHEN. Progress of visual depth estimation and point cloud mapping[J/OL]. Chinese journal of liquid crystals and displays, 2021, 1-18. DOI： 10.37188/CJLCD.2021-0047.

摘要

即时定位导航（SLAM）是无人驾驶和机器人实现自主移动的关键技术，而目前广泛应用于SLAM技术中的激光雷达传感器存在成本高昂、激光点云空间分辨率低及难以获得精确的语义信息等一系列问题。相比之下视觉传感器（摄像头等）可以有效避免以上问题，但是在深度预测和建图等方面需要更复杂的算法。近年来，随着处理器算力的提升、数据集的不断丰富，以及新的机器视觉算法的提出，视觉深度预测和建图算法的精度和效率都已经有了较大提升。本文对现有视觉深度预测与视觉建图方法进行了总结，从视觉数据的采集和算法设计等方面进行分类阐述，最后针对视觉深度估计和建图方案的应用场景和未来发展方向进行了分析。

Abstract

Simultaneous location and mapping （SLAM） is a key technology for autonomous driving vehicles and robots to realize autonomous movement. The lidar currently widely used in SLAM technology present a series of issues， including high cost， low spatial resolution of laser point clouds， and difficulty in obtaining accurate semantic information. In contrast， cameras can effectively avoid the above problems， but more complex algorithms are required in depth prediction and mapping. In recent years， with the increase in computing power， the continuous enrichment of data sets， as well as the introduction of new machine vision algorithms， the accuracy and efficiency of vision depth prediction and mapping algorithms have been greatly improved. This article summarizes the existing methods of visual depth prediction and point cloud mapping and classifies the methodology based on visual data collection approach and algorithm design， then analyzes the application scenarios and prospect of visual depth estimation and point cloud mapping.

关键词

Keywords

references

Schneider T . Maplab： An Open Framework for Research in Visual-inertial Mapping and Localization ［J］. IEEE Robotics & Automation Letters ， 2018 . 3 （ 3 ）： 1418 - 1425 .

Konolige K ， Agrawal M . FrameSLAM： From Bundle Adjustment to Real-Time Visual Mapping ［J］. IEEE Transactions on Robotics ， 2008 . 24 （ 5 ）： 1066 - 1077 .

Xavier R S ， da SilvaB M ， GoncalvesL M . Accuracy analysis of augmented reality markers for visual mapping and localization ［C］// 2017 Workshop of Computer Vision （WVC） . 2017 ， IEEE .

Ortiz-Fernandez L E . Smart Artificial Markers for Accurate Visual Mapping and Localization ［J］. Sensors ， 2021 . 21 （ 2 ）： 625 .

Chai Y ， Cao X . Stereo Matching Algorithm Based on Joint Matching Cost and Adaptive Window ［C］// 2018 IEEE 3rd Advanced Information Technology ， Electronic and Automation Control Conference （IAEAC） . 2018 .

Guo X . Deep Eyes： Binocular Depth-from-Focus on Focal Stack Pairs . 2019 .

Song X ， Zhao X ， Lin T . Face Spoofing Detection by Fusing Binocular Depth and Spatial Pyramid Coding Micro-Texture Features ［C］// 2017 IEEE International Conference on Image Processing （ICIP） . 2017 .

Pilzer A . Progressive fusion for unsupervised binocular depth estimation using cycled networks ［J］. IEEE Transactions on Pattern Analysis And Machine Intelligence ， 2019 . 42 （ 10 ）： 2380 - 2395 .

［J］ Xu K， Chen Z， Jia F， Unsupervised binocular depth prediction network for laparoscopic surgery ［J］. Computer Assisted Surgery ， 2019 ： 1 - 6 .

Sinha A . DELTAS： Depth Estimation by Learning Triangulation and densification of Sparse points . 2020 .

Yao Y . Mvsnet： Depth inference for unstructured multi-view stereo ［C］// Proceedings of the European Conference on Computer Vision （ECCV） . 2018 .

Kusupati U . Normal Assisted Stereo Depth Estimation . 2019 .

Wang K ， Shen S . Mvdepthnet： Real-time multiview depth estimation neural network ［C］// 2018 International conference on 3d vision （3DV） . 2018 . IEEE .

Hou Y ， Solin A ， Kannala J . Unstructured Multi-View Depth Estimation Using Mask-Based Multiplane Representation ［C］// 2019： Springer ， Cham .

Long X . Multi-view Depth Estimation using Epipolar Spatio-Temporal Networks . 2020 .

Godard C . Digging Into Self-Supervised Monocular Depth Estimation ［C］// 2019 IEEE/CVF International Conference on Computer Vision （ICCV） . 2020 .

Aleotti F . Generative Adversarial Networks for Unsupervised Monocular Depth Prediction ［C］// Proceedings， Part I. Munich， Germany ， 2019 .

Atapour-Abarghouei A . Real-time monocular depth estimation using synthetic data with domain adaptation ［C］// IEEE/CVF Conference on Computer Vision & Pattern Recognition . 2018 .

Madhu B V . A Deeper Insight into the UnDEMoN： Unsupervised Deep Network for Depth and Ego-Motion Estimation ［C］// 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems （IROS） . 2018 .

Casser V . Depth Prediction without the Sensors： Leveraging Structure for Unsupervised Learning from Monocular Videos ［C］// Thirty-Third AAAI Conference on Artificial Intelligence （AAAI'

2019 .

Fu H . Deep ordinal regression network for monocular depth estimation ［C］// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2018 .

Guo X . Learning monocular depth by distilling cross-domain stereo networks ［C］// Proceedings of the European Conference on Computer Vision （ECCV） . 2018 .

Jiang H . Self-supervised relative depth learning for urban scene understanding ［C］// Proceedings of the European Conference on Computer Vision （ECCV） . 2018 .

Kuznietsov Y ， Stückler J ， Leibe B . Semi-Supervised Deep Learning for Monocular Depth Map Prediction ［C］// IEEE Conference on Computer Vision & Pattern Recognition . 2017 .

Mahjourian R ， Wicke M ， Angelova A . Unsupervised learning of depth and ego-motion from monocular video using 3d geometric constraints ［C］// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2018 .

Li R . Deep attention-based classification network for robust depth prediction ［C］// Asian Conference on Computer Vision . 2018 . Springer .

Ranjan A . Competitive collaboration： Joint unsupervised learning of depth， camera motion， optical flow and motion segmentation ［C］// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 2019 .

Poggi M . Towards Real-Time Unsupervised Monocular Depth Estimation on CPU ［C］// 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems （IROS） . 2018 .

Zhan H . Unsupervised learning of monocular depth estimation and visual odometry with deep feature reconstruction ［C］// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2018 .

Pillai S ， Ambrus R ， Gaidon A . SuperDepth ： Self-Supervised， Super-Resolved Monocular Depth Estimation ［C］// 2019 International Conference on Robotics and Automation （ICRA） . 2019 .

程明明，王 . 贺，安. 平，基于特征点匹配的多视图像校正［J］. 液晶与显示， 2010 . 25 （ 4 ）： 593 - 597 .

Cheng Mingming ， Wang . He， An. Ping， Multi-view image correction based on feature point matching ［J］. Chin ， J . Liq . Cryst. Displays ， 2010 . 25 （ 4 ）： 593 - 597 . （in Chinese）

张建业，朴燕，基于改进稳态匹配概率的立体匹配算法研究［J］. 液晶与显示， 2018 . 33 （ 4 ）： 357 - 364 .

Zhang Jianye ， Park Yan ， Research on stereo matching algorithm based on improved steady-state matching probability ［J］. Chin， J. Liq. Cryst. Displays ， 2018 . 33 （ 4 ）： 357 - 364 . （in Chinese）

Huimin Yin . A High Spatial Resolution Depth Sensing Method Based on Binocular Structured Light ［J］. Sensors ， 2017 . 17 （ 4 ）： 805 - 811 .

Ende W . Stereo matching algorithm based on the combination of matching costs ［C］// 2017 IEEE 7th Annual International Conference on CYBER Technology in Automation ， Control， and Intelligent Systems （CYBER） . 2017 .

汪伟，基于深度学习的双目立体视觉研究［D］. 合肥：合肥工业大学， 2020 ，

Wang Wei， Research on Binocular Stereo Vision Based on Deep Learning ［D］ Hefei ： Hefei University of Technology ， 2020 ，（in Chinese）

Anantrasirichai N . Dynamic Programming for Multi-View Disparity/Depth Estimation ［C］// 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings . 2006 .

Montserrat T . Depth estimation based on multiview matching with depth/color segmentation and memory efficient belief propagation ［C］// Image Processing （ICIP）， 2009 16th IEEE International Conference on . 2009 .

Liu Y . Continuous depth estimation for multi-view stereo ［C］// IEEE Conference on Computer Vision & Pattern Recognition . 2009 .

Lee S B ， Ho Y S . View-consistent multi-view depth estimation for three-dimensional video generation ［C］// 3dtv-conference： the True Vision-capture . 2010 .

Mieloch D . Graph-based multiview depth estimation using segmentation ［C］// IEEE International Conference on Multimedia & Expo . 2017 .

Facil J M . Single-View and Multi-View Depth Fusion . IEEE Robotics & Automation Letters ， 2017 . 2 （ 4 ）： 1994 - 2001 .

Yang M ， Cao ， X ， Dai Q . Multiview video depth estimation with spatial-temporal consistency ［C］// British Machine Vision Conference ， BMVC 2010 ， Aberystwyth ， UK ， 2010 .

Ince S . Depth Estimation for View Synthesis in Multiview Video Coding ［C］// 3dtv Conference . 2007 .

Long X . Occlusion-Aware Depth Estimation with Adaptive Normal Constraints . 2020 .

Strecha C . Gool L V. PDE-based Multi-view Depth Estimation ［C］// International Symposium on 3d Data Processing Visualization & Transmission . 2002 .

Garg R . Unsupervised CNN for Single View Depth Estimation： Geometry to the Rescue ［C］// European Conference on Computer Vision . 2016 .

Godard C ， Mac Aodha O ， Brostow G J . Unsupervised Monocular Depth Estimation with Left-Right Consistency ［C］// Computer Vision & Pattern Recognition . 2017 .

Zhou T . Unsupervised Learning of Depth and Ego-Motion from Video . in 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR） . 2017 .

Eigen D ， Fergus R . Predicting Depth， Surface Normals and Semantic Labels with a Common Multi-scale Convolutional Architecture ［C］// IEEE International Conference on Computer Vision . 2015 .

Chen W . Single-image depth perception in the wild ［J］. arXiv preprint arXiv： 1604.03901 ， 2016 .

Kendall A . End-to-end learning of geometry and context for deep stereo regression ［C］// Proceedings of the IEEE International Conference on Computer Vision . 2017 .

Li Z ， Snavely N . Megadepth： Learning single-view depth prediction from internet photos ［C］// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2018 .

Liu M ， Salzmann M ， He X . Discrete-Continuous Depth Estimation from a Single Image ［C］// IEEE Conference on Computer Vision & Pattern Recognition . 2014 .

Mur-Artal R ， Montiel J M M ， Tardos J D . ORB-SLAM： a versatile and accurate monocular SLAM system ［J］. IEEE Transactions on Robotics ， 2015 . 31 （ 5 ）： 1147 - 1163 .

Wu Y ， Ying S ， Zheng L ， Size-to-depth ： a new perspective for single image depth estimation ［J］. arXiv preprint arXiv： 1801.04461 ， 2018 .

Yang Z . Unsupervised learning of geometry with edge-aware depth-normal consistency ［J］. arXiv preprint arXiv： 1711.03665 ， 2017 .

Williams O ， Blake A ， Cipolla R . Sparse and Semi-supervised Visual Mapping with the S^3GP ［C］// 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition （CVPR'

2006 .

Qin T . AVP-SLAM： Semantic Visual Mapping and Localization for Autonomous Vehicles in the Parking Lot ［J］. arXiv preprint arXiv： 2007.01813 ， 2020 .

Saeedi S . Characterizing visual localization and mapping datasets ［C］// 2019 International Conference on Robotics and Automation （ICRA） . 2019 . IEEE .

Usenko V . Visual-Inertial Mapping With Non-Linear Factor Recovery ［J］. IEEE Robotics and Automation Letters ， 2020 . 5 （ 2 ）： 422 - 429 .

Xiao L . Dynamic-SLAM： Semantic monocular visual localization and mapping based on deep learning in dynamic environment ［J］. Robotics & Autonomous Systems ， 2019 .

Hong S . Kim J， Three-Dimensional Visual Mapping of Underwater Ship Hull Surface using View-based Piecewise-Planar Measurements ［J］. IFAC-PapersOnLine ， 2019 . 52 （ 21 ）： 384 - 389 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Progress of visual depth estimation and point cloud mapping

Research progress on absolute visual localization of unmanned aerial vehicles

3D hand reconstruction method based on adaptive occlusion recovery and topology-pose bidirectional perception

Reconstruction method of video snapshot compressive imaging based on Mamba-2

3D light-field display with corrected perspective based on optical flow prediction

Related Author

Yuan-feng CHEN

MIAO Zongcheng

ZHOU Runxi

LIU Jie

YANG Haiyan

LIU Jia

HUANG Nanxuan

CHEN Dapeng

Related Institution

Midea Group(Shanghai) Co., Ltd.

Technological Institute of Materials & Energy Science （TIMES）， School of Electronic Information， Xijing University

School of Artificial Intelligence， Optics and Electronics （iOPEN）， Northwestern Polytechnical University

School of Automation， Nanjing University of Information Science & Technology

School of Computer and Computing Science， Hangzhou City University

AI问答

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China 130033 Postal code：130033
Tel：0431-86176059 Email：yjxs@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰