视觉深度估计与点云建图研究进展

陈苑锋

doi:10.37188/CJLCD.2020-0047

您当前的位置：

首页 >

文章列表页 >

视觉深度估计与点云建图研究进展

图像处理 | 更新时间：2021-05-28

- 视觉深度估计与点云建图研究进展
- Progress of visual depth estimation and point cloud mapping
- 液晶与显示 2021年36卷第6期页码：896-911
- 作者机构：
  
  美的集团(上海)有限公司, 上海 201799
- 作者简介：
  
  [ "陈苑锋(1979-), 男, 福建安溪人, 博士, 工程师, 2006年于复旦大学获得博士学位, 主要从事机器人方面的研究。E-mail:yuanfeng_chen@hotmail.com" ]
- 基金信息：
- DOI：10.37188/CJLCD.2020-0047
  中图分类号： TP394.1;TH691.9
- 收稿日期：2020-02-27，
  
  修回日期：2021-03-29，
  
  纸质出版日期：2021-06
- 稿件说明：
移动端阅览
陈苑锋. 视觉深度估计与点云建图研究进展[J]. 液晶与显示, 2021,36(6):896-911.

Yuan-feng CHEN. Progress of visual depth estimation and point cloud mapping[J]. Chinese journal of liquid crystals and displays, 2021, 36(6): 896-911.
陈苑锋. 视觉深度估计与点云建图研究进展[J]. 液晶与显示, 2021,36(6):896-911. DOI： 10.37188/CJLCD.2020-0047.

Yuan-feng CHEN. Progress of visual depth estimation and point cloud mapping[J]. Chinese journal of liquid crystals and displays, 2021, 36(6): 896-911. DOI： 10.37188/CJLCD.2020-0047.

摘要

即时定位导航（SLAM）是无人驾驶和机器人实现自主移动的关键技术。目前广泛应用于SLAM技术中的激光雷达传感器存在成本高昂、激光点云空间分辨率低及难以获得精确的语义信息等一系列问题。视觉传感器可以有效避免上述问题，但是在深度预测和建图等方面需要更复杂的算法。近年来，随着处理器算力的提升、数据集的丰富和新机器视觉算法的出现，视觉深度预测和建图算法的精度和效率都有了明显提升。本文对现有视觉深度预测与视觉建图方法进行了总结，从视觉数据的采集和算法设计等方面进行分类阐述，最后针对应用场景和未来发展方向进行了分析。

Abstract

Simultaneous location and mapping (SLAM) is a key technology for autonomous driving vehicles and robots to realize autonomous movement. The lidar currently used in SLAM technology presents a series of issues

including high cost

low spatial resolution of laser point clouds

and difficulty in obtaining accurate semantic information. In contrast

cameras can effectively avoid the above problems

but more complex algorithms are required in depth prediction and mapping. In recent years

with the progress in computing hardware

the continuous expansion of data sets

as well as the introduction of new computer vision algorithms

the accuracy and efficiency of visual depth prediction and mapping algorithms have been significantly improved. This article summarizes the existing methods of visual depth prediction and point cloud mapping and classifies the methodology based on data collection approach and algorithm design

then analyzes the application scenarios and prospect of visual depth estimation and point cloud mapping.

关键词

Keywords

references

T SCHNEIDER , M DYMCZYK , M FEHR , 等 . Maplab: an open framework for research in visual-inertial mapping and localization . IEEE Robotics and Automation Letters , 2018 . 3 ( 3 ): 1418 - 1425 . DOI: 10.1109/LRA.2018.2800113 http://doi.org/10.1109/LRA.2018.2800113 .

K KONOLIGE , M AGRAWAL . FrameSLAM: from bundle adjustment to real-time visual mapping . IEEE Transactions on Robotics , 2008 . 24 ( 5 ): 1066 - 1077 . DOI: 10.1109/TRO.2008.2004832 http://doi.org/10.1109/TRO.2008.2004832 .

XAVIER R S, DA SILVA B M F, GONCALVES L M G. Accuracy analysis of augmented reality markers for visual mapping and localization[C]// 2017 Workshop of Computer Vision ( WVC ). Natal, Brazil: IEEE, 2017: 73-77.

L E ORTIZ-FERNANDEZ , E V CABRERA-AVILA , B M F DA SILVA , 等 . Smart artificial markers for accurate visual mapping and localization . Sensors , 2021 . 21 ( 2 ): 625 DOI: 10.3390/s21020625 http://doi.org/10.3390/s21020625 .

CHAI Y, CAO X J. Stereo matching algorithm based on joint matching cost and adaptive window[C]//2018 IEEE 3 rd Advanced Information Technology, Electronic and Automation Control Conference ( IAEAC ). Chongqing, China: IEEE, 2018: 442-446.

GUO X Q, CHEN Z, LI S Y, et al . Deep eyes: binocular depth-from-focus on focal stack pairs[J]. arXiv , 1711.10729, 2019.

SONG X, ZHAO X, LIN T W. Face spoofing detection by fusing binocular depth and spatial pyramid coding micro-texture features[C]//2017 IEEE International Conference on Image Processing ( ICIP ). Beijing, China: IEEE, 2017: 96-100.

A PILZER , S LATHUILIÈRE , D XU , 等 . Progressive fusion for unsupervised binocular depth estimation using cycled networks . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2020 . 42 ( 10 ): 2380 - 2395 . DOI: 10.1109/TPAMI.2019.2942928 http://doi.org/10.1109/TPAMI.2019.2942928 .

K XU , Z Y CHEN , F C JIA . Unsupervised binocular depth prediction network for laparoscopic surgery . Computer Assisted Surgery , 2019 . 24 ( S1 ): 30 - 35 .

SINHA A, MUREZ Z, BARTOLOZZI J, et al . DELTAS: depth estimation by learning triangulation and densification of sparse points[C]// Proceedings of the 16 th European Conference on Computer Vision . Glasgow, UK: Springer, 2020: 104-121.

YAO Y, LUO Z X, LI S W, et al . MVSNet: depth inference for unstructured multi-view stereo[C]// Proceedings of the 15 th European Conference on Computer Vision ( ECCV ). Munich, Germany: Springer, 2018: 785-801.

KUSUPATI U, CHENG S, CHEN R, et al . Normal assisted stereo depth estimation[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition ( CVPR ). Seattle, WA, USA: IEEE, 2020: 2186-2196.

WANG K X, SHEN S J. MVDepthNet: real-time multiview depth estimation neural network[C]//2018 International Conference on 3 D Vision (3 DV ). Verona, Italy: IEEE, 2018: 248-257.

HOU Y X, SOLIN A, KANNALA J. Unstructured multi-view depth estimation using mask-based multiplane representation[C]// Proceedings of the 21 st Scandinavian Conference on Image Analysis . Norrköping, Sweden: Springer, 2019: 54-66.

LONG X X, LIU L J, LI W, et al . Multi-view depth estimation using epipolar spatio-temporal networks[J]. arXiv , 2011.13118, 2020.

GODARD C, AODHA O M, FIRMAN M, et al . Digging into self-supervised monocular depth estimation[C]//2019 IEEE/CVF International Conference on Computer Vision ( ICCV ). Seoul, Korea (South): IEEE, 2019: 3827-3837.

ALEOTTI F, TOSI F, POGGI M, et al . Generative adversarial networks for unsupervised monocular depth prediction[C]// Computer Vision-ECCV 2018 Workshops . Munich, Germany: Springer, 2018: 337-354.

ATAPOUR-ABARGHOUEI A, BRECKON T P. Real-time monocular depth estimation using synthetic data with domain adaptation via image style transfer[C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City, UT, USA: IEEE, 2018: 2800-2810.

BABU V M, DAS K, MAJUMDAR A, et al . UnDEMoN: unsupervised deep network for depth and ego-motion estimation[C]//2018 IEEE/RSJ International Conference on Intelligent Robots and Systems ( IROS ). Madrid, Spain: IEEE, 2018: 1082-1088.

CASSER V, PIRK S, MAHJOURIAN R, et al . Depth prediction without the sensors: leveraging structure for unsupervised learning from monocular videos[C]// Thirty-Third AAAI Conference on Artificial Intelligence ( AAAI '19). Hawaii, USA: AAAI, 2019.

FU H, GONG M M, WANG C H, et al . Deep ordinal regression network for monocular depth estimation[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City, UT, USA: IEEE, 2018: 2002-2011.

GUO X Y, LI H S, YI S, et al . Learning monocular depth by distilling cross-domain stereo networks[C]// Proceedings of the 15 th European Conference on Computer Vision ( ECCV ). Munich, Germany: Springer, 2018: 506-523.

JIANG H Z, LARSSON G, MAIRE M, et al . Self-supervised relative depth learning for urban scene understanding[C]// Proceedings of the 15 th European Conference on Computer Vision ( ECCV ). Munich, Germany: Springer, 2018: 20-37.

KUZNIETSOV Y, STVCKLER J LEIBE B. Semi-supervised deep learning for monocular depth map prediction[C]// IEEE Conference on Computer Vision and Pattern Recognition . Honolulu, HI, USA: IEEE, 2017: 2215-2223.

MAHJOURIAN R, WICKE M, ANGELOVA A. Unsupervised learning of depth and ego-motion from monocular video using 3D geometric constraints[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City, UT, USA: IEEE, 2018: 5667-5675.

LI R B, XIAN K, SHEN C H, et al . Deep attention-based classification network for robust depth prediction[C]// Proceedings of the 14 th Asian Conference on Computer Vision . Perth, Australia: Springer, 2019: 663-678.

RANJAN A, JAMPANI V, BALLES L, et al . Competitive collaboration: Joint unsupervised learning of depth, camera motion, optical flow and motion segmentation[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Long Beach, CA, USA: IEEE, 2019: 12232-12241.

POGGI M, ALEOTTI F, TOSI F, et al . Towards real-time unsupervised monocular depth estimation on CPU[C]//2018 IEEE/RSJ International Conference on Intelligent Robots and Systems ( IROS ). Madrid, Spain: IEEE, 2018: 5848-5854.

ZHAN H Y, GARG R, WEERASEKERA C S, et al . Unsupervised learning of monocular depth estimation and visual odometry with deep feature reconstruction[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City, UT, USA: IEEE, 2018: 340-349.

PILLAI S, AMBRUŞ R, GAIDON A. SuperDepth: self-supervised, super-resolved monocular depth estimation[C]//2019 International Conference on Robotics and Automation ( ICRA ). Montreal, QC, Canada: IEEE, 2019: 9250-9256.

程明明 , 王贺 , 安平 , 等 . 基于特征点匹配的多视图像校正 . 液晶与显示 , 2010 . 25 ( 4 ): 593 - 597 . DOI: 10.3969/j.issn.1007-2780.2010.04.031 http://doi.org/10.3969/j.issn.1007-2780.2010.04.031 .

M M CHENG , H WANG , P AN , 等 . Multi-view image correction based on feature point matching . Chinese Journal of Liquid Crystals and Displays , 2010 . 25 ( 4 ): 593 - 597 . DOI: 10.3969/j.issn.1007-2780.2010.04.031 http://doi.org/10.3969/j.issn.1007-2780.2010.04.031 .

张建业 , 朴燕 . 基于改进稳态匹配概率的立体匹配算法研究 . 液晶与显示 , 2018 . 33 ( 4 ): 357 - 364 . http://cjlcd.lightpublishing.cn/thesisDetails#10.3788/YJYXS20183304.0357 http://cjlcd.lightpublishing.cn/thesisDetails#10.3788/YJYXS20183304.0357 .

J Y ZHANG , Y PIAO . Research on stereo matching algorithm based on improved steady-state matching probability . Chinese Journal of Liquid Crystals and Displays , 2018 . 33 ( 4 ): 357 - 364 . http://cjlcd.lightpublishing.cn/thesisDetails#10.3788/YJYXS20183304.0357 http://cjlcd.lightpublishing.cn/thesisDetails#10.3788/YJYXS20183304.0357 .

H M YAO , C Y GE , J R XUE , 等 . A high spatial resolution depth sensing method based on binocular structured light . Sensors , 2017 . 17 ( 4 ): 805 DOI: 10.3390/s17040805 http://doi.org/10.3390/s17040805 .

WANG E D, ZHU Y L, PENG L Y, et al . Stereo matching algorithm based on the combination of matching costs[C]//2017 IEEE 7 th Annual International Conference on CYBER Technology in Automation , Control , and Intelligent Systems ( CYBER ). Honolulu, HI, USA: IEEE, 2017: 1001-1004.

汪伟. 基于深度学习的双目立体视觉研究[D]. 合肥: 合肥工业大学, 2020.

WANG W. Research on binocular stereo vision based on deep learning[D]. Hefei: Hefei University of Technology, 2020. (in Chinese)

ANANTRASIRICHAI N, CANAGARAJAH C N, REDMILL D W, et al . Dynamic programming for multi-view disparity/depth estimation[C]//2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings . Toulouse, France: IEEE, 2006.

MONTSERRAT T, CIVIT J, ESCODA O D, et al . Depth estimation based on multiview matching with depth/color segmentation and memory efficient belief propagation[C]//2009 16 th IEEE International Conference on Image Processing ( ICIP ). Cairo, Egypt: IEEE, 2009: 2353-2356.

LIU Y B, CAO X, DAI Q H, et al . Continuous depth estimation for multi-view stereo[C]// IEEE Conference on Computer Vision and Pattern Recognition . Miami, FL, USA: IEEE, 2009: 2121-2128.

LEE S B, HO Y S. View-consistent multi-view depth estimation for three-dimensional video generation[C]//3 DTV-Conference : The True Vision-Capture , Transmission and Display of 3 D Video . Tampere, Finland: IEEE, 2010: 1-4.

MIELOCH D, DZIEMBOWSKI A, GRZELKA A, et al . Graph-based multiview depth estimation using segmentation[C]// IEEE International Conference on Multimedia and Expo . Hong Kong, China: IEEE, 2017: 217-222.

J M FÁCIL , A CONCHA , L MONTESANO , 等 . Single-view and multi-view depth fusion . IEEE Robotics and Automation Letters , 2017 . 2 ( 4 ): 1994 - 2001 . DOI: 10.1109/LRA.2017.2715400 http://doi.org/10.1109/LRA.2017.2715400 .

YANG M, CAO X, DAI Q. Multiview video depth estimation with spatial-temporal consistency[C]// British Machine Vision Conference . Aberystwyth, UK: BMVA Press, 2010: 1-11.

INCE S, MARTINIAN E, YEA S, et al . Depth estimation for view synthesis in multiview video coding[C]//3 DTV Conference . Kos, Greece: IEEE, 2007: 1-4.

LONG X X, LIU L J, THEOBALT C, et al . Occlusion-aware depth estimation with adaptive normal constraints[C]//16 th European Conference on Computer Vision . Glasgow, UK: Springer, 2020: 640-657.

STRECHA C, VAN GOOL L. PDE-based multi-view depth estimation[C]// First International Symposium on 3 D Data Processing Visualization and Transmission . Padua, Italy: IEEE, 2002: 416-425.

GARG R, VIJAY KUMAR B G, CARNEIRO G, et al . Unsupervised CNN for single view depth estimation: geometry to the rescue[C]//14 th European Conference on Computer Vision . Amsterdam, Netherlands: Springer, 2016: 740-756.

GODARD C, AODHA O M, BROSTOW G J. Unsupervised monocular depth estimation with left-right consistency[C] // IEEE Conference on Computer Vision and Pattern Recognition . Honolulu, HI, USA: IEEE, 2017: 6602-6611.

ZHOU T H, BROWN M, SNAVELY N, et al . Unsupervised learning of depth and ego-motion from video[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition ( CVPR ). Honolulu, HI, USA: IEEE, 2017: 6612-6619.

EIGEN D, FERGUS R. Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture[C]// IEEE International Conference on Computer Vision . Santiago, Chile: IEEE, 2015: 2650-2658.

CHEN W F, FU Z, YANG D W, et al . Single-image depth perception in the wild[C]// Proceedings of the 30 th International Conference on Neural Information Processing Systems . Barcelona: Curran Associates Inc., 2016: 730-738.

KENDALL A, MARTIROSYAN H, DASGUPTA S, et al . End-to-end learning of geometry and context for deep stereo regression[C]// Proceedings of the IEEE International Conference on Computer Vision . Venice, Italy: IEEE, 2017: 66-75.

LI Z Q, SNAVELY N. MegaDepth: learning single-view depth prediction from internet photos[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City, UT, USA: IEEE, 2018: 2041-2050.

LIU M M, SALZMANN M, HE X M. Discrete-continuous depth estimation from a single image[C]// IEEE Conference on Computer Vision and Pattern Recognition . Columbus, OH, USA: IEEE, 2014: 716-723.

R MUR-ARTAL , J M M MONTIEL , J D TARDOS . ORB-SLAM: a versatile and accurate monocular SLAM system . IEEE Transactions on Robotics , 2015 . 31 ( 5 ): 1147 - 1163 . DOI: 10.1109/TRO.2015.2463671 http://doi.org/10.1109/TRO.2015.2463671 .

WU Y R, YING S H, ZHENG L M. Size-to-depth: a new perspective for single image depth estimation[J]. arXiv , 1801.04461, 2018.

YANG Z H, WANG P, XU W, et al . Unsupervised learning of geometry with edge-aware depth-normal consistency[J]. arXiv , 1711.03665, 2017.

WILLIAMS O, BLAKE A, CIPOLLA R. Sparse and semi-supervised visual mapping with the S 3 GP[C]//2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition ( CVPR '06). New York, NY, USA: IEEE, 2006: 230-237.

QIN T, CHEN T Q, CHEN Y L, et al . AVP-SLAM: semantic visual mapping and localization for autonomous vehicles in the parking lot[C]//2020 IEEE/RSJ International Conference on Intelligent Robots and Systems ( IROS ). Las Vegas, NV, USA: IEEE, 2020: 5939-5945.

SAEEDI S, CARVALHO E D C, LI W B, et al . Characterizing visual localization and mapping datasets[C]//2019 International Conference on Robotics and Automation ( ICRA ). Montreal, QC, Canada: IEEE, 2019: 6699-6705.

V USENKO , N DEMMEL , D SCHUBERT , 等 . Visual-inertial mapping with non-linear factor recovery . IEEE Robotics and Automation Letters , 2020 . 5 ( 2 ): 422 - 429 . DOI: 10.1109/LRA.2019.2961227 http://doi.org/10.1109/LRA.2019.2961227 .

L H XIAO , J G WANG , X S QIU , 等 . Dynamic-SLAM: semantic monocular visual localization and mapping based on deep learning in dynamic environment . Robotics and Autonomous Systems , 2019 . 117 1 - 16 . DOI: 10.1016/j.robot.2019.03.012 http://doi.org/10.1016/j.robot.2019.03.012 .

S HONG , J KIM . Three-dimensional visual mapping of underwater ship hull surface using view-based piecewise-planar measurements . IFAC-PapersOnLine , 2019 . 52 ( 21 ): 384 - 389 . DOI: 10.1016/j.ifacol.2019.12.337 http://doi.org/10.1016/j.ifacol.2019.12.337 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

视觉深度估计与点云建图研究进展

无人机绝对视觉定位的研究进展

基于自适应遮挡恢复与拓扑姿态双向感知的三维手部重建方法

基于Mamba-2的视频快照压缩成像重构方法

基于光流预测的三维光场显示透视关系校正