基于实例分割的目标三维位置估计方法

刘长吉; 郝志成; 杨锦程; 朱明; 聂海涛

doi:10.37188/CJLCD.2021-0069

您当前的位置：

首页 >

文章列表页 >

基于实例分割的目标三维位置估计方法

目标检测与跟踪 | 更新时间：2021-11-08

- 基于实例分割的目标三维位置估计方法
- Object 3D position estimation based on instance segmentation
- 液晶与显示 2021年36卷第11期页码：1535-1544
- 作者机构：
  
  1.中国科学院长春光学精密机械与物理研究所, 吉林长春 130033
  2.中国科学院大学, 北京 100049
  3.重庆嘉陵华光光电科技有限公司, 重庆 400000
- 作者简介：
  
  [ "刘长吉(1990-), 男, 辽宁营口人, 硕士研究生, 2017年于辽宁工业大学获得学士学位, 主要从事三维物体检测方面的研究。E-mail:liuchangji18@mails.ucas.ac.cn" ]
  [ "郝志成(1978-)男, 辽宁营口人, 博士, 研究员, 2007年于中国科学院长春光学精密机械与物理研究所获得博士学位, 主要从事数字图像处理、自动目标识别、图像跟踪与测量方面的研究。E-mail: hzc972513@sina.com" ]
- 基金信息：
  
  广东省重点领域研发计划项目(2019B010155003);吉林省教育厅“十三五”科学技术项目(JJKH20200780KJ)
- DOI：10.37188/CJLCD.2021-0069
  中图分类号： TP391.4;TH691.9
- 收稿日期：2021-03-12，
  
  修回日期：2021-03-23，
  
  纸质出版日期：2021-11
- 稿件说明：
移动端阅览
刘长吉, 郝志成, 杨锦程, 等. 基于实例分割的目标三维位置估计方法[J]. 液晶与显示, 2021,36(11):1535-1544.

Chang-ji LIU, Zhi-cheng HAO, Jin-cheng YANG, et al. Object 3D position estimation based on instance segmentation[J]. Chinese journal of liquid crystals and displays, 2021, 36(11): 1535-1544.
刘长吉, 郝志成, 杨锦程, 等. 基于实例分割的目标三维位置估计方法[J]. 液晶与显示, 2021,36(11):1535-1544. DOI： 10.37188/CJLCD.2021-0069.

Chang-ji LIU, Zhi-cheng HAO, Jin-cheng YANG, et al. Object 3D position estimation based on instance segmentation[J]. Chinese journal of liquid crystals and displays, 2021, 36(11): 1535-1544. DOI： 10.37188/CJLCD.2021-0069.

摘要

三维目标检测在实际工程应用中的难点在于深度传感器价格高、点云质量差、缺少丰富的纹理信息、三维数据训练集制作困难。对此本文提出一种基于实例分割的三维目标位置估计方法，可以用在多种传感器中，如相机-雷达，双目相机等。首先在二维图像下对目标进行实例分割，根据目标的分割掩码提取出目标的深度图像与RGB图像融合转化为粗略点云，最终进行异常噪声点去除，得到精细的目标点云。在KITTI数据集上进行了测试，平均精度值（AP）可以达到50%，表明该方法可以准确地估计到目标位置信息。本文提出的方法无需三维数据训练集，可以快速准确地进行三维物体点云的提取，仅使用二维检测器就可以达到三维物体检测的目的。

Abstract

The difficulty of 3D target detection in practical engineering applications lies in the high price of depth perception equipment

poor point cloud quality

lack of rich texture information

difficulty in creating 3D data training sets. This paper proposes a three-dimensional target position estimation method based on instance segmentation. It can be used in a variety of sensors

such as camera-radar

binocular camera

etc

. Firstly

the target segmentation is performed under the two-dimensional image

the target's depth image are extracted and RGB image according to the target segmentation mask

and it is converted into a rough point cloud. Finally

the abnormal noise points is removed to obtain a fine target point cloud. Tested on the KITTI data set

the AP can reach 50%. The results show that this method can accurately estimate the target location information. The method proposed in this paper does not need 3D data training set can quickly and accurately extract the point cloud of three-dimensional objects

and only use a two-dimensional detector to achieve the purpose of three-dimensional object detection.

关键词

Keywords

references

STEIN S C, SCHOELER M, PAPON J, et al . Object partitioning using local convexity[C]// Proceedings of the 2014 IEEE Computer Vision and Pattern Recognition . Columbus, OH, USA: IEEE, 2014.

T RABBANI , F VAN DEN HEUVEL , G VOSSELMAN . Segmentation of point clouds using smoothness constraint . International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences , 2006 . 36 ( 5 ): 248 - 253 . http://www.researchgate.net/profile/George_Vosselman/publication/228340970_Segmentation_of_point_clouds_using_smoothness_constraint/links/00463524c7063d243a000000.pdf http://www.researchgate.net/profile/George_Vosselman/publication/228340970_Segmentation_of_point_clouds_using_smoothness_constraint/links/00463524c7063d243a000000.pdf .

M A FISCHLER , R C BOLLES . Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography . Communications of the ACM , 1981 . 24 ( 6 ): 381 - 395 . DOI: 10.1145/358669.358692 http://doi.org/10.1145/358669.358692 .

QI CHARLES R, SU H, KAICHUN M O, et al . PointNet: deep learning on point sets for 3D classification and segmentation[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition ( CVPR ). Honolulu, HI, USA: IEEE, 2017.

QI C R, YI L, SU H, et al . PointNet++: deep hierarchical feature learning on point sets in a metric space[J]. arXiv : 1706.02413, 2017.

QI C R, LIU W, WU C X, et al . Frustum PointNets for 3D object detection from RGB-D data[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City, UT, USA: IEEE, 2018: 918-927.

任彬 , 王宇庆 , 聂海涛 , 等 . 基于MPSOC的航空图像目标检测系统设计 . 液晶与显示 , 2020 . 1 - 12 . DOI: 10.37188/CJLCD.2020-0310 http://doi.org/10.37188/CJLCD.2020-0310 .

B REN , Y Q WANG , H T NIE , 等 . Design of aerial image target detection systembased on MPSOC . Chinese Journal of Liquid Crystals and Displays , 2020 . 1 - 12 . DOI: 10.37188/CJLCD.2020-0310 http://doi.org/10.37188/CJLCD.2020-0310 .

李娜 , 姜志 , 王军 , 等 . 基于Faster R-CNN的仪表识别方法 . 液晶与显示 , 2020 . 35 ( 12 ): 1291 - 1298 . DOI: 10.37188/YJYXS20203512.1291 http://doi.org/10.37188/YJYXS20203512.1291 .

N LI , Z JIANG , J WANG , 等 . Instrument recognition method based on Faster R-CNN . Chinese Journal of Liquid Crystals and Displays , 2020 . 35 ( 12 ): 1291 - 1298 . DOI: 10.37188/YJYXS20203512.1291 http://doi.org/10.37188/YJYXS20203512.1291 .

K M HE , G GKIOXARI , P DOLLÁR , 等 . Mask R-CNN . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2020 . 42 ( 2 ): 386 - 397 . DOI: 10.1109/TPAMI.2018.2844175 http://doi.org/10.1109/TPAMI.2018.2844175 .

S Q REN , K M HE , R GIRSHICK , 等 . Faster R-CNN: Towards real-time object detection with region proposal networks . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2017 . 39 ( 6 ): 1137 - 1149 . DOI: 10.1109/TPAMI.2016.2577031 http://doi.org/10.1109/TPAMI.2016.2577031 .

E SHELHAMER , J LONG , T DARRELL . Fully convolutional networks for semantic segmentation . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2015 . 39 ( 4 ): 640 - 651 . http://ieeexplore.ieee.org/document/7298965/references http://ieeexplore.ieee.org/document/7298965/references .

ZHOU Q Y, PARK J, KOLTUN V. Open3D: A modern library for 3D data processing[J]. arXiv : 1801.09847, 2018.

KIRILLOV A, WU Y X, HE K M, et al . PointRend: image segmentation as rendering[C]// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition ( CVPR ). Seattle, WA, USA: IEEE, 2020.

马鑫 , 魏仲慧 , 何昕 , 等 . 三维枪弹痕点云数据处理及特征提取研究 . 液晶与显示 , 2016 . 31 ( 9 ): 889 - 896 . DOI: 10.3788/YJYXS20163109.0889 http://doi.org/10.3788/YJYXS20163109.0889 .

X MA , Z H WEI , X HE , 等 . Processing and feature extraction for three-dimensional bullet point cloud data . Chinese Journal of Liquid Crystals and Displays , 2016 . 31 ( 9 ): 889 - 896 . DOI: 10.3788/YJYXS20163109.0889 http://doi.org/10.3788/YJYXS20163109.0889 .

CHHIKARA R S, HORTON C L, MENDLOWITZ M A, et al . Statistical outlier detection (SOD): A computer program for detecting outliers in data[R]. NTRS. NASNA, 1980.

STURM J, ENGELHARD N, ENDRES F, et al . A benchmark for the evaluation of RGB-D SLAM systems[C]// Proceedings of 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems ( IROS ). Vilamoura-Algarve, Portugal: IEEE, 2012.

A GEIGER , P LENZ , C STILLER , 等 . Vision meets robotics: The KITTI dataset . The International Journal of Robotics Research , 2013 . 32 ( 11 ): 1231 - 1237 . DOI: 10.1177/0278364913491297 http://doi.org/10.1177/0278364913491297 .

GEIGER A, LENZ P, URTASUN R. Are we ready for autonomous driving? The KITTI vision benchmark suite[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Providence, RI, USA: IEEE, 2012.

SIMON M, MILZ S, AMENDE K, et al . Complex-YOLO: real-time 3D object detection on point clouds[J]. arXiv : 1803.06199, 2018.

SHI S S, WANG X G, LI H S. PointRCNN: 3D object proposal generation and detection from point cloud[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Long Beach, CA, USA: IEEE, 2019: 770-779.

CHEN X, MA H, WAN J, et al . Multi-view 3D object detection network for autonomous driving[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition . Honolulu, HI, USA: IEEE, 2017.

浏览量

157

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

无人机绝对视觉定位的研究进展

基于自适应遮挡恢复与拓扑姿态双向感知的三维手部重建方法

基于Mamba-2的视频快照压缩成像重构方法

基于光流预测的三维光场显示透视关系校正