Real-time high-precision model reconstruction based on global optimization

XU Xin-ao; LI Yi-xuan; QIAN Jia-ming; FENG Shi-jie; ZUO Chao

doi:10.37188/CJLCD.2023-0086

您当前的位置：

首页 >

文章列表页 >

Real-time high-precision model reconstruction based on global optimization

Intelligent Imaging | 更新时间：2023-06-12

- Real-time high-precision model reconstruction based on global optimization
- Chinese Journal of Liquid Crystals and Displays Vol. 38, Issue 6, Pages: 748-758(2023)
- 作者机构：
  
  1.南京理工大学电子工程与光电技术学院智能计算成像实验室（SCILab），江苏南京 210094
  2.南京理工大学智能计算成像研究院（SCIRI），江苏南京 210019
  3.江苏省光谱成像与智能感知重点实验室，江苏南京 210094
- 作者简介：
- 基金信息：
  
  National Natural Science Foundation of China(62075096;62005121;U21B2033);National Key Research and Development Program of China(2022YFB2804600;2022YFB2804603);Leading Technology of Jiangsu Basic Research Plan(BK20192003);“333 Engineering” Research Project of Jiangsu Province(BRA2016407);Jiangsu Provincial “One Belt and One Road” Innovation Cooperation Project(BZ2020007);Fundamental Research Funds for the Central Universities(30921011208;30919011222;30920032101);Open Research Fund of Jiangsu Key Laboratory of Spectral Imaging & Intelligent Sense(JSGP202105;JSGP202201);Postgraduate Research & Practice Innovation Program of Jiangsu Province(KYCX22_0412)
- DOI：10.37188/CJLCD.2023-0086
  CLC：
扫描看全文
XU Xin-ao, LI Yi-xuan, QIAN Jia-ming, et al. Real-time high-precision model reconstruction based on global optimization. [J]. Chinese Journal of Liquid Crystals and Displays 38(6):748-758(2023)
DOI：

XU Xin-ao, LI Yi-xuan, QIAN Jia-ming, et al. Real-time high-precision model reconstruction based on global optimization. [J]. Chinese Journal of Liquid Crystals and Displays 38(6):748-758(2023) DOI： 10.37188/CJLCD.2023-0086.

摘要

三维形貌测量在先进制造、航空航天、生物医学等领域发挥着重要的应用。凭借高精度、全视场、非接触等优点，条纹投影轮廓术是目前使用最广泛的一种光学三维测量手段。为了获得物体全局三维信息，通常需要将待测物置于转台之上，通过不断地扫描和拼接来获得物体的全局信息。然而，传统的扫描和拼接是以离线的方式进行的，导致整个三维模型的重建速度缓慢。现有的实时点云配准方法虽然能够有效提高点云扫描与拼接的速度，但实时点云拼接的精度依然受待测物的运动状态影响。本文针对上述问题进行优化改进，提出一种基于全局优化的实时高精度模型重建方法。首先，介绍了一种由粗配准到精配准的快速点云配准算法并提出了基于点云法向量约束的点云初始化算法，能够提升粗配准过程中点云初始位姿计算的稳定性与精度。其次，在精配准阶段引入了图优化算法以获得全局点云位姿的最优解，进一步提升了全局点云配准的精度。实验结果表明，所提方法相比于现有实时模型重建方法，能够实现更高精度且稳定的全局点云配准。特别地，针对动态场景中由于抖动等因素引起的被测物体速度突变等情况，本方法依然能够鲁棒地完成三维模型重建，全方位模型重建的精度达84 μm。

Abstract

Three-dimensional （3D） shape measurement plays an important role in advanced manufacturing， aerospace， biomedicine and other fields. With the advantages of high precision， full field of view， and non-contact， fringe projection profilometry is currently the most widely used optical three-dimensional measurement method. In order to obtain 360° global three-dimensional information， it is usually necessary to place the object to be measured on the turntable， and obtain the global information of the object by continuous scanning and stitching. However， the traditional scanning and stitching are performed offline， resulting in slow reconstruction of the entire 3D model. Although the existing real-time point cloud registration methods can effectively improve the speed of point cloud scanning and stitching， the accuracy of real-time point cloud stitching is still limited by the motion state of the object under test. This paper optimizes and improves the above problems， and proposes a real-time high-precision model reconstruction method based on global optimization. Firstly， a fast point cloud registration algorithm from coarse to fine registration is introduced， and a point cloud initialization algorithm based on point cloud normal vector constraints is proposed on this basis， which can improve the stability and accuracy of the point cloud initial pose calculated during the rough registration process. Secondly， a graph optimization algorithm is introduced in the fine registration stage to obtain the optimal solution of the global point cloud pose， which further improves the accuracy of global point cloud registration. The experimental results show that the proposed method can achieve higher precision and stable global point cloud registration than the current real-time point cloud registration method. In particular， this method can still robustly complete the reconstruction of the 3D model， and the accuracy of the omni-directional model reconstruction reaches 84 μm， especially for situations such as sudden changes in the speed of the measured object caused by factors such as jitter in the dynamic scene.

关键词

条纹投影轮廓术图优化实时三维重建点云配准

Keywords

fringe projection profilometrygraph optimizationreal-time3D reconstructionpoint cloud registration

references

FENG S J， ZUO C， TAO T Y， et al. Robust dynamic 3-D measurements with motion-compensated phase-shifting profilometry ［J］. Optics and Lasers in Engineering， 2018， 103： 127-138. doi: 10.1016/j.optlaseng.2017.12.001http://dx.doi.org/10.1016/j.optlaseng.2017.12.001

PAN B， XIE H M， WANG Z Y， et al. Study on subset size selection in digital image correlation for speckle patterns ［J］. Optics Express， 2008， 16（10）： 7037-7048. doi: 10.1364/oe.16.007037http://dx.doi.org/10.1364/oe.16.007037

HU Y， CHEN Q， FENG S J， et al. Microscopic fringe projection profilometry： a review ［J］. Optics and Lasers in Engineering， 2020， 135： 106192. doi: 10.1016/j.optlaseng.2020.106192http://dx.doi.org/10.1016/j.optlaseng.2020.106192

TAO T Y， CHEN Q， FENG S J， et al. High-speed real-time 3D shape measurement based on adaptive depth constraint ［J］. Optics Express， 2018， 26（17）： 22440-22456. doi: 10.1364/oe.26.022440http://dx.doi.org/10.1364/oe.26.022440

ZUO C， QIAN J M， FENG S J， et al. Deep learning in optical metrology： a review ［J］. Light： Science & Applications， 2022， 11（1）： 39. doi: 10.1038/s41377-022-00714-xhttp://dx.doi.org/10.1038/s41377-022-00714-x

GENG J. Structured-light 3D surface imaging： a tutorial ［J］. Advances in Optics and Photonics， 2011， 3（2）： 128-160. doi: 10.1364/aop.3.000128http://dx.doi.org/10.1364/aop.3.000128

FENG S J， CHEN Q， GU G H， et al. Fringe pattern analysis using deep learning ［J］. Advanced Photonics， 2019， 1（2）： 025001. doi: 10.1117/1.ap.1.2.025001http://dx.doi.org/10.1117/1.ap.1.2.025001

LI Y X， QIAN J M， FENG S J， et al. Deep-learning-enabled dual-frequency composite fringe projection profilometry for single-shot absolute 3D shape measurement ［J］. Opto-Electronic Advances 2022， 5（5）： 210021. doi: 10.29026/oea.2022.210021http://dx.doi.org/10.29026/oea.2022.210021

SOLAV D， MOERMAN K M， JAEGER A M， et al. MultiDIC： an open-source toolbox for multi-view 3D digital image correlation ［J］. IEEE Access， 2018， 6： 30520-30535. doi: 10.1109/access.2018.2843725http://dx.doi.org/10.1109/access.2018.2843725

EPSTEIN E， GRANGER-PICHÉ M， POULIN P. Exploiting mirrors in interactive reconstruction with structured light ［C］. 9th International Fall Workshop on Vision， Modeling， and Visualization. Stanford： Aka GmbH， 2004： 125-132.

赵云涛，齐佳祥，李维刚，等.基于改进三维形状上下文的点云配准［J］.液晶与显示，2022，37（12）：1590-1597． doi: 10.37188/CJLCD.2022-0156http://dx.doi.org/10.37188/CJLCD.2022-0156

ZHAO Y T， QI J X， LI W G， et al. Point cloud registration based on improved 3DSC ［J］. Chinese Journal of Liquid Crystals and Displays， 2022， 37（12）： 1590-1597. （in Chinese）. doi: 10.37188/CJLCD.2022-0156http://dx.doi.org/10.37188/CJLCD.2022-0156

RUSU R B， BLODOW N， BEETZ M. Fast point feature histograms （FPFH） for 3D registration ［C］.2009 IEEE International Conference on Robotics and Automation. Kobe： IEEE， 2009： 3212-3217. doi: 10.1109/robot.2009.5152473http://dx.doi.org/10.1109/robot.2009.5152473

RUSU R B， MARTON Z C， BLODOW N， et al. Learning informative point classes for the acquisition of object model maps ［C］.2008 10th International Conference on Control， Automation， Robotics and Vision. Hanoi： IEEE， 2008： 643-650. doi: 10.1109/icarcv.2008.4795593http://dx.doi.org/10.1109/icarcv.2008.4795593

RUSU R B， BLODOW N， MARTON Z C， et al. Aligning point cloud views using persistent feature histograms ［C］. 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems. Nice： IEEE， 2008： 3384-3391. doi: 10.1109/iros.2008.4650967http://dx.doi.org/10.1109/iros.2008.4650967

BESL P J， MCKAY N D. Method for registration of 3-D shapes ［C］//Proceedings of SPIE 1611， Sensor Fusion IV： Control Paradigms and Data Structures. Boston： SPIE， 1992： 586-606. doi: 10.1117/12.57955http://dx.doi.org/10.1117/12.57955

LU F， MILIOS E. Robot pose estimation in unknown environments by matching 2D range scans ［J］. Journal of Intelligent and Robotic Systems， 1997， 18（3）： 249-275. doi: 10.1023/a:1007957421070http://dx.doi.org/10.1023/a:1007957421070

MINGUEZ J， LAMIRAUX F， MONTESANO L. Metric-based scan matching algorithms for mobile robot displacement estimation ［C］//Proceedings of the 2005 IEEE International Conference on Robotics and Automation. Barcelona： IEEE， 2005： 3557-3563. doi: 10.1109/tro.2006.878961http://dx.doi.org/10.1109/tro.2006.878961

SOBREIRA H， COSTA C M， SOUSA I， et al. Map-matching algorithms for robot self-localization： a comparison between perfect match， iterative closest point and normal distributions transform ［J］. Journal of Intelligent & Robotic Systems， 2019， 93（3）： 533-546. doi: 10.1007/s10846-017-0765-5http://dx.doi.org/10.1007/s10846-017-0765-5

BIBER P， FLECK S， STRASSER W. A probabilistic framework for robust and accurate matching of point clouds ［C］//Proceedings of the 26th DAGM Symposium on Pattern Recognition. Tübingen： Springer， 2004： 480-487. doi: 10.1007/978-3-540-28649-3_59http://dx.doi.org/10.1007/978-3-540-28649-3_59

SEGAL A V， HAEHNEL D， THRUN S. Generalized-ICP ［M］//TRINKLE J， MATSUOKA Y， CASTELLANOS J A. Robotics： Science and Systems V. Cambridge： The MIT Press， 2010： 161-168. doi: 10.15607/rss.2009.v.021http://dx.doi.org/10.15607/rss.2009.v.021

KOIDE K， YOKOZUKA M， OISHI S， et al. Voxelized GICP for fast and accurate 3D point cloud registration ［C］//Proceedings of the 2021 IEEE International Conference on Robotics and Automation （ICRA）. Xi'an： IEEE， 2021： 11054-11059. doi: 10.1109/icra48506.2021.9560835http://dx.doi.org/10.1109/icra48506.2021.9560835

ZHOU Q Y， PARK J， KOLTUN V. Fast global registration ［C］. 14th European Conference on Computer Vision. Amsterdam， The Netherlands： Springer， 2016： 766-782. doi: 10.1007/978-3-319-46475-6_47http://dx.doi.org/10.1007/978-3-319-46475-6_47

RUSINKIEWICZ S， HALL-HOLT O， LEVOY M. Real-time 3D model acquisition ［J］. ACM Transactions on Graphics， 2002， 21（3）： 438-446. doi: 10.1145/566654.566600http://dx.doi.org/10.1145/566654.566600

MUR-ARTAL R， TARDÓS J D. ORB-SLAM2： an open-source SLAM system for monocular， stereo， and RGB-D cameras ［J］. IEEE Transactions on Robotics， 2017， 33（5）： 1255-1262. doi: 10.1109/tro.2017.2705103http://dx.doi.org/10.1109/tro.2017.2705103

NEWCOMBE R A， IZADI S， HILLIGES O， et al. KinectFusion： real-time dense surface mapping and tracking ［C］. 2011 10th IEEE International Symposium on Mixed and Augmented Reality. Basel： IEEE， 2011： 127-136. doi: 10.1109/ismar.2011.6092378http://dx.doi.org/10.1109/ismar.2011.6092378

IZADI S， KIM D， HILLIGES O， et al. KinectFusion： real-time 3D reconstruction and interaction using a moving depth camera ［C］//Proceedings of the 24th annual ACM Symposium on User Interface Software and Technology. Santa Barbara： ACM， 2011： 559-568. doi: 10.1145/2047196.2047270http://dx.doi.org/10.1145/2047196.2047270

QIAN J M， FENG S J， TAO T Y， et al. High-resolution real-time 360° 3D model reconstruction of a handheld object with fringe projection profilometry ［J］. Optics Letters， 2019， 44（23）： 5751-5754. doi: 10.1364/ol.44.005751http://dx.doi.org/10.1364/ol.44.005751

SPRICKERHOF J， NÜCHTER A， LINGEMANN K， et al. A heuristic loop closing technique for large-scale 6D SLAM ［J］. Automatika， 2011， 52（3）： 199-222. doi: 10.1080/00051144.2011.11828420http://dx.doi.org/10.1080/00051144.2011.11828420

Views

101

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Point cloud registration based on improved 3DSC

Point cloud registration method based on electro-magnetic tracking system

Design of general-purpose video image process system based on Compact-PCI bus and FPGA + DSP structure

Module design for liquid crystal display based on DSP

Related Author

No data

Related Institution

College of Information Science and Engineering， Wuhan University of Science and Technology

Engineering Research Center for Metallurgical Automation and Detecting Technology of Ministry of Education， Wuhan University of Science and Technology

College of Computer Science and Technology, Shandong University of Technology

Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences

University of Chinese Academy of Sciences

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China 130033 Postal code：130033
Tel：0431-86176059 Email：yjxs@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰