Depth image restoration method based oncolor image contour

Dong SU; Yan ZHANG; Cheng-zhi QU; Xin ZHANG

doi:10.37188/CJLCD.2020-0222

您当前的位置：

首页 >

文章列表页 >

Depth image restoration method based oncolor image contour

Image Processing | 更新时间：2021-03-23

- Depth image restoration method based oncolor image contour
- Chinese Journal of Liquid Crystals and Displays Vol. 36, Issue 3, Pages: 456-464(2021)
- 作者机构：
  
  中山大学航空航天学院, 广东深圳 518060
- 作者简介：
- 基金信息：
  
  Supported by National Natural Science Foundation
- DOI：10.37188/CJLCD.2020-0222
  CLC： TP394.1
- Received：31 August 2020，
  
  Accepted：27 September 2020，
  
  Published：2021-03
- 稿件说明：
移动端阅览
Dong SU, Yan ZHANG, Cheng-zhi QU, et al. Depth image restoration method based oncolor image contour[J]. Chinese journal of liquid crystals and displays, 2021, 36(3): 456-464.
DOI：

Dong SU, Yan ZHANG, Cheng-zhi QU, et al. Depth image restoration method based oncolor image contour[J]. Chinese journal of liquid crystals and displays, 2021, 36(3): 456-464. DOI： 10.37188/CJLCD.2020-0222.

摘要

针对飞行时间相机获取的深度图像存在大量无效点的问题，提出了一种基于彩色图像轮廓的深度图像修复方法。该方法先通过无效点邻域内像素的值和高斯函数来确定该邻域的方差，结合无效点邻域内有效点个数和方差计算所有无效点的填充顺序，完成填充过程的优先级评估；再将彩色图像中目标物体轮廓投射到深度图像中，结合投影轮廓和欧式距离函数实现对无效点的填充。实验结果表明，与其他常用的修复算法相比，本文算法可以在保护目标物体边缘的前提下，实现对无效点的有效填充，针对不同目标物体，信噪比指标提高了2~14 dB。

Abstract

Aiming at the problem of a large number of invalid points in the depth image obtained by the time-of-flight camera

a depth image restoration method based on the color image contour is proposed. This method first determines the variance of the neighborhood through the value of the pixel in the neighborhood of the invalid point and the Gaussian function

and calculates the filling order of all invalid points based on the number and variance of the valid point in the neighborhood of the invalid point

and completes the priority evaluation of the filling process. The contour of the target object in the color image is projected into the depth image

combined with the projection contour and the Euclidean distance function to realize the filling of invalid points. The experimental results show that compared with other commonly used denoising algorithms

the algorithm in this paper can effectively fill invalid points on the premise of protecting the edge of the target object. The signal-to-noise ratio index is improved by 2-14 dB for different target objects.

关键词

Keywords

references

吴冬鹏, 毕笃彦, 何林远, 等. 基于NSSCT的红外与可见光图像融合[J]. 光学学报, 2017, 37(7): 0710003.

WU D P, BI D Y, HE L Y, et al . A fusion algorithm of infrared and visible image based on NSSCT[J]. Acta Optica Sinica , 2017, 37(7): 0710003. (in Chinese)

于潇, 姚园, 徐正平. 采用APD阵列的共口径激光成像光学系统设计[J]. 中国光学, 2016, 9(3): 349-355.

YU X, YAO Y, XU Z P. Laser imaging optical system design with a shared aperture employing APD array[J]. Chinese Optics , 2016, 9(3): 349-355. (in Chinese)

LIU J Y, GONG X J, LIU J L. Guided inpainting and filtering for Kinect depth maps[C]// Proceedings of the 21st International Conference on Pattern Recognition . Tsukuba: IEEE, 2012.

李诗锐, 李琪, 李海洋, 等. 基于Kinect v2的实时精确三维重建系统[J]. 软件学报, 2016, 27(10): 2519-2529.

LI S R, LI Q, LI H Y, et al . Real-time accurate 3D reconstruction based on Kinect V2[J]. Journal of Software , 2016, 27(10): 2519-2529. (in Chinese)

NEWCOMBE R A, IZADI S, HILLIGES O, et al . KinectFusion: real-time dense surface mapping and tracking[C]// Proceedings of 2011 10 th IEEE International Symposium on Mixed and Augmented Reality . Basel: IEEE, 2011: 127-136.

王亚强, 陈波. 一种改进的各向异性扩散超声图像去噪算法[J]. 液晶与显示, 2015, 30(2): 310-316.

WANG Y Q, CHEN B. Improved anisotropic diffusion ultrasound image denoising algorithm[J]. Chinese Journal of Liquid Crystals and Displays , 2015, 30(2): 310-316. (in Chinese)

LE A V, JUNG S W, WON C S. Directional joint bilateral filter for depth images[J]. Sensors , 2014, 14(7): 11362-11378.

CHEN L, LIN H, LI S T. Depth image enhancement for Kinect using region growing and bilateral filter[C]// Proceedings of the 21 st International Conference on Pattern Recognition . Tsukuba: IEEE, 2012.

WANG Z Y, SONG X W, WANG S Z, et al . Filling Kinect depth holes via position-guided matrix completion[J]. Neurocomputing , 2016, 215: 48-52.

DIEBEL J, THRUN S. An application of Markov random fields to range sensing[C]// Proceedings of the 18 th International Conference on Neural Information Processing Systems . Cambridge: MIT Press, 2005: 291-298.

YANG N E, KIM Y G, Park R H. Depth hole filling using the depth distribution of neighboring regions of depth holes in the Kinect sensor[C]// Proceedings of 2012 IEEE International Conference on Signal Processing, Communication and Computing. Hong Kong, China: IEEE, 2012: 658-661.

CRIMINISI A, PEREZ P, TOYAMA K. Region filling and object removal by exemplar-based image inpainting[J]. IEEE Transactions on Image Processing , 2004, 13(9): 1200-1212.

XIONG X L, CHEN D Y, ZHANG W W, et al . Depth map recovery based on optimal scheduling of invalid point filling[C]//2018 Chinese Control And Decision Conference . Shenyang: IEEE, 2018: 6684-6690.

ATAPOUR-ABARGHOUEI A, DE LA GARANDERIE G P, BRECKON T P. Back to Butterworth-a Fourier basis for 3D surface relief hole filling within RGB-D imagery[C]//2016 23 rd International Conference on Pattern Recognition. Cancun: IEEE, 2016.

朱文杰, 王广龙, 田杰, 等. 基于多特征的复杂场景运动目标检测[J]. 光学学报, 2018, 38(6): 0612004.

ZHU W J, WANG G L, TIAN J, et al . Detection of moving objects in complex scenes based on multiple features[J]. Acta Optica Sinica , 2008, 38(6): 0612004. (in Chinese)

刘艳, 李腾飞. 对张正友相机标定法的改进研究[J]. 光学技术, 2014, 40(6): 565-570.

LIU Y, LI T F. Reaserch of the improvement of Zhang's camera calibration method[J]. Optical Technique , 2014, 40(6): 565-570. (in Chinese)

HIRSCHMULLER H, SCHARSTEIN D. Evaluation of cost functions for stereo matching[C]// Proceedings of 2007 IEEE Conference on Computer Vision and Pattern Recognition . Minneapolis: IEEE, 2007: 1-8.

SCHARSTEIN D, PAL C. Learning conditional random fields for stereo[C]// Proceedings of 2007 IEEE Conference on Computer Vision and Pattern Recognition . Minneapolis: IEEE, 2007: 1-8.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

No data

Related Author

dong Su

yan Zhang

chengzhi Qu

xin Zhang

Related Institution

College of aeronautics and astronautics，Sun Yat-Sen University

AI问答

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China 130033 Postal code：130033
Tel：0431-86176059 Email：yjxs@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰