基于彩色图像轮廓的深度图像修复方法

苏东; 张艳; 曲承志; 张鑫

doi:10.37188/CJLCD.2020-0222

您当前的位置：

首页 >

文章列表页 >

基于彩色图像轮廓的深度图像修复方法

图像处理 | 更新时间：2021-03-23

- 基于彩色图像轮廓的深度图像修复方法
- Depth image restoration method based oncolor image contour
- 液晶与显示 2021年36卷第3期页码：456-464
- 作者机构：
  
  中山大学航空航天学院, 广东深圳 518060
- 作者简介：
  
  [ "苏东(1992-), 男, 新疆乌鲁木齐人, 硕士研究生, 2015年于四川大学获得学士学位, 主要从事多源信息融合、图像处理方面的研究。E-mail:sudong6@mail.sysu.edu.cn" ]
  [ "张艳(1975-), 女, 江苏盐城人, 博士, 教授, 2005年于国防科技大学获得博士学位, 主要从事空间目标探测与识别技术、多源信息融合处理技术方面的研究。E-mail: zhangyan25@mail.sysu.edu.cn" ]
- 基金信息：
  
  国家自然科学基金(U1734208)
- DOI：10.37188/CJLCD.2020-0222
  中图分类号： TP394.1
- 收稿日期：2020-08-31，
  
  录用日期：2020-9-27，
  
  纸质出版日期：2021-03
- 稿件说明：
移动端阅览
苏东, 张艳, 曲承志, 等. 基于彩色图像轮廓的深度图像修复方法[J]. 液晶与显示, 2021,36(3):456-464.

Dong SU, Yan ZHANG, Cheng-zhi QU, et al. Depth image restoration method based oncolor image contour[J]. Chinese journal of liquid crystals and displays, 2021, 36(3): 456-464.
苏东, 张艳, 曲承志, 等. 基于彩色图像轮廓的深度图像修复方法[J]. 液晶与显示, 2021,36(3):456-464. DOI： 10.37188/CJLCD.2020-0222.

Dong SU, Yan ZHANG, Cheng-zhi QU, et al. Depth image restoration method based oncolor image contour[J]. Chinese journal of liquid crystals and displays, 2021, 36(3): 456-464. DOI： 10.37188/CJLCD.2020-0222.

摘要

针对飞行时间相机获取的深度图像存在大量无效点的问题，提出了一种基于彩色图像轮廓的深度图像修复方法。该方法先通过无效点邻域内像素的值和高斯函数来确定该邻域的方差，结合无效点邻域内有效点个数和方差计算所有无效点的填充顺序，完成填充过程的优先级评估；再将彩色图像中目标物体轮廓投射到深度图像中，结合投影轮廓和欧式距离函数实现对无效点的填充。实验结果表明，与其他常用的修复算法相比，本文算法可以在保护目标物体边缘的前提下，实现对无效点的有效填充，针对不同目标物体，信噪比指标提高了2~14 dB。

Abstract

Aiming at the problem of a large number of invalid points in the depth image obtained by the time-of-flight camera

a depth image restoration method based on the color image contour is proposed. This method first determines the variance of the neighborhood through the value of the pixel in the neighborhood of the invalid point and the Gaussian function

and calculates the filling order of all invalid points based on the number and variance of the valid point in the neighborhood of the invalid point

and completes the priority evaluation of the filling process. The contour of the target object in the color image is projected into the depth image

combined with the projection contour and the Euclidean distance function to realize the filling of invalid points. The experimental results show that compared with other commonly used denoising algorithms

the algorithm in this paper can effectively fill invalid points on the premise of protecting the edge of the target object. The signal-to-noise ratio index is improved by 2-14 dB for different target objects.

关键词

Keywords

references

吴冬鹏, 毕笃彦, 何林远, 等. 基于NSSCT的红外与可见光图像融合[J]. 光学学报, 2017, 37(7): 0710003.

WU D P, BI D Y, HE L Y, et al . A fusion algorithm of infrared and visible image based on NSSCT[J]. Acta Optica Sinica , 2017, 37(7): 0710003. (in Chinese)

于潇, 姚园, 徐正平. 采用APD阵列的共口径激光成像光学系统设计[J]. 中国光学, 2016, 9(3): 349-355.

YU X, YAO Y, XU Z P. Laser imaging optical system design with a shared aperture employing APD array[J]. Chinese Optics , 2016, 9(3): 349-355. (in Chinese)

LIU J Y, GONG X J, LIU J L. Guided inpainting and filtering for Kinect depth maps[C]// Proceedings of the 21st International Conference on Pattern Recognition . Tsukuba: IEEE, 2012.

李诗锐, 李琪, 李海洋, 等. 基于Kinect v2的实时精确三维重建系统[J]. 软件学报, 2016, 27(10): 2519-2529.

LI S R, LI Q, LI H Y, et al . Real-time accurate 3D reconstruction based on Kinect V2[J]. Journal of Software , 2016, 27(10): 2519-2529. (in Chinese)

NEWCOMBE R A, IZADI S, HILLIGES O, et al . KinectFusion: real-time dense surface mapping and tracking[C]// Proceedings of 2011 10 th IEEE International Symposium on Mixed and Augmented Reality . Basel: IEEE, 2011: 127-136.

王亚强, 陈波. 一种改进的各向异性扩散超声图像去噪算法[J]. 液晶与显示, 2015, 30(2): 310-316.

WANG Y Q, CHEN B. Improved anisotropic diffusion ultrasound image denoising algorithm[J]. Chinese Journal of Liquid Crystals and Displays , 2015, 30(2): 310-316. (in Chinese)

LE A V, JUNG S W, WON C S. Directional joint bilateral filter for depth images[J]. Sensors , 2014, 14(7): 11362-11378.

CHEN L, LIN H, LI S T. Depth image enhancement for Kinect using region growing and bilateral filter[C]// Proceedings of the 21 st International Conference on Pattern Recognition . Tsukuba: IEEE, 2012.

WANG Z Y, SONG X W, WANG S Z, et al . Filling Kinect depth holes via position-guided matrix completion[J]. Neurocomputing , 2016, 215: 48-52.

DIEBEL J, THRUN S. An application of Markov random fields to range sensing[C]// Proceedings of the 18 th International Conference on Neural Information Processing Systems . Cambridge: MIT Press, 2005: 291-298.

YANG N E, KIM Y G, Park R H. Depth hole filling using the depth distribution of neighboring regions of depth holes in the Kinect sensor[C]// Proceedings of 2012 IEEE International Conference on Signal Processing, Communication and Computing. Hong Kong, China: IEEE, 2012: 658-661.

CRIMINISI A, PEREZ P, TOYAMA K. Region filling and object removal by exemplar-based image inpainting[J]. IEEE Transactions on Image Processing , 2004, 13(9): 1200-1212.

XIONG X L, CHEN D Y, ZHANG W W, et al . Depth map recovery based on optimal scheduling of invalid point filling[C]//2018 Chinese Control And Decision Conference . Shenyang: IEEE, 2018: 6684-6690.

ATAPOUR-ABARGHOUEI A, DE LA GARANDERIE G P, BRECKON T P. Back to Butterworth-a Fourier basis for 3D surface relief hole filling within RGB-D imagery[C]//2016 23 rd International Conference on Pattern Recognition. Cancun: IEEE, 2016.

朱文杰, 王广龙, 田杰, 等. 基于多特征的复杂场景运动目标检测[J]. 光学学报, 2018, 38(6): 0612004.

ZHU W J, WANG G L, TIAN J, et al . Detection of moving objects in complex scenes based on multiple features[J]. Acta Optica Sinica , 2008, 38(6): 0612004. (in Chinese)

刘艳, 李腾飞. 对张正友相机标定法的改进研究[J]. 光学技术, 2014, 40(6): 565-570.

LIU Y, LI T F. Reaserch of the improvement of Zhang's camera calibration method[J]. Optical Technique , 2014, 40(6): 565-570. (in Chinese)

HIRSCHMULLER H, SCHARSTEIN D. Evaluation of cost functions for stereo matching[C]// Proceedings of 2007 IEEE Conference on Computer Vision and Pattern Recognition . Minneapolis: IEEE, 2007: 1-8.

SCHARSTEIN D, PAL C. Learning conditional random fields for stereo[C]// Proceedings of 2007 IEEE Conference on Computer Vision and Pattern Recognition . Minneapolis: IEEE, 2007: 1-8.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

暂无数据