自由立体显示中基于深度卷积神经网络的虚拟视点生成方法

付傲威; 赵敏; 罗令; 邢妍; 邓欢; 王琼华

doi:10.3788/YJYXS20193411.1031

您当前的位置：

首页 >

文章列表页 >

自由立体显示中基于深度卷积神经网络的虚拟视点生成方法

材料与器件 | 更新时间：2020-07-05

- 自由立体显示中基于深度卷积神经网络的虚拟视点生成方法
- Virtual viewpoint image generation method based on deep convolutional neural network in autostereoscopic display
- 液晶与显示 2019年34卷第11期页码：1031-1036
- 作者机构：
  
  1.四川大学电子信息学院, 四川成都 610065
  2.北京航空航天大学仪器科学与光电工程学院, 北京 100191
- 作者简介：
  
  [ "付傲威(1995-), 男, 湖北襄阳人, 硕士研究生, 2017年于三峡大学获得学士学位, 主要从事三维显示方面的研究工作。E-mail:Aowei_fu@163.com" ]
  [ "王琼华(1969-), 女, 四川巴中人, 博士, 教授, 博士生导师, 2001年于电子科技大学获得博士学位, 主要从事信息显示方面的研究工作。E-mail:qhwang@scu.edu.cn" ]
- 基金信息：
  
  国家重点研发计划(2017YFB1002900);Supported by the National Key R&D Program of China(2017YFB1002900)
- DOI：10.3788/YJYXS20193411.1031
  中图分类号： TB133
- 收稿日期：2019-08-22，
  
  录用日期：2019-9-2，
  
  纸质出版日期：2019-11-05
- 稿件说明：
移动端阅览
付傲威, 赵敏, 罗令, 等. 自由立体显示中基于深度卷积神经网络的虚拟视点生成方法[J]. 液晶与显示, 2019,34(11):1031-1036.

Ao-wei FU, Min ZHAO, Ling LUO, et al. Virtual viewpoint image generation method based on deep convolutional neural network in autostereoscopic display[J]. Chinese journal of liquid crystals and displays, 2019, 34(11): 1031-1036.
付傲威, 赵敏, 罗令, 等. 自由立体显示中基于深度卷积神经网络的虚拟视点生成方法[J]. 液晶与显示, 2019,34(11):1031-1036. DOI： 10.3788/YJYXS20193411.1031.

Ao-wei FU, Min ZHAO, Ling LUO, et al. Virtual viewpoint image generation method based on deep convolutional neural network in autostereoscopic display[J]. Chinese journal of liquid crystals and displays, 2019, 34(11): 1031-1036. DOI： 10.3788/YJYXS20193411.1031.

摘要

传统虚拟视点生成采用像素填充法对生成的虚拟视点图像进行空洞填充和伪影修复，其修复效果无法满足自由立体显示需求。为了获取高质量的虚拟视点图像，提出了一种基于深度卷积神经网络的虚拟视点生成方法。该方法采用随机初始化的深度卷积神经网络作为图像先验，经过卷积神经网络结构的不断迭代，对虚拟视点图像的空洞和伪影进行修复，并将得到的高质量虚拟视点图像合成为自由立体图像，用于自由立体显示。修复后的虚拟视点图像的PSNR均值为25.6，相比传统像素填充方法有明显提升。实验结果表明，所提方法能够实现高质量的自由立体显示效果。

Abstract

The conventional virtual viewpoint generation methods use the pixel filling algorithm to fill the image holes and restore artifacts on the generated virtual viewpoint images. However

the repaired virtual viewpoint images cannot meet the requirements of the autostereoscopic display. In order to obtain virtual viewpoint images with high quality

we propose a virtual viewpoint image generation method for autostereoscopic display based on deep convolutional neural network (DCNN). The method uses a randomly initialized deep convolutional neural network as an image prior

iterates through the structure of the convolutional neural network

and effectively repairs the image holes and artifacts of the virtual viewpoint image. The repaired virtual viewpoint images are then synthesized into the autostereoscopic image for autostereoscopic 3D display. The average value of PSNR of the virtual viewpoint images is 25.6

which is higher than that of the conventional pixel filling method. The experimental results demonstrate the proposed method can obtain high quality autostereoscopic 3D images.

关键词

Keywords

references

王琼华.3D显示技术与器件[M].北京:科学出版社, 2011.

WANG Q H. 3 D Display Technology and Device [M]. Beijing:Science Press, 2011. (in Chinese)

王琼华, 王爱红.三维立体显示综述[J].计算机应用, 2010, 30(3):579-581, 588.

WANG Q H, WANG A H. Survey on stereoscopic three-dimensional display[J]. Journal of Computer Applications , 2010, 30(3):579-581, 588. (in Chinese)

ZHANG Y Q, WANG Q H, WANG A H, et al . Depth-map generation for multi-view autostereoscopic 3-D displays based on the sift algorithm constrained by boundary[J]. Journal of the Society for Information Display , 2010, 18(7):513-518.

XIONG Z L, WANG Q H, LI S L, et al . Partially-overlapped viewing zone based integral imaging system with super wide viewing angle[J]. Optics Express , 2014, 22(19):22268-22277.

DENG H, WANG Q H, Li D H. Method of generating orthoscopic elemental image array from sparse camera array[J]. Chinese Optics Letters , 2012, 10(6):061102.

XIAO F, XIONG Z L, DENG H, et al . P-100:Approach on two-step 3D warping with background filling for multi-view autostereoscopic display[J]. SID Symposium Digest of Technical Papers , 2017, 48(1):1624-1626.

赵悟翔, 王琼华, 李大海, 等.阶梯光栅多视点自由立体显示的子像素排列[J].四川大学学报(工程科学版), 2009, 41(6):216-218, 236.

ZHAO W X, WANG Q H, LI D H, et al . Sub-pixel arrangement for multi-view autostereoscopic display based on step parallax barrier[J]. Journal of Sichuan University ( Engineering Science Edition ), 2009, 41(6):216-218, 236. (in Chinese)

FEHN C. Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV[C]// Proceedings of SPIE Stereoscopic Displays and Virtual Reality Systems XI . San Jose, California, United States: SPIE, 2004: 93-104. https://www.researchgate.net/publication/228747651_Depth-image-based_rendering_DIBR_compression_and_transmission_for_a_new_approach_on_3D-TV

MORI Y, FUKUSHIMA N, YENDO T, et al . View generation with 3D warping using depth information for FTV[J]. Signal Processing : Image Communication , 2009, 24(1/2):65-72.

LEMPITSKY V, VEDALDI A, ULYANOV D. Deep image prior[C]// Proceedings of 2018 IEEE / CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City, UT, USA: IEEE, 2018: 9446-9454. https://www.researchgate.net/publication/329745621_Deep_Image_Prior

CHAWLA N V, BOWYER K W, HALL L O, et al . SMOTE:synthetic minority over-sampling technique[J]. Journal of Artificial Intelligence Research , 2002, 16(1):321-357.

BERTALMIO M, VESE L, SAPIRO G, et al . Simultaneous structure and texture image inpainting[J]. IEEE Transactions on Image Processing , 2003, 12(8):882-889.

PATHAK D, KRÄHENBUHL P, DONAHUE J, et al . Context encoders: feature learning by inpainting[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas, NV, USA: IEEE, 2016: 2536-2544. https://www.researchgate.net/publication/301816789_Context_Encoders_Feature_Learning_by_Inpainting

YANG C, LU X, LIN Z, et al . High-resolution image inpainting using multi-scale neural patch synthesis[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Honolulu, HI, USA: IEEE, 2017: 4076-4084. https://www.researchgate.net/publication/320964396_High-Resolution_Image_Inpainting_Using_Multi-scale_Neural_Patch_Synthesis

黄冠婷, 韩学辉, 龚晓婷, 等.基于图像分割和区域匹配的灰度图像彩色化算法[J].液晶与显示, 2019, 34(6):619-626.

HUANG G T, HAN X H, GONG X T, et al . Gray image colorization algorithm based on image segmentation and region matching[J]. Chinese Journal of Liquid Crystals and Displays , 2019, 34(6):619-626. (in Chinese)

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于门控卷积与注意迁移的二阶图像修复

垂直/倾斜型交错狭缝光栅对立体显示效果的影响

基于深度卷积神经网络的多特征融合的手势识别

基于边缘结构的Criminisi图像修复

柱透镜光栅投影3D显示的视点数与串扰容限