1.辽宁工程技术大学 电子与信息工程学院, 辽宁 葫芦岛 125105
2.辽宁工程技术大学 电气与控制工程学院, 辽宁 葫芦岛 125105
[ "安彤(1999—),女,辽宁本溪人,硕士研究生,2021年于辽宁工程技术大学获得学士学位,主要从事光流估计方面的研究。E-mail:1319423118@qq.com" ]
[ "贾迪(1982—),男,辽宁沈阳人,博士,教授,2011年于东北大学获得博士学位,主要从事立体匹配与三位重建、摄影测量、视觉空间定位等方面的研究。E-mail:lntu_jiadi@163.com" ]
扫 描 看 全 文
安彤, 贾迪, 张家宝, 等. 融合序列影像相关区域信息的光流估计网络[J]. 液晶与显示, 2023,38(10):1434-1444.
AN Tong, JIA Di, ZHANG Jia-bao, et al. Optical flow estimation
安彤, 贾迪, 张家宝, 等. 融合序列影像相关区域信息的光流估计网络[J]. 液晶与显示, 2023,38(10):1434-1444. DOI: 10.37188/CJLCD.2022-0384.
AN Tong, JIA Di, ZHANG Jia-bao, et al. Optical flow estimation
针对现有光流估计方法在目标轮廓分割不清晰、缺乏细粒度的问题,本文提出融合序列影像相关区域信息的光流估计网络。通过特征编码器和全局编码器分别提取图像的编码特征和上下文特征,并通过下采样处理缩减特征尺寸。在构建4D相关体前,对输入的连续两帧特征图进行分区处理,以强弱相关的方式计算稠密的视觉相似度,建立更为精细的4D相关体积。在迭代更新阶段,提出残差卷积滤波器和细粒度模块,分别应用于处理相关体和光流传递,使得在融合相关体信息和光流信息前保留更多的局部小位移信息。在KITTI-2015数据集和MPI-Sintel数据集上与其他方法进行对比,光流估计评价指标分别提升了8.2%和6.15%。本文给出的网络模型可以更好地提高光流估计的准确性,有效解决了光流场过于平滑、缺乏细粒度和忽略小物体运动等问题。
Aiming at the problems of unclear target contour segmentation and poor granularity in existing optical flow estimation methods, an optical flow estimation ,via, fusing sequence image intensity correlation information is proposed. First, The coding features and contextual features of the images are extracted by the feature encoder and the global encoder, respectively, and the feature sizes are reduced by downsampling processing. Then, before constructing 4D correlation volume, the input two consecutive frames of feature maps are divided into regions to calculate dense visual similarity in the form of strong and weak correlation to build a more refined 4D correlation volume. Finally, in the iterative update stage, the residual convolution filter and the fine-grained module are proposed to be applied to process the correlation volume and optical flow transmission, respectively, which allows to retain more local small displacement information before fusing the correlation volume information and optical flow information. In comparison with other methods on the KITTI-2015 and MPI-Sintel, the optical flow estimation evaluation metric (Endpoint error, EPE) is improved by 8.2% and 6.15%, respectively. The network model given in this paper can better improve the accuracy of optical flow estimation and effectively solve the problems of the optical flow prediction field being over smooth, lacking of fine granularity and ignoring of small object motion.
计算机视觉深度学习光流区域匹配迭代更新
computer visiondeep learningoptical flowregion matchingiterative update
DÉRIAN P, ALMAR R. Wavelet-based optical flow estimation of instant surface currents from shore-based and UAV videos [J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(10): 5790-5797. doi: 10.1109/tgrs.2017.2714202http://dx.doi.org/10.1109/tgrs.2017.2714202
李亚玮,金立左,孙长银,等. 基于光流约束自编码器的动作识别[J]. 东南大学学报(自然科学版),2017,47(4):691-696. doi: 10.3969/j.issn.1001-0505.2017.04.011http://dx.doi.org/10.3969/j.issn.1001-0505.2017.04.011
LI Y W, JIN L Z, SUN C Y, et al. Action recognition based on optical flow constrained auto-encoder [J]. Journal of Southeast University (Natural Science Edition), 2017, 47(4): 691-696. (in Chinese). doi: 10.3969/j.issn.1001-0505.2017.04.011http://dx.doi.org/10.3969/j.issn.1001-0505.2017.04.011
张博,龙慧,刘刚. 基于特征约束与光流场模型的多通道视频目标跟踪算法[J]. 液晶与显示,2021,36(11):1554-1564. doi: 10.37188/CJLCD.2021-0113http://dx.doi.org/10.37188/CJLCD.2021-0113
ZHANG B, LONG H, LIU G. Multi-channel video target tracking algorithm based on feature constraint and optical flow field model [J]. Chinese Journal of Liquid Crystals and Displays, 2021, 36(11): 1554-1564. (in Chinese). doi: 10.37188/CJLCD.2021-0113http://dx.doi.org/10.37188/CJLCD.2021-0113
蒋欣兰,王胜春,罗四维,等. 车载前向运动视频的实时全景成像方法[J]. 光学学报,2017,37(5):0515003. doi: 10.3788/aos201737.0515003http://dx.doi.org/10.3788/aos201737.0515003
JIANG X L, WANG S C, LUO S W, et al. Real-time panoramic imaging method for train-borne forward motion video [J]. Acta Optica Sinica, 2017, 37(5): 0515003. (in Chinese). doi: 10.3788/aos201737.0515003http://dx.doi.org/10.3788/aos201737.0515003
HORN B K P, SCHUNCK B G. Determining optical flow [J]. Artificial Intelligence, 1981, 17(1/3): 185-203. doi: 10.1016/0004-3702(81)90024-2http://dx.doi.org/10.1016/0004-3702(81)90024-2
BLACK M J, ANANDAN P. A framework for the robust estimation of optical flow [C]//Proceedings of the 4th International Conference on Computer Vision. Berlin: IEEE, 1993: 231-236.
ZACH C, POCK T, BISCHOF H. A duality based approach for realtime TV-L1 optical flow [C]//Proceedings of the 29th Pattern Recognition Symposium. Heidelberg: Springer, 2007: 214-223. doi: 10.1007/978-3-540-74936-3_22http://dx.doi.org/10.1007/978-3-540-74936-3_22
WEINZAEPFEL P, REVAUD J, HARCHAOUI Z, et al. DeepFlow: large displacement optical flow with deep matching [C]//Proceedings of 2013 IEEE International Conference on Computer Vision. Sydney: IEEE, 2013: 1385-1392. doi: 10.1109/iccv.2013.175http://dx.doi.org/10.1109/iccv.2013.175
SIMONYAN K, ZISSERMAN A. Two-stream convolutional networks for action recognition in videos [C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. Montreal: MIT Press, 2014: 568-576.
ILG E, MAYER N, SAIKIA T, et al. FlowNet 2.0: Evolution of optical flow estimation with deep networks [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 1647-1655. doi: 10.1109/cvpr.2017.179http://dx.doi.org/10.1109/cvpr.2017.179
RANJAN A, BLACK M J. Optical flow estimation using a spatial pyramid network [C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 2720-2729. doi: 10.1109/cvpr.2017.291http://dx.doi.org/10.1109/cvpr.2017.291
SUN D Q, YANG X D, LIU M Y, et al. PWC-Net: CNNs for optical flow using pyramid, warping, and cost volume [C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 8934-8943. doi: 10.1109/cvpr.2018.00931http://dx.doi.org/10.1109/cvpr.2018.00931
YANG G S, RAMANAN D. Volumetric correspondence networks for optical flow [C]//Proceedings of the 33rd International Conference on Neural Information Processing Systems. Vancouver: Curran Associates Inc., 2019: 72.
HUR J, ROTH S. Iterative residual refinement for joint optical flow and occlusion estimation [C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 5747-5756. doi: 10.1109/cvpr.2019.00590http://dx.doi.org/10.1109/cvpr.2019.00590
LU Y, VALMADRE J, WANG H, et al. Devon: deformable volume network for learning optical flow [C]//2020 IEEE Winter Conference on Applications of Computer Vision (WACV). Snowmass: IEEE, 2020: 2694-2702. doi: 10.1109/wacv45572.2020.9093590http://dx.doi.org/10.1109/wacv45572.2020.9093590
AMOS B, KOLTER J Z. OptNet: differentiable optimization as a layer in neural networks [C]//Proceedings of the 34th International Conference on Machine Learning. Sydney: JMLR.org, 2017: 136-145.
AGRAWAL A, AMOS B, BARRATT S, et al. Differentiable convex optimization layers [C]//Proceedings of the 33rd International Conference on Neural Information Processing Systems. Vancouver: Curran Associates Inc., 2019: 858.
TEED Z, DENG J. RAFT: Recurrent all-pairs field transforms for optical flow [C]//Proceedings of the 16th European Conference on Computer Vision. Glasgow: Springer, 2020: 402-419. doi: 10.1007/978-3-030-58536-5_24http://dx.doi.org/10.1007/978-3-030-58536-5_24
GEIGER A, LENZ P, STILLER C, et al. Vision meets robotics: the KITTI dataset [J]. International Journal of Robotics Research, 2013, 32(11):1231-1237. doi: 10.1177/0278364913491297http://dx.doi.org/10.1177/0278364913491297
BUTLER D J, WULFF J, STANLEY G B, et al. A naturalistic open source movie for optical flow evaluation [C]// Proceedings of European Conference on Computer Vision. Berlin, Heidelberg: ECCV, 2012: 611-625. doi: 10.1007/978-3-642-33783-3_44http://dx.doi.org/10.1007/978-3-642-33783-3_44
WANG J Y, ZHONG Y R, DAI Y C, et al. Displacement-invariant matching cost learning for accurate optical flow estimation [C]//Proceedings of the 34th International Conference on Neural Information Processing Systems. Vancouver: Curran Associates Inc., 2020: 1276.
0
浏览量
69
下载量
0
CSCD
关联资源
相关文章
相关作者
相关机构