基于神经网络与卡尔曼滤波的手部实时追踪方法

曾公任; 姚剑敏; 严群; 林志贤; 郭太良; 林畅

doi:10.3788/YJYXS20203505.0464

您当前的位置：

首页 >

文章列表页 >

基于神经网络与卡尔曼滤波的手部实时追踪方法

图像处理 | 更新时间：2020-08-12

- 基于神经网络与卡尔曼滤波的手部实时追踪方法
- Hand real-time tracking method based on neural network and Kalman filter
- 液晶与显示 2020年35卷第5期页码：464-470
- 作者机构：
  
  1.平板显示技术国家地方联合工程实验室, 福州大学物理与信息工程学院, 福建福州 350108
  2.晋江市博感电子科技有限公司, 福建晋江 362200
- 作者简介：
  
  [ "曾公任(1994-), 男, 福建福州人, 硕士, 2017年于重庆交通大学获得学士学位, 主要从事图形处理相关技术的研究。E-mail:934121296@qq.com" ]
  [ "姚剑敏(1978-), 男, 福建莆田人, 博士, 副研究员, 2005年于中科院长春光学精密机械与物理研究所获得博士学位, 研究领域为人工智能、图像处理、信息显示技术。E-mail:yaojm@fzu.edu.cn" ]
- 基金信息：
  
  国家重点研发计划资助(2016YFB0401503);广东省科技重大专项(2016B090906001);福建省科技重大专项(2014HZ0003-1);广东省光信息材料与技术重点实验室开放基金资助项目(2017B030301007)
- DOI：10.3788/YJYXS20203505.0464
  中图分类号： TP391.41
- 收稿日期：2019-09-12，
  
  修回日期：2019-10-16，
  
  录用日期：2019-10-16，
  
  纸质出版日期：2020-05-05
- 稿件说明：
移动端阅览
曾公任, 姚剑敏, 严群, 等. 基于神经网络与卡尔曼滤波的手部实时追踪方法[J]. 液晶与显示, 2020,35(5):464-470.

Gong-ren ZENG, Jian-min YAO, Qun YAN, et al. Hand real-time tracking method based on neural network and Kalman filter[J]. Chinese journal of liquid crystals and displays, 2020, 35(5): 464-470.
曾公任, 姚剑敏, 严群, 等. 基于神经网络与卡尔曼滤波的手部实时追踪方法[J]. 液晶与显示, 2020,35(5):464-470. DOI： 10.3788/YJYXS20203505.0464.

Gong-ren ZENG, Jian-min YAO, Qun YAN, et al. Hand real-time tracking method based on neural network and Kalman filter[J]. Chinese journal of liquid crystals and displays, 2020, 35(5): 464-470. DOI： 10.3788/YJYXS20203505.0464.

摘要

针对传统的手部跟踪算法存在实时性差、识别精度低、易受环境影响等问题，提出了一种基于神经网络与卡尔曼滤波的手部实时追踪方法。该方法首先通过神经网络对视频中出现的检测目标进行定位，接着用卡尔曼滤波对目标运动进行估计，将估计的结果与下一帧图像中检测到的目标进行比对；然后对检测到的目标进行跟踪，将手部运动的轨迹实时显示。实验结果表明，该方法能够对多个手部目标实时进行追踪，并在手部运动过程中出现交叉和形变的情况下还能保持跟踪，其平均处理帧数为21.212 f/s，追踪正确率为94.88%，基本满足手部跟踪的稳定可靠、高实时性、高鲁棒性等要求。

Abstract

Aiming at the problems of poor hand-time tracking

low recognition accuracy and environmental impact

a hand tracking method based on neural network and Kalman filtering is proposed. The method firstly locates the detection target appearing in the video through the neural network

then estimates the target motion by Kalman filter

compares the estimated result with the detected target in the next frame image

and finally detects the target

tracking and displaying the trajectory of the hand movement in real time. Experiments show that the method can track multiple hand targets in real time and keep tracking when the cross and deformation occur during hand movement. The average processing frame number is 21.212 f/s

and the tracking accuracy rate is 94.88%. It basically meets the requirements of stable

reliable

high real-time and high robustness of hand tracking.

关键词

Keywords

references

WANG Q, CHEN X L, ZHANG L G, et al . Viewpoint invariant sign language recognition[C]// Proceedings of 2005 IEEE International Conference on Image Processing . Genova, Italy: IEEE, 2005: 274-281.

C W NG , S RANGANATH . Real-time gesture recognition system and application . Image and Vision Computing , 2002 . 20 ( 13-14 ): 993 - 1007 . DOI: 10.1016/S0262-8856(02)00113-0 http://doi.org/10.1016/S0262-8856(02)00113-0 .

LIPTON A J, FUJIYOSHI H, PATIL R S. Moving target classification and tracking from real-time video[C]// Proceedings of the 4 th IEEE Workshop on Applications of Computer Vision . Princeton, NJ, USA, USA: IEEE, 1998: 8-14.

林开颜 , 吴军辉 , 徐立鸿 . 彩色图像分割方法综述 . 中国图象图形学报 , 2005 . 10 ( 1 ): 1 - 10 . DOI: 10.3969/j.issn.1006-8961.2005.01.001 http://doi.org/10.3969/j.issn.1006-8961.2005.01.001 .

K Y LIN , J H WU , L H XU . A survey on color image segmentation techniques . Journal of Image and Graphics , 2005 . 10 ( 1 ): 1 - 10 . DOI: 10.3969/j.issn.1006-8961.2005.01.001 http://doi.org/10.3969/j.issn.1006-8961.2005.01.001 .

吴秋红 , 吴谨 , 朱磊 , 等 . 基于图论和FCM的图像分割算法 . 液晶与显示 , 2016 . 31 ( 1 ): 112 - 117 . http://yjyxs.com/CN/abstract/abstract9966.shtml http://yjyxs.com/CN/abstract/abstract9966.shtml .

Q H WU , J WU , L ZHU , 等 . Image segmentation algorithm based on graph theory and FCM . Chinese Journal of Liquid Crystals and Displays , 2016 . 31 ( 1 ): 112 - 117 . http://yjyxs.com/CN/abstract/abstract9966.shtml http://yjyxs.com/CN/abstract/abstract9966.shtml .

P PEURSUM , S VENKATESH , G WEST . A study on smoothing for particle-filtered 3D human body tracking . International Journal of Computer Vision , 2010 . 87 ( 1-2 ): 53 - 74 . DOI: 10.1007/s11263-009-0205-5 http://doi.org/10.1007/s11263-009-0205-5 .

SHAMAIE A, SUTHERLAND A. A dynamic model for real-time tracking of hands in bimanual movements[C]// Proceedings of the 5 th International Gesture Workshop . Genova, Italy: Springer, 2004: 172-179.

REN S Q, HE K M, GIRSHICK R, et al . Faster R-CNN: towards real-time object detection with region proposal networks[C]// Proceedings of the 28 th International Conference on Neural Information Processing Systems . Montreal, Canada: MIT Press, 2015: 91-99.

REDMON J, DIVVALA S, GIRSHICK R, et al . You only look once: unified, real-time object detection[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas, NV, USA: IEEE, 2016: 779-788.

方明 , 孙腾腾 , 邵桢 . 基于改进YOLOv2的快速安全帽佩戴情况检测 . 光学精密工程 , 2019 . 27 ( 5 ): 1196 - 1205 . http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201905023 http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201905023 .

M FANG , T T SUN , Z SHAO . Fast helmet-wearing-condition detection based on improved YOLOv2 . Optics and Precision Engineering , 2019 . 27 ( 5 ): 1196 - 1205 . http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201905023 http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201905023 .

潘蓉 , 孙伟 . 基于预分割和回归的深度学习目标检测 . 光学精密工程 , 2017 . 25 ( 10s ): 221 - 227 . http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=CN201710598875.X http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=CN201710598875.X .

R PAN , W SUN . Deep learning target detection based on pre-segmentation and regression . Optics and Precision Engineering , 2017 . 25 ( 10s ): 221 - 227 . http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=CN201710598875.X http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=CN201710598875.X .

LIU W, ANGUELOV D, ERHAN D, et al . SSD: single shot MultiBox detector[C]// Proceedings of the 14th European Conference on Computer Vision . Amsterdam, The Netherlands: Springer, 2016: 21-37.

K SIMONYAN , A ZISSERMAN . Very deep convolutional networks for large-scale image recognition . ArXiv , 2014 . 1409 1556 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=10.1080/01431161.2018.1506593 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=10.1080/01431161.2018.1506593 .

YU F, KOLTUN V. Multi-scale context aggregation by dilated convolutions[J]. ArXiv, 2015, 1511.07122.

BAMBACH S, LEE S, CRANDALL D J, et al . Lending a hand: detecting hands and recognizing activities in complex egocentric interactions[C]// Proceedings of the IEEE International Conference on Computer Vision . Santiago, Chile: IEEE, 2015: 1949-1957.

ZHAO L M, LI X, ZHUANG Y T, et al . Deeply-learned part-aligned representations for person re-identification[C]// Proceedings of the IEEE International Conference on Computer Vision . Venice, Italy: IEEE, 2017: 3239-3248.

浏览量

203

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于迭代传播网络的快速鲁棒的微光图像增强

基于侧窗滤波改进的神经网络非均匀性校正算法

FRKDNet：基于知识蒸馏的特征提炼语义分割网络

面向光场显示的虚拟视点生成技术进展

基于多种眼动行为的裸眼3D显示视觉疲劳评估方法