基于深度卷积神经网络的多特征融合的手势识别

贠卫国; 史其琦; 王民

doi:10.3788/YJYXS20193404.0417

您当前的位置：

首页 >

文章列表页 >

基于深度卷积神经网络的多特征融合的手势识别

图像处理 | 更新时间：2020-07-05

- 基于深度卷积神经网络的多特征融合的手势识别
- Multi-feature fusion gesture recognition based ondeep convolutional neural network
- 液晶与显示 2019年34卷第4期页码：417-422
- 作者机构：
  
  西安建筑科技大学信息与控制工程学院, 陕西西安 710055
- 作者简介：
  
  [ "贠卫国(1962-), 男, 山西运城人, 硕士, 教授, 硕士生导师, 1988年于西安建筑科技大学获得硕士学位, 主要从事智能识别与控制方面的研究。E-mail:1372245990@qq.com" ]
  [ "史其琦(1993-), 男, 陕西咸阳人, 硕士研究生, 2016年于西安建筑科技大学获得学士学位, 主要从事智能信息处理相关技术的研究。E-mail:sqq187000978170@163.com" ]
- 基金信息：
  
  国家自然科学基金(61373112);住房城乡建设部科学技术项目计划(2016-R2-045);陕西省自然科学基础研究资金(2014JM8348);Natural Science Foundation of China(61373112);Housing and Urbsn Ministry of Science project(2016-R2-045);Fundamental Research Fund of Natural Science of Shaanxi Province(2014JM8348)
- DOI：10.3788/YJYXS20193404.0417
  中图分类号： TP391.41
- 收稿日期：2018-10-16，
  
  录用日期：2019-3-7，
  
  纸质出版日期：2019-04
- 稿件说明：
移动端阅览
贠卫国, 史其琦, 王民. 基于深度卷积神经网络的多特征融合的手势识别[J]. 液晶与显示, 2019,34(4):417-422.

Wei-guo YUN, Qi-qi SHI, Min WANG. Multi-feature fusion gesture recognition based ondeep convolutional neural network[J]. Chinese journal of liquid crystals and displays, 2019, 34(4): 417-422.
贠卫国, 史其琦, 王民. 基于深度卷积神经网络的多特征融合的手势识别[J]. 液晶与显示, 2019,34(4):417-422. DOI： 10.3788/YJYXS20193404.0417.

Wei-guo YUN, Qi-qi SHI, Min WANG. Multi-feature fusion gesture recognition based ondeep convolutional neural network[J]. Chinese journal of liquid crystals and displays, 2019, 34(4): 417-422. DOI： 10.3788/YJYXS20193404.0417.

摘要

针对传统的分类方法由于提取的特征比较单一或者分类器结构过于简单，导致手语识别率较低的问题，本文将深度卷积神经网络架构作为分类器与多特征融合算法进行结合，通过使用纹理特征结合形状特征做到有效识别。首先纹理特征通过LBP、卷积神经网络和灰度共生矩阵方法得到，其中形状特征向量由Hu氏不变量和傅里叶级数组成。为了避免过拟合现象，使用"dropout"方法训练深度卷积神经网络。这种基于深度卷积神经网络的多特征融合的手语识别方法，在"hand"数据库中，对32种势的识别率为97.73%。相比一般的手语识别方法，此方法鲁棒性更强，并且识别率更高。

Abstract

To solve the problem that the traditional classification method has a relatively single extraction feature or the structure of the classifier is too simple

leading to a low recognition rate of the sign language

in this paper

the deep convolutional neural network architecture was combined as a classifier and a multi-feature fusion algorithm

and it was effectively identified by using texture features and shape features. Firstly

the texture features were obtained by LBP

convolutional neural network and gray-level co-occurrence matrix methods

in which the shape feature vector consisted of Hu's invariant and Fourier descriptors. To avoid overfitting

the "dropout" method was used to train deep convolutional neural networks. This kind of multi-feature fusion sign language recognition method based on deep convolutional neural network has a recognition rate of 97.73% for 32 gestures in the "hand" database. Compared with the general sign language recognition method

this method is more robust and has a higher recognition rate.

关键词

Keywords

references

王民, 石新源, 王稚慧, 等.基于Vibe和时空上下文的运动手势跟踪算法[J].液晶与显示, 2018, 33(1):92-98.

WANG M, SHI X Y, WANG Z H, et al . Moving gesture tracking algorithm based on Vibe and space-time context[J]. Chinese Journal of Liquid Crystals and Displays, 2018, 33(1):92-98. (in Chinese)

刘淑琴.图像特征提取方法及其应用研究[D].西安: 西北大学, 2016.

LIU S Q. Research on Image feature extraction and its application[D]. Xi'an: Northwest University, 2016. (in Chinese)

李丹娇, 彭进业, 冯晓毅, 等.结合CSS与傅里叶描述子的手势特征提取[J].计算机工程, 2012, 38(6):178-180.

LI D J, PENG J Y, FENG X Y, et al . Gesture feature extraction combining CSS with fourier descriptor[J]. Computer Engineering, 2012, 38(6):178-180. (in Chinese)

刘念, 阚江明.基于多特征融合和深度信念网络的植物叶片识别[J].北京林业大学学报, 2016, 38(3):110-119.

LIU N, KUANG J M. Plant leafidentification based on the multi-feature fusion and deep belief networks method[J]. Journal of Beijing Forestry University, 2016, 38(3):110-119. (in Chinese)

高良, 闫民, 赵方.基于多特征融合的植物叶片识别研究[J].浙江农业学报, 2017, 29(4):668-675.

GAO L, YAN M, ZHAO F. Plant leaf recognition based on fusion of multiple features[J].Acta Agriculturae Zhejiangensis, 2017, 29(4):668-675. (in Chinese)

尚小晶, 田彦涛, 李阳, 等.基于改进概率神经网络的手势动作识别[J].吉林大学学报(信息科学版), 2010, 28(5):459-466.

SHANG X J, TIAN Y T, LI Y, et al . Recognition of gestures and movements based on MPNN[J]. Journal of Jilin University (Information Science Edition), 2010, 28(5):459-466.

吴健健, 陈玮.基于多特征融合与支持向量机的手势识别[J].电子科技, 2015, 28(5):127-131.

WU J J, CHEN H. Hand gesture recognition based on multi-feature fusion and support vector machines[J]. Electronic Science and Technology, 2015, 28(5):127-131. (in Chinese)

刘念.基于深度信念网络的植物叶片识别研究[D].北京: 北京林业大学, 2016.

LIU N. The study of plant leaf recognition based on the deep belief networks[D]. Beijing: Beijing Forestry University, 2016. (in Chinese)

黄非非.基于LBP的人脸识别研究[D].重庆: 重庆大学, 2009.

HUANG F F. Research on face recognition based on LBP operator[D]. Chongqing: Chongqing University, 2009. (in Chinese)

李彦冬, 郝宗波, 雷航.卷积神经网络研究综述[J].计算机应用, 2016, 36(9):2508-2515, 2565.

LI Y D, HAO Z B, LEI H.Survey of convolutional neural network[J]. Journal of Computer Applications, 2016, 36(9):2508-2515, 2565. (in Chinese)

WANG X F, HUANG D S, DU J X, et al . Classification of plant leaf images with complicated background[J]. Applied Mathematics and Computation, 2008, 205(2):916-926.

高程程, 惠晓威.基于灰度共生矩阵的纹理特征提取[J].计算机系统应用, 2010, 19(6):195-198.

GAO C C, HUI X W. GLCM-based texture feature extraction[J]. Computer Systems & Applications, 2010, 19(6):195-198. (in Chinese)

晋杰.基于多特征融合的服装图片检索关键技术研究[D].昆明: 昆明理工大学, 2017.

JIN J. Research on key techniques of garment image retrieval based on multi-feature fusion[D]. Kunming: Kunming University of Science and Technology, 2017. (in Chinese)

卢宏涛, 张秦川.深度卷积神经网络在计算机视觉中的应用研究综述[J].数据采集与处理, 2016, 31(1):1-17.

LU H T, ZHANG Q C. Applications of deep convolutional neural network in computer vision[J].Journal of Data Acquisition and Processing, 2016, 31(1):1-17. (in Chinese)

LE CUN Y, BOSER B, DENKER J S, et al . Handwritten digit recognition with a back-propagation network[M]//TOURETZKY D. Advances in Neural Information Processing Systems . San Francisco, CA, USA: Morgan Kaufmann Publishers, 1990: 396-404.

吴尧, 邱卫根.基于改进的深度卷积神经网络的人脸识别[J].计算机工程与设计, 2017, 38(8):2246-2250.

WU Y, QIU W G. Face recognitionalgorithm based on improved deep convolutional neural network[J]. Computer Engineering and Design, 2017, 38(8):2246-2250. (in Chinese)

王雅.基于改进的HMM和自适应技术的非特定人手语识别的研究[D].西安: 西安建筑科技大学, 2017.

WANG Y. Research on signer-independent sign language recognition based on improved HMM and adaptive technology[D]. Xi'an: Xi'an University of Architecture and Technology, 2017. (in Chinese)

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于级联特征和图卷积的三维手部姿态估计算法

一种多特征融合的目标跟踪算法

自由立体显示中基于深度卷积神经网络的虚拟视点生成方法

融合高低层多特征的显著性检测算法

基于Vibe和时空上下文的运动手势跟踪算法