Multi-feature fusion gesture recognition based ondeep convolutional neural network

Wei-guo YUN; Qi-qi SHI; Min WANG

doi:10.3788/YJYXS20193404.0417

您当前的位置：

首页 >

文章列表页 >

Multi-feature fusion gesture recognition based ondeep convolutional neural network

Image Processing | 更新时间：2020-07-05

- Multi-feature fusion gesture recognition based ondeep convolutional neural network
- Chinese Journal of Liquid Crystals and Displays Vol. 34, Issue 4, Pages: 417-422(2019)
- 作者机构：
  
  西安建筑科技大学信息与控制工程学院, 陕西西安 710055
- 作者简介：
- 基金信息：
- DOI：10.3788/YJYXS20193404.0417
  CLC： TP391.41
- Received：16 October 2018，
  
  Accepted：07 March 2019，
  
  Published：2019-04
- 稿件说明：
移动端阅览
Wei-guo YUN, Qi-qi SHI, Min WANG. Multi-feature fusion gesture recognition based ondeep convolutional neural network[J]. Chinese journal of liquid crystals and displays, 2019, 34(4): 417-422.
DOI：

Wei-guo YUN, Qi-qi SHI, Min WANG. Multi-feature fusion gesture recognition based ondeep convolutional neural network[J]. Chinese journal of liquid crystals and displays, 2019, 34(4): 417-422. DOI： 10.3788/YJYXS20193404.0417.

摘要

针对传统的分类方法由于提取的特征比较单一或者分类器结构过于简单，导致手语识别率较低的问题，本文将深度卷积神经网络架构作为分类器与多特征融合算法进行结合，通过使用纹理特征结合形状特征做到有效识别。首先纹理特征通过LBP、卷积神经网络和灰度共生矩阵方法得到，其中形状特征向量由Hu氏不变量和傅里叶级数组成。为了避免过拟合现象，使用"dropout"方法训练深度卷积神经网络。这种基于深度卷积神经网络的多特征融合的手语识别方法，在"hand"数据库中，对32种势的识别率为97.73%。相比一般的手语识别方法，此方法鲁棒性更强，并且识别率更高。

Abstract

To solve the problem that the traditional classification method has a relatively single extraction feature or the structure of the classifier is too simple

leading to a low recognition rate of the sign language

in this paper

the deep convolutional neural network architecture was combined as a classifier and a multi-feature fusion algorithm

and it was effectively identified by using texture features and shape features. Firstly

the texture features were obtained by LBP

convolutional neural network and gray-level co-occurrence matrix methods

in which the shape feature vector consisted of Hu's invariant and Fourier descriptors. To avoid overfitting

the "dropout" method was used to train deep convolutional neural networks. This kind of multi-feature fusion sign language recognition method based on deep convolutional neural network has a recognition rate of 97.73% for 32 gestures in the "hand" database. Compared with the general sign language recognition method

this method is more robust and has a higher recognition rate.

关键词

Keywords

references

王民, 石新源, 王稚慧, 等.基于Vibe和时空上下文的运动手势跟踪算法[J].液晶与显示, 2018, 33(1):92-98.

WANG M, SHI X Y, WANG Z H, et al . Moving gesture tracking algorithm based on Vibe and space-time context[J]. Chinese Journal of Liquid Crystals and Displays, 2018, 33(1):92-98. (in Chinese)

刘淑琴.图像特征提取方法及其应用研究[D].西安: 西北大学, 2016.

LIU S Q. Research on Image feature extraction and its application[D]. Xi'an: Northwest University, 2016. (in Chinese)

李丹娇, 彭进业, 冯晓毅, 等.结合CSS与傅里叶描述子的手势特征提取[J].计算机工程, 2012, 38(6):178-180.

LI D J, PENG J Y, FENG X Y, et al . Gesture feature extraction combining CSS with fourier descriptor[J]. Computer Engineering, 2012, 38(6):178-180. (in Chinese)

刘念, 阚江明.基于多特征融合和深度信念网络的植物叶片识别[J].北京林业大学学报, 2016, 38(3):110-119.

LIU N, KUANG J M. Plant leafidentification based on the multi-feature fusion and deep belief networks method[J]. Journal of Beijing Forestry University, 2016, 38(3):110-119. (in Chinese)

高良, 闫民, 赵方.基于多特征融合的植物叶片识别研究[J].浙江农业学报, 2017, 29(4):668-675.

GAO L, YAN M, ZHAO F. Plant leaf recognition based on fusion of multiple features[J].Acta Agriculturae Zhejiangensis, 2017, 29(4):668-675. (in Chinese)

尚小晶, 田彦涛, 李阳, 等.基于改进概率神经网络的手势动作识别[J].吉林大学学报(信息科学版), 2010, 28(5):459-466.

SHANG X J, TIAN Y T, LI Y, et al . Recognition of gestures and movements based on MPNN[J]. Journal of Jilin University (Information Science Edition), 2010, 28(5):459-466.

吴健健, 陈玮.基于多特征融合与支持向量机的手势识别[J].电子科技, 2015, 28(5):127-131.

WU J J, CHEN H. Hand gesture recognition based on multi-feature fusion and support vector machines[J]. Electronic Science and Technology, 2015, 28(5):127-131. (in Chinese)

刘念.基于深度信念网络的植物叶片识别研究[D].北京: 北京林业大学, 2016.

LIU N. The study of plant leaf recognition based on the deep belief networks[D]. Beijing: Beijing Forestry University, 2016. (in Chinese)

黄非非.基于LBP的人脸识别研究[D].重庆: 重庆大学, 2009.

HUANG F F. Research on face recognition based on LBP operator[D]. Chongqing: Chongqing University, 2009. (in Chinese)

李彦冬, 郝宗波, 雷航.卷积神经网络研究综述[J].计算机应用, 2016, 36(9):2508-2515, 2565.

LI Y D, HAO Z B, LEI H.Survey of convolutional neural network[J]. Journal of Computer Applications, 2016, 36(9):2508-2515, 2565. (in Chinese)

WANG X F, HUANG D S, DU J X, et al . Classification of plant leaf images with complicated background[J]. Applied Mathematics and Computation, 2008, 205(2):916-926.

高程程, 惠晓威.基于灰度共生矩阵的纹理特征提取[J].计算机系统应用, 2010, 19(6):195-198.

GAO C C, HUI X W. GLCM-based texture feature extraction[J]. Computer Systems & Applications, 2010, 19(6):195-198. (in Chinese)

晋杰.基于多特征融合的服装图片检索关键技术研究[D].昆明: 昆明理工大学, 2017.

JIN J. Research on key techniques of garment image retrieval based on multi-feature fusion[D]. Kunming: Kunming University of Science and Technology, 2017. (in Chinese)

卢宏涛, 张秦川.深度卷积神经网络在计算机视觉中的应用研究综述[J].数据采集与处理, 2016, 31(1):1-17.

LU H T, ZHANG Q C. Applications of deep convolutional neural network in computer vision[J].Journal of Data Acquisition and Processing, 2016, 31(1):1-17. (in Chinese)

LE CUN Y, BOSER B, DENKER J S, et al . Handwritten digit recognition with a back-propagation network[M]//TOURETZKY D. Advances in Neural Information Processing Systems . San Francisco, CA, USA: Morgan Kaufmann Publishers, 1990: 396-404.

吴尧, 邱卫根.基于改进的深度卷积神经网络的人脸识别[J].计算机工程与设计, 2017, 38(8):2246-2250.

WU Y, QIU W G. Face recognitionalgorithm based on improved deep convolutional neural network[J]. Computer Engineering and Design, 2017, 38(8):2246-2250. (in Chinese)

王雅.基于改进的HMM和自适应技术的非特定人手语识别的研究[D].西安: 西安建筑科技大学, 2017.

WANG Y. Research on signer-independent sign language recognition based on improved HMM and adaptive technology[D]. Xi'an: Xi'an University of Architecture and Technology, 2017. (in Chinese)

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

3D hand pose estimation algorithm based on cascaded features and graph convolution

Multi-feature fusion target tracking algorithm

Virtual viewpoint image generation method based on deep convolutional neural network in autostereoscopic display

Saliency detection algorithm integrating multiple features of high and low level

Moving gesture tracking algorithm based on vibe and space-time context

Related Author

Zhi-xian LIN

Shan-ling LIN

Yi-lin LIN

Wei-feng ZHANG

Wei-zhen JIA

Qiu-sheng HE

Hui-hui LIANG

Qiong-hua WANG

Related Institution

College of Physics and Information Engineering， Fuzhou University

Fujian Science & Technology Innovation Laboratory for Optoelectronic Information of China

School of Advanced Manufacturing， Fuzhou University

School of Electronic Information Engineering, Taiyuan University of Science and Technology

School of Electronics and Information Engineering, Sichuan University

AI问答

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China 130033 Postal code：130033
Tel：0431-86176059 Email：yjxs@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰