Lightweight human pose estimation based on adaptive feature sensing

WU Ning; WANG Peng; LI Xiao-yan; LÜ Zhi-gang; SUN Meng-yu

doi:10.37188/CJLCD.2022-0351

您当前的位置：

首页 >

文章列表页 >

Lightweight human pose estimation based on adaptive feature sensing

Image Processing | 更新时间：2023-08-10

- Lightweight human pose estimation based on adaptive feature sensing
- Chinese Journal of Liquid Crystals and Displays Vol. 38, Issue 8, Pages: 1107-1117(2023)
- 作者机构：
  
  1.西安工业大学兵器科学与技术学院，陕西西安 710021
  2.西安工业大学发展规划处，陕西西安 710021
  3.西安工业大学电子信息工程学院，陕西西安 710021
  4.西安工业大学光电工程学院，陕西西安 710021
- 作者简介：
- 基金信息：
  
  National Natural Science Foundation of China（No.‍62171360）(62171360);Scientific Research Program Funded by Shaanxi Science and Technology Department（No.‍2022GY-110）;Xi'an Technological University President’s Fund Surface Cultivation Project（No.‍XGPY200217）;Xi'an Key Laboratory of Intelligent Weapons（No.‍2019220514SYS020CG042）
- DOI：10.37188/CJLCD.2022-0351
  CLC：
扫描看全文
WU Ning, WANG Peng, LI Xiao-yan, et al. Lightweight human pose estimation based on adaptive feature sensing. [J]. Chinese Journal of Liquid Crystals and Displays 38(8):1107-1117(2023)
DOI：

WU Ning, WANG Peng, LI Xiao-yan, et al. Lightweight human pose estimation based on adaptive feature sensing. [J]. Chinese Journal of Liquid Crystals and Displays 38(8):1107-1117(2023) DOI： 10.37188/CJLCD.2022-0351.

摘要

针对现有人体姿态估计网络在追求高精度检测时，网络结构设计复杂、模型参数量较大、检测效率较低的问题，本文提出了一种基于自适应特征感知的轻量级人体姿态估计算法。首先利用轻量化Ghost模块重构人体姿态估计的特征提取网络，减少网络参数量；其次设计了一种轻量级自适应特征感知的注意力机制，在降低网络模型复杂度的同时加强通道间信息有效交流，有效改善关键点定位效果；最后采用Huber Loss损失函数优化模型训练，实现异常点的更优预测，增强模型鲁棒性。在COCO数据集上进行验证，实验结果表明，与基准RMPE算法相比，改进后模型的检测精度提升了约0.5%，参数量减少了56.0%，网络运算量降低了32.6%，模型体积压缩了约57.0%，模型检测速率提升约2.1倍。本文改进后的人体姿态估计模型在压缩模型体积的同时提高了检测效率，增强了模型鲁棒性。

Abstract

For the problems of complex network structure design，large number of model parameters and low detection efficiency in the existing human pose estimation network pursues high-precision detection， this paper proposes a lightweight human pose estimation algorithm based on adaptive feature perception. Firstly， the lightweight Ghost module is used to reconstruct the feature extraction network of human pose estimation to reduce the computation amount of the network. Secondly， a lightweight adaptive feature sensing attention mechanism is designed to reduce the complexity of network model and enhance the effective communication between channels， which can improve the positioning effect of key points. Finally， Huber Loss（Exponential square loss function） is used to optimize the loss function training model to achieve better prediction of outliers and enhance the robustness of the model. Verified on the COCO dataset， the experimental results show that compared with the benchmark RMPE algorithm， the detection accuracy of the improved model is increased by about 0.5%， the number of parameters is reduced by 56.0%， the network calculation amount is reduced by 32.6%， the model volume is compressed by about 57.0%， and the model detection rate is increased by about 2.1 times. In this paper， the improved human pose estimation model improves the detection efficiency and enhances the robustness of the model while compressing the model volume.

关键词

人体姿态估计轻量化自适应特征感知Ghost模块Huber Loss

Keywords

human pose estimationlightweightadaptive feature perceptionghost moduleHuber Loss

references

马双双，王佳，曹少中，等.基于深度学习的二维人体姿态估计算法综述［J］.计算机系统应用，2022，31（10）：36-43.

MA S S， WANG J， CAO S Z， et al. Overview on two-dimensional human pose estimation methods based on deep learning ［J］. Computer Systems & Applications， 2022， 31（10）： 36-43. （in Chinese）

曾文献，马月，李伟光.轻量化二维人体骨骼关键点检测算法综述［J］.科学技术与工程，2022，22（16）：6377-6392. doi: 10.3969/j.issn.1671-1815.2022.16.002http://dx.doi.org/10.3969/j.issn.1671-1815.2022.16.002

ZENG W X， MA Y， LI W G. A survey of lightweight two-dimensional human skeleton key point detection algorithms ［J］. Science Technology and Engineering， 2022， 22（16）： 6377-6392. （in Chinese）. doi: 10.3969/j.issn.1671-1815.2022.16.002http://dx.doi.org/10.3969/j.issn.1671-1815.2022.16.002

陈仁文，袁婷婷，黄文斌，等.卷积神经网络在驾驶员姿态估计上的应用［J］.光学精密工程，2021，29（4）：813-821. doi: 10.37188/OPE.20212904.0813http://dx.doi.org/10.37188/OPE.20212904.0813

CHEN R W， YUAN T T， HUANG W B， et al. Driver pose estimation using convolutional neural networks ［J］. Optics and Precision Engineering， 2021， 29（4）： 813-821. （in Chinese）. doi: 10.37188/OPE.20212904.0813http://dx.doi.org/10.37188/OPE.20212904.0813

王洪雁，袁海.基于骨骼及表观特征融合的动作识别方法［J］.通信学报，2022，43（1）：138-148. doi: 10.11959/j.issn.1000-436x.2022020http://dx.doi.org/10.11959/j.issn.1000-436x.2022020

WANG H Y， YUAN H. Action recognition method based on fusion of skeleton and apparent features ［J］. Journal on Communications， 2022， 43（1）： 138-148. （in Chinese）. doi: 10.11959/j.issn.1000-436x.2022020http://dx.doi.org/10.11959/j.issn.1000-436x.2022020

林依林，林珊玲，林志贤.基于级联特征和图卷积的三维手部姿态估计算法［J］.液晶与显示，2022，37（6）：736-745. doi: 10.37188/cjlcd.2021-0307http://dx.doi.org/10.37188/cjlcd.2021-0307

LIN Y L， LIN S L， LIN Z X. 3D hand pose estimation algorithm based on cascaded features and graph convolution ［J］. Chinese Journal of Liquid Crystals and Displays， 2022， 37（6）： 736-745. （in Chinese）. doi: 10.37188/cjlcd.2021-0307http://dx.doi.org/10.37188/cjlcd.2021-0307

李国友，李晨光，王维江，等.基于单样本学习的多特征人体姿态模型识别研究［J］.光电工程，2021，48（2）：31-40. doi: 10.12086/oee.2021.200099http://dx.doi.org/10.12086/oee.2021.200099

LI G Y， LI C G， WANG W J， et al. Research on multi-feature human pose model recognition based on one-shot learning ［J］. Opto-Electronic Engineering， 2021， 48（2）： 31-40. （in Chinese）. doi: 10.12086/oee.2021.200099http://dx.doi.org/10.12086/oee.2021.200099

ZUO C， QIAN J M， FENG S J， et al. Deep learning in optical metrology： a review ［J］. Light： Science & Applications， 2022， 11（1）： 39. doi: 10.1038/s41377-022-00714-xhttp://dx.doi.org/10.1038/s41377-022-00714-x

LUO Y， ZHAO Y F， LI J X， et al. Computational imaging without a computer： seeing through random diffusers at the speed of light ［J］. eLight， 2022， 2（1）： 4. doi: 10.1186/s43593-022-00012-4http://dx.doi.org/10.1186/s43593-022-00012-4

SITU G H. Deep holography ［J］. Light： Advanced Manufacturing， 2022， 3（2）： 13. doi: 10.37188/lam.2022.013http://dx.doi.org/10.37188/lam.2022.013

胡凯，陈旭，朱俊，等.基于多尺度3D卷积神经网络的行为识别方法［J］.重庆邮电大学学报（自然科学版），2021，33（6）：970-976. doi: 10.3979/j.issn.1673-825X.201910240366http://dx.doi.org/10.3979/j.issn.1673-825X.201910240366

HU K， CHEN X， ZHU J， et al. Multiscale 3D convolutional neural network for action recognition ［J］. Journal of Chongqing University of Posts and Telecommunications （Natural Science Edition）， 2021， 33（6）： 970-976. （in Chinese）. doi: 10.3979/j.issn.1673-825X.201910240366http://dx.doi.org/10.3979/j.issn.1673-825X.201910240366

KREISS S， BERTONI L， ALAHI A. PifPaf： composite fields for human pose estimation ［C］//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach： IEEE， 2019： 11969-11978. doi: 10.1109/cvpr.2019.01225http://dx.doi.org/10.1109/cvpr.2019.01225

CAO Z， HIDALGO G， SIMON T， et al. OpenPose： realtime multi-person 2D pose estimation using part affinity fields ［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2021， 43（1）： 172-186. doi: 10.1109/tpami.2019.2929257http://dx.doi.org/10.1109/tpami.2019.2929257

林林，王延杰，孙海超.基于改进热图损失函数的目标6D姿态估计算法［J］.液晶与显示，2022，37（7）：913-923. doi: 10.37188/cjlcd.2021-0317http://dx.doi.org/10.37188/cjlcd.2021-0317

LIN L， WANG Y J， SUN H C. Object 6D pose estimation algorithm based on improved heatmap loss function ［J］. Chinese Journal of Liquid Crystals and Displays， 2022， 37（7）： 913-923. （in Chinese）. doi: 10.37188/cjlcd.2021-0317http://dx.doi.org/10.37188/cjlcd.2021-0317

FANG H S， XIE S Q， TAI Y W， et al. RMPE： regional multi-person pose estimation ［C］//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice： IEEE， 2017： 2353-2362. doi: 10.1109/iccv.2017.256http://dx.doi.org/10.1109/iccv.2017.256

CHEN Y L， WANG Z C， PENG Y X， et al. Cascaded pyramid network for multi-person pose estimation ［C］//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City： IEEE， 2018： 7103-7102. doi: 10.1109/cvpr.2018.00742http://dx.doi.org/10.1109/cvpr.2018.00742

SUN K， XIAO B， LIU D， et al. Deep high-resolution representation learning for human pose estimation ［C］//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach： IEEE， 2019： 5686-5696. doi: 10.1109/cvpr.2019.00584http://dx.doi.org/10.1109/cvpr.2019.00584

CHENG B W， XIAO B， WANG J D， et al. HigherHRNet： scale-aware representation learning for bottom-up human pose estimation ［C］//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle： IEEE， 2020： 5385-5394. doi: 10.1109/cvpr42600.2020.00543http://dx.doi.org/10.1109/cvpr42600.2020.00543

YU C Q， XIAO B， GAO C X， et al. Lite-HRNet： a lightweight high-resolution network ［C］//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville： IEEE， 2021： 10435-10445. doi: 10.1109/cvpr46437.2021.01030http://dx.doi.org/10.1109/cvpr46437.2021.01030

BULAT A， KOSSAIFI J， TZIMIROPOULOS G， et al. Toward fast and accurate human pose estimation via soft-gated skip connections ［C］//Proceedings of the 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition. Buenos Aires： IEEE， 2020： 8-15. doi: 10.1109/fg47880.2020.00014http://dx.doi.org/10.1109/fg47880.2020.00014

HAN K， WANG Y H， TIAN Q， et al. GhostNet： more features from cheap operations ［C］//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle： IEEE， 2020： 1577-1586. doi: 10.1109/cvpr42600.2020.00165http://dx.doi.org/10.1109/cvpr42600.2020.00165

SUN Q， ZHOU W X， FAN J Q. Adaptive Huber regression ［J］. Journal of the American Statistical Association， 2020， 115（529）： 254-265. doi: 10.1080/01621459.2018.1543124http://dx.doi.org/10.1080/01621459.2018.1543124

WANG Q L， WU B G， ZHU P F， et al. ECA-Net： efficient channel attention for deep convolutional neural networks ［C］//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle： IEEE， 2020： 11531-11539. doi: 10.1109/cvpr42600.2020.01155http://dx.doi.org/10.1109/cvpr42600.2020.01155

HU J， SHEN L， SUN G. Squeeze-and-excitation networks ［C］//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City： IEEE， 2018： 7132-7141. doi: 10.1109/cvpr.2018.00745http://dx.doi.org/10.1109/cvpr.2018.00745

XIAO B， WU H P， WEI Y C. Simple baselines for human pose estimation and tracking ［C］//Proceedings of the 15th European Conference on Computer Vision. Munich： Springer， 2018： 472-487. doi: 10.1007/978-3-030-01231-1_29http://dx.doi.org/10.1007/978-3-030-01231-1_29

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Lightweight and high-precision object detection algorithm based on YOLO framework

Improved lightweight human pose estimation algorithm

Mobile-based portrait segmentation network with attention mechanism

NCA-MobileNet： a lightweight facial expression recognition method

Related Author

No data

Related Institution

School of Information Engineering， Southwest University of Science and Technology

Engineering Research Center for Metallurgical Automation and Detecting Technology of Ministry of Education， Wuhan University of Science and Technology

School of Information Science and Engineering ， Wuhan University of Science and Technology

Jinjiang RichSense Electronic Technology Co., Ltd., Jinjiang

Nation & Local United Engineering Laboratory of Flat Panel DisplayTechnology, College of Physics and Information Engineering, Fuzhou University

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China 130033 Postal code：130033
Tel：0431-86176059 Email：yjxs@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰