融合注意力机制的移动端人像分割网络

周鹏; 姚剑敏; 林志贤; 严群; 郭太良

doi:10.3788/YJYXS20203506.0547

您当前的位置：

首页 >

文章列表页 >

融合注意力机制的移动端人像分割网络

图像处理 | 更新时间：2020-07-13

- 融合注意力机制的移动端人像分割网络
- Mobile-based portrait segmentation network with attention mechanism
- 液晶与显示 2020年35卷第6期页码：547-554
- 作者机构：
  
  1.平板显示技术国家地方联合工程实验室, 福州大学物理与信息工程学院, 福建福州 350108
  2.晋江市博感电子科技有限公司, 福建晋江 362200
- 作者简介：
  
  [ "周鹏(1994-), 男, 安徽芜湖人, 硕士研究生, 2017年于武夷学院获得学士学位, 主要从事计算机视觉、深度学习等方面的研究。E-mail:946375455@qq.com" ]
  [ "姚剑敏(1978-), 男, 福建莆田人, 博士, 副研究员, 2005年于中科院长春光学精密机械与物理研究所获得博士学位, 主要从事人工智能、图像处理、信息显示技术等方面的研究。E-mail:yaojm@fzu.edu.cn" ]
- 基金信息：
  
  国家重点研发计划资助(2016YFB0401503);广东省科技重大专项(2016B090906001);福建省科技重大专项(2014HZ0003-1);广东省光信息材料与技术重点实验室开放基金资助项目(2017B030301007)
- DOI：10.3788/YJYXS20203506.0547
  中图分类号： TP391.41
- 收稿日期：2019-09-12，
  
  修回日期：2019-10-17，
  
  录用日期：2019-10-17，
  
  纸质出版日期：2020-06-05
- 稿件说明：
移动端阅览
周鹏, 姚剑敏, 林志贤, 等. 融合注意力机制的移动端人像分割网络[J]. 液晶与显示, 2020,35(6):547-554.

Peng ZHOU, Jian-min YAO, Zhi-xian LIN, et al. Mobile-based portrait segmentation network with attention mechanism[J]. Chinese journal of liquid crystals and displays, 2020, 35(6): 547-554.
周鹏, 姚剑敏, 林志贤, 等. 融合注意力机制的移动端人像分割网络[J]. 液晶与显示, 2020,35(6):547-554. DOI： 10.3788/YJYXS20203506.0547.

Peng ZHOU, Jian-min YAO, Zhi-xian LIN, et al. Mobile-based portrait segmentation network with attention mechanism[J]. Chinese journal of liquid crystals and displays, 2020, 35(6): 547-554. DOI： 10.3788/YJYXS20203506.0547.

摘要

现有的移动端人像分割网络存在分割精度差、分割边缘模糊等问题。为此，提出了一种融合注意力机制的轻量化人像分割网络。首先，利用MobileNetV2网络提取图像特征。然后对注意力模块NLNet（Non-local neural networks）进行轻量化处理，随后将优化过的注意力模块嵌入到四层解码网络中。利用融合注意力机制的解码网络自适应地学习有效特征，最后通过SoftMax层得到人像分割结果图。同时改进了损失函数，引入多损失函数（Multi-Loss），使网络更容易收敛。解码网络融合注意力机制的方式使得轻量化网络可以在语义分割任务上取得较好的效果。实验结果表明，模型在550张自采集的人像测试集上达到了92.29%的交并比（MeanIOU），单张图片在Inter（R）Core i5 CPU上的分割时间为0.74 s。与传统的人像分割网络相比，研究网络的分割精度和分割速度优势明显，适合应用于移动端设备。

Abstract

Existing mobile terminal segmentation network has problems such as poor segmentation accuracy and segmentation edge blurring. To this end

a lightweight portrait segmentation network that incorporates attention mechanisms is proposed. Firstly

the image features are extracted by MobileNetV2 network. Then

the attention module NLNet (Non-local neural networks) is lightened

and then the optimized attention module is embedded in the four-layer decoding network. The decoding network with attention mechanism adaptively learns the effective features

and finally obtains the portrait segmentation result map through SoftMax. At the same time

the loss function is improved

and Multi-Loss (multi-loss function) is introduced to make the network easier to converge. The way of attention mechanisms fused in decoding networks makes the lightweight network achieve better results in the semantic segmentation task. Experimental results show that the model achieves 92.29% MeanIOU on the 550 self-collected portrait test set

and the split time of the single picture on the Inter(R) Core i5 CPU is 0.74 s. Compared with the traditional portrait segmentation network

the research network has obvious advantages in segmentation precision and segmentation speed

and is suitable for mobile devices.

关键词

Keywords

references

张健 , 李宏升 . 基于图论阈值算法的图像分割研究 . 液晶与显示 , 2014 . 29 ( 4 ): 592 - 597 . http://d.old.wanfangdata.com.cn/Periodical/yjyxs201404021 http://d.old.wanfangdata.com.cn/Periodical/yjyxs201404021 .

J ZHANG , H S LI . Image mosaic research based on wavelet and rough set algorithm . Chinese Journal of Liquid Crystals and Displays , 2014 . 29 ( 4 ): 592 - 597 . http://d.old.wanfangdata.com.cn/Periodical/yjyxs201404021 http://d.old.wanfangdata.com.cn/Periodical/yjyxs201404021 .

孙志军 , 薛磊 , 许阳明 , 等 . 深度学习研究综述 . 计算机应用研究 , 2012 . 29 ( 8 ): 2806 - 2810 . DOI: 10.3969/j.issn.1001-3695.2012.08.002 http://doi.org/10.3969/j.issn.1001-3695.2012.08.002 .

Z J SUN , L XUE , Y M XU . Overview of deep learning . Application Research of Computers , 2012 . 29 ( 8 ): 2806 - 2810 . DOI: 10.3969/j.issn.1001-3695.2012.08.002 http://doi.org/10.3969/j.issn.1001-3695.2012.08.002 .

周筑博 , 高佼 , 张巍 , 等 . 基于深度卷积神经网络的输电线路可见光图像目标检测 . 液晶与显示 , 2018 . 33 ( 4 ): 317 - 325 . http://d.old.wanfangdata.com.cn/Periodical/yjyxs201804008 http://d.old.wanfangdata.com.cn/Periodical/yjyxs201804008 .

Z B ZHOU , J GAO , W ZHANG , 等 . Object detection of transmission line visual images based on deep convolutional neural network . Chinese Journal of Liquid Crystals and Displays , 2018 . 33 ( 4 ): 317 - 325 . http://d.old.wanfangdata.com.cn/Periodical/yjyxs201804008 http://d.old.wanfangdata.com.cn/Periodical/yjyxs201804008 .

梁华 , 宋玉龙 , 钱锋 , 等 . 基于深度学习的航空对地小目标检测 . 液晶与显示 , 2018 . 33 ( 9 ): 793 - 800 . http://d.old.wanfangdata.com.cn/Periodical/yjyxs201809011 http://d.old.wanfangdata.com.cn/Periodical/yjyxs201809011 .

H LIANG , Y L SONG , F QIAN , 等 . Detection of small target in aerial photography based on deep learning . Chinese Journal of Liquid Crystals and Displays , 2018 . 33 ( 9 ): 793 - 800 . http://d.old.wanfangdata.com.cn/Periodical/yjyxs201809011 http://d.old.wanfangdata.com.cn/Periodical/yjyxs201809011 .

C Farabet , C Couprie , L Najman , 等 . Learning hierarchical features for scene labeling . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2012 . 35 ( 8 ): 1915 - 1929 . http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=5cc7b6a367461d67d312159bdfb15f7c http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=5cc7b6a367461d67d312159bdfb15f7c .

LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Boston, MA, USA: IEEE, 2015: 3431-3440.

CHEN L C, PAPANDREOU G, SCHROFF F, et al . Rethinking atrous convolution for semantic image segmentation[J]. ArXiv , 2017: 1706.05587.

ZHAO H S, SHI J P, QI X J, et al . Pyramid scene parsing network[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Honolulu, HI, USA: IEEE, 2017: 6230-6239.

LIN G S, MILAN A, SHEN C H, et al . RefineNet: multi-path refinement networks for high-resolution semantic segmentation[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Honolulu, HI, USA: IEEE, 2017: 5168-5177.

SANDLER M, HOWARD A, ZHU M L, et al . MobileNetV2: inverted residuals and linear bottlenecks[C]// Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City, UT, USA: IEEE, 2018: 4510-4520.

V BADRINARAYANAN , A KENDALL , R CIPOLLA . SegNet:a deep convolutional encoder-decoder architecture for image segmentation . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2017 . 39 ( 12 ): 2481 - 2495 . DOI: 10.1109/TPAMI.2016.2644615 http://doi.org/10.1109/TPAMI.2016.2644615 .

RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C] // Proceedings of the 18th International Conference on Medical Image Computing and Computer-assisted Intervention . Munich, Germany: Springer, 2015: 234-241.

ZHANG Y L, LI K P, LI K, et al . Image super-resolution using very deep residual channel attention networks[C]// Proceedings of the 15th European Conference on Computer Vision . Munich, Germany: Springer, 2018: 294-310.

WANG X L, GIRSHICK R, GUPTA A, et al . Non-local neural networks[C]// Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City, UT, USA: IEEE, 2018: 7794-7803.

HOWARD A G, ZHU M L, CHEN B, et al . MobileNets: Efficient convolutional neural networks for mobile vision applications[J]. ArXiv , 2017: 1704.04861.

浏览量

177

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

融合渐进式策略的轻量化特征点提取与匹配方法

残差学习与层注意力相结合的轻量级图像超分辨

NCA-MobileNet：一种轻量化人脸表情识别方法