Mobile-based portrait segmentation network with attention mechanism

Peng ZHOU; Jian-min YAO; Zhi-xian LIN; Qun YAN; Tai-liang GUO

doi:10.3788/YJYXS20203506.0547

您当前的位置：

首页 >

文章列表页 >

Mobile-based portrait segmentation network with attention mechanism

Image Processing | 更新时间：2020-07-13

- Mobile-based portrait segmentation network with attention mechanism
- Chinese Journal of Liquid Crystals and Displays Vol. 35, Issue 6, Pages: 547-554(2020)
- 作者机构：
  
  1.平板显示技术国家地方联合工程实验室, 福州大学物理与信息工程学院, 福建福州 350108
  2.晋江市博感电子科技有限公司, 福建晋江 362200
- 作者简介：
- 基金信息：
- DOI：10.3788/YJYXS20203506.0547
  CLC： TP391.41
- Received：12 September 2019，
  
  Revised：17 October 2019，
  
  Accepted：17 October 2019，
  
  Published：05 June 2020
- 稿件说明：
移动端阅览
Peng ZHOU, Jian-min YAO, Zhi-xian LIN, et al. Mobile-based portrait segmentation network with attention mechanism[J]. Chinese journal of liquid crystals and displays, 2020, 35(6): 547-554.
DOI：

Peng ZHOU, Jian-min YAO, Zhi-xian LIN, et al. Mobile-based portrait segmentation network with attention mechanism[J]. Chinese journal of liquid crystals and displays, 2020, 35(6): 547-554. DOI： 10.3788/YJYXS20203506.0547.

摘要

现有的移动端人像分割网络存在分割精度差、分割边缘模糊等问题。为此，提出了一种融合注意力机制的轻量化人像分割网络。首先，利用MobileNetV2网络提取图像特征。然后对注意力模块NLNet（Non-local neural networks）进行轻量化处理，随后将优化过的注意力模块嵌入到四层解码网络中。利用融合注意力机制的解码网络自适应地学习有效特征，最后通过SoftMax层得到人像分割结果图。同时改进了损失函数，引入多损失函数（Multi-Loss），使网络更容易收敛。解码网络融合注意力机制的方式使得轻量化网络可以在语义分割任务上取得较好的效果。实验结果表明，模型在550张自采集的人像测试集上达到了92.29%的交并比（MeanIOU），单张图片在Inter（R）Core i5 CPU上的分割时间为0.74 s。与传统的人像分割网络相比，研究网络的分割精度和分割速度优势明显，适合应用于移动端设备。

Abstract

Existing mobile terminal segmentation network has problems such as poor segmentation accuracy and segmentation edge blurring. To this end

a lightweight portrait segmentation network that incorporates attention mechanisms is proposed. Firstly

the image features are extracted by MobileNetV2 network. Then

the attention module NLNet (Non-local neural networks) is lightened

and then the optimized attention module is embedded in the four-layer decoding network. The decoding network with attention mechanism adaptively learns the effective features

and finally obtains the portrait segmentation result map through SoftMax. At the same time

the loss function is improved

and Multi-Loss (multi-loss function) is introduced to make the network easier to converge. The way of attention mechanisms fused in decoding networks makes the lightweight network achieve better results in the semantic segmentation task. Experimental results show that the model achieves 92.29% MeanIOU on the 550 self-collected portrait test set

and the split time of the single picture on the Inter(R) Core i5 CPU is 0.74 s. Compared with the traditional portrait segmentation network

the research network has obvious advantages in segmentation precision and segmentation speed

and is suitable for mobile devices.

关键词

Keywords

references

张健 , 李宏升 . 基于图论阈值算法的图像分割研究 . 液晶与显示 , 2014 . 29 ( 4 ): 592 - 597 . http://d.old.wanfangdata.com.cn/Periodical/yjyxs201404021 http://d.old.wanfangdata.com.cn/Periodical/yjyxs201404021 .

J ZHANG , H S LI . Image mosaic research based on wavelet and rough set algorithm . Chinese Journal of Liquid Crystals and Displays , 2014 . 29 ( 4 ): 592 - 597 . http://d.old.wanfangdata.com.cn/Periodical/yjyxs201404021 http://d.old.wanfangdata.com.cn/Periodical/yjyxs201404021 .

孙志军 , 薛磊 , 许阳明 , 等 . 深度学习研究综述 . 计算机应用研究 , 2012 . 29 ( 8 ): 2806 - 2810 . DOI: 10.3969/j.issn.1001-3695.2012.08.002 http://doi.org/10.3969/j.issn.1001-3695.2012.08.002 .

Z J SUN , L XUE , Y M XU . Overview of deep learning . Application Research of Computers , 2012 . 29 ( 8 ): 2806 - 2810 . DOI: 10.3969/j.issn.1001-3695.2012.08.002 http://doi.org/10.3969/j.issn.1001-3695.2012.08.002 .

周筑博 , 高佼 , 张巍 , 等 . 基于深度卷积神经网络的输电线路可见光图像目标检测 . 液晶与显示 , 2018 . 33 ( 4 ): 317 - 325 . http://d.old.wanfangdata.com.cn/Periodical/yjyxs201804008 http://d.old.wanfangdata.com.cn/Periodical/yjyxs201804008 .

Z B ZHOU , J GAO , W ZHANG , 等 . Object detection of transmission line visual images based on deep convolutional neural network . Chinese Journal of Liquid Crystals and Displays , 2018 . 33 ( 4 ): 317 - 325 . http://d.old.wanfangdata.com.cn/Periodical/yjyxs201804008 http://d.old.wanfangdata.com.cn/Periodical/yjyxs201804008 .

梁华 , 宋玉龙 , 钱锋 , 等 . 基于深度学习的航空对地小目标检测 . 液晶与显示 , 2018 . 33 ( 9 ): 793 - 800 . http://d.old.wanfangdata.com.cn/Periodical/yjyxs201809011 http://d.old.wanfangdata.com.cn/Periodical/yjyxs201809011 .

H LIANG , Y L SONG , F QIAN , 等 . Detection of small target in aerial photography based on deep learning . Chinese Journal of Liquid Crystals and Displays , 2018 . 33 ( 9 ): 793 - 800 . http://d.old.wanfangdata.com.cn/Periodical/yjyxs201809011 http://d.old.wanfangdata.com.cn/Periodical/yjyxs201809011 .

C Farabet , C Couprie , L Najman , 等 . Learning hierarchical features for scene labeling . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2012 . 35 ( 8 ): 1915 - 1929 . http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=5cc7b6a367461d67d312159bdfb15f7c http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=5cc7b6a367461d67d312159bdfb15f7c .

LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Boston, MA, USA: IEEE, 2015: 3431-3440.

CHEN L C, PAPANDREOU G, SCHROFF F, et al . Rethinking atrous convolution for semantic image segmentation[J]. ArXiv , 2017: 1706.05587.

ZHAO H S, SHI J P, QI X J, et al . Pyramid scene parsing network[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Honolulu, HI, USA: IEEE, 2017: 6230-6239.

LIN G S, MILAN A, SHEN C H, et al . RefineNet: multi-path refinement networks for high-resolution semantic segmentation[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . Honolulu, HI, USA: IEEE, 2017: 5168-5177.

SANDLER M, HOWARD A, ZHU M L, et al . MobileNetV2: inverted residuals and linear bottlenecks[C]// Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City, UT, USA: IEEE, 2018: 4510-4520.

V BADRINARAYANAN , A KENDALL , R CIPOLLA . SegNet:a deep convolutional encoder-decoder architecture for image segmentation . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2017 . 39 ( 12 ): 2481 - 2495 . DOI: 10.1109/TPAMI.2016.2644615 http://doi.org/10.1109/TPAMI.2016.2644615 .

RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C] // Proceedings of the 18th International Conference on Medical Image Computing and Computer-assisted Intervention . Munich, Germany: Springer, 2015: 234-241.

ZHANG Y L, LI K P, LI K, et al . Image super-resolution using very deep residual channel attention networks[C]// Proceedings of the 15th European Conference on Computer Vision . Munich, Germany: Springer, 2018: 294-310.

WANG X L, GIRSHICK R, GUPTA A, et al . Non-local neural networks[C]// Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City, UT, USA: IEEE, 2018: 7794-7803.

HOWARD A G, ZHU M L, CHEN B, et al . MobileNets: Efficient convolutional neural networks for mobile vision applications[J]. ArXiv , 2017: 1704.04861.

Views

177

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Lightweight method of feature point extraction and matching incorporating a progressive strategy

Lightweight image super-resolution combining residual learning and layer attention

NCA-MobileNet： a lightweight facial expression recognition method

Related Author

YANG Luxia

REN Jiale

ZHANG Hongrui

HAN Rui

CUI Yaowen

MA Yongjie

WU Difan

ZHANG Xuande

Related Institution

School of Computer Science and Technology， Taiyuan Normal University

Shanxi Provincial Key Laboratory of Intelligent Optimization Computing and Blockchain Technology Taiyuan Normal University

School of Physics and Electrical Engineering， Northwest Normal University

School of Electronic Information and Artificial Intelligence， Shaanxi University of Science & Technology， Xi′an

Engineering Training Center， Taiyuan Institute of Technology

AI问答

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China 130033 Postal code：130033
Tel：0431-86176059 Email：yjxs@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰