Real-time automatic image cropping based on portrait detection

WU Yu-hang; LIN Shan-ling; LIN Zhi-xian; GUO Tai-liang

doi:10.37188/CJLCD.2022-0258

您当前的位置：

首页 >

文章列表页 >

Real-time automatic image cropping based on portrait detection

Image Processing | 更新时间：2023-05-09

- Real-time automatic image cropping based on portrait detection
- Chinese Journal of Liquid Crystals and Displays Vol. 38, Issue 5, Pages: 617-624(2023)
- 作者机构：
  
  1.福州大学先进制造学院，福建泉州 362200
  2.福州大学物理与信息工程学院，福建福州 350116
  3.中国福建光电信息科学与技术实验室，福建福州 350116
- 作者简介：
- 基金信息：
  
  National Key R&D Program of China(2021YFB3600603);Projects supported by Natural Science Foundation of Fujian Province(2020J01468);Youth Science Foundation of the National Natural Science Foundation of China(62101132)
- DOI：10.37188/CJLCD.2022-0258
  CLC： TP391
- Received：01 August 2022，
  
  Revised：02 September 2022，
  
  Published：05 May 2023
- 稿件说明：
移动端阅览
WU Yu-hang, LIN Shan-ling, LIN Zhi-xian, et al. Real-time automatic image cropping based on portrait detection[J]. Chinese journal of liquid crystals and displays, 2023, 38(5): 617-624.
DOI：

WU Yu-hang, LIN Shan-ling, LIN Zhi-xian, et al. Real-time automatic image cropping based on portrait detection[J]. Chinese journal of liquid crystals and displays, 2023, 38(5): 617-624. DOI： 10.37188/CJLCD.2022-0258.

摘要

针对现有图像智能裁剪算法在处理人像图片时，存在误将主要人物整个人或部分身体部位裁减掉等致使图像关键信息缺失和构图不佳的问题，本文提出一种基于人像检测的实时图像智能裁剪方法。该方法将图像裁剪分为人像检测和智能构图两个阶段，旨在将提取的人像坐标信息作为智能裁剪算法的输入，然后结合基于美学的构图法则对图片进行自动构图，确保裁剪结果中人像信息的完整性并提升图片的构图美感。实验结果表明，本文在Center-Net基础上改造的轻量级检测网络，运算量减少了86%，精度提升了3.34%，便于将该裁剪算法应用于移动端设备。整个裁剪算法的FPS达到了77，并且裁剪后的人像信息完整，图片整体构图得到改善。

Abstract

In order to solve the problem that the existing automatic image cropping algorithm mistakenly cuts off the whole person or part of the body parts of the main figure， which leads to the missing of the key information of the image and the poor composition of the image， this paper proposes a real-time automatic image cropping method based on portrait detection. This method divides image cropping into two stages： portrait detection and intelligent composition. It aims to take the extracted portrait coordinate information as the input of the intelligent cropping algorithm， and then combines the composition rule based on aesthetics to automatically compose the picture， so as to ensure the integrity of the portrait information in the cropping result and improve the composition beauty of the picture. The experimental results show that the lightweight detection network modified on the basis of Center-Net in this paper reduces the computation amount by 86% and improves the accuracy by 3.34%， which is convenient to apply the cropping algorithm to mobile devices. The FPS of the whole cropping algorithm reaches 77， and the cropped portrait information is complete， and the overall composition of the picture is improved.

关键词

Keywords

references

SANTELLA A ， AGRAWALA M ， DECARLO D ， et al . Gaze-based interaction for semi-automatic photo cropping ［C］// Proceedings of SIGCHI Conference on Human Factors in Computing Systems . Montréal ： ACM ， 2006 ： 771 - 780 . doi: 10.1145/1124772.1124886 http://dx.doi.org/10.1145/1124772.1124886

STENTIFORD F . Attention based auto image cropping ［C］// Proceedings of the 5th International Conference on Computer Vision Systems . Germany ： Applied Computer Science Group， Bielefeld University ， 2007 ： 140 - 148 . doi: 10.1109/icip.2006.312482 http://dx.doi.org/10.1109/icip.2006.312482

KAO Y Y ， WANG C ， HUANG K Q . Visual aesthetic quality assessment with a regression model ［C］// Proceedings of 2015 IEEE International Conference on Image Processing . Quebec City ： IEEE ， 2015 ： 1583 - 1587 . doi: 10.1109/icip.2015.7351067 http://dx.doi.org/10.1109/icip.2015.7351067

LU X ， LIN Z ， JIN H L ， et al . RAPID： rating pictorial aesthetics using deep learning ［C］// Proceedings of the 22nd ACM International Conference on Multimedia . Orlando ： ACM ， 2014 ： 457 - 466 . doi: 10.1145/2647868.2654927 http://dx.doi.org/10.1145/2647868.2654927

LI D B ， WU H K ， ZHANG J G ， et al . A2-RL： aesthetics aware reinforcement learning for image cropping ［C］// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City ： IEEE ， 2018 ： 8193 - 8201 . doi: 10.1109/cvpr.2018.00855 http://dx.doi.org/10.1109/cvpr.2018.00855

CHEN Y L ， KLOPP J ， SUN M ， et al . Learning to compose with professional photographs on the web ［C］// Proceedings of the 25th ACM International Conference on Multimedia . Mountain ： ACM ， 2017 ： 37 - 45 . doi: 10.1145/3123266.3123274 http://dx.doi.org/10.1145/3123266.3123274

WEI Z J ， ZHANG J M ， SHEN X H ， et al . Good view hunting： learning photo composition from dense view pairs ［C］// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City ： IEEE ， 2018 ： 5437 - 5446 . doi: 10.1109/cvpr.2018.00570 http://dx.doi.org/10.1109/cvpr.2018.00570

YAN B ， FAN P ， LEI X Y ， et al . A real-time apple targets detection method for picking robot based on improved YOLOv5 ［J］. Remote Sensing ， 2021 ， 13 （ 9 ）： 1619 . doi: 10.3390/rs13091619 http://dx.doi.org/10.3390/rs13091619

DUAN K W ， BAI S ， XIE L X ， et al . CenterNet： keypoint triplets for object detection ［C］// Proceedings of 2019 IEEE/CVF International Conference on Computer Vision . Seoul ： IEEE ， 2019 ： 6568 - 6577 . doi: 10.1109/iccv.2019.00667 http://dx.doi.org/10.1109/iccv.2019.00667

XU Z H ， YUAN X Y ， ZHOU T K ， et al . A multichannel optical computing architecture for advanced machine vision ［J］. Light： Science & Applications ， 2022 ， 11 （ 1 ）： 255 . doi: 10.1038/s41377-022-00945-y http://dx.doi.org/10.1038/s41377-022-00945-y

王建林，付雪松，黄展超，等 . 改进YOLOv2卷积神经网络的多类型合作目标检测［J］. 光学精密工程， 2020 ， 28 （ 1 ）： 251 - 260 . doi: 10.3788/ope.20202801.0251 http://dx.doi.org/10.3788/ope.20202801.0251

WANG J L ， FU X S ， HUANG Z C ， et al . Multi-type cooperative targets detection using improved YOLOv2 convolutional neural network ［J］. Optics and Precision Engineering ， 2020 ， 28 （ 1 ）： 251 - 260 . （in Chinese） . doi: 10.3788/ope.20202801.0251 http://dx.doi.org/10.3788/ope.20202801.0251

SANDLER M ， HOWARD A ， ZHU M L ， et al . MobilenetV2： inverted residuals and linear bottlenecks ［C］// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City ： IEEE ， 2018 ： 4510 - 4520 . doi: 10.1109/cvpr.2018.00474 http://dx.doi.org/10.1109/cvpr.2018.00474

Views

128

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Image dehazing method combining diffusion model and Fourier filter

Multi-attention micro-expression recognition based on color and optical flow

Advances in twin network research in visual tracking technology

Text-to-image generation method based on self-supervised attention and image features fusion

Related Author

Wu Yu-hang

Lin Zhi-xian

Guo Tai-liang

XIAO Polin

LIU Jinfeng

HUANG Kai

WANG Feng

WANG Ye

Related Institution

College of Information Engineering， Ningxia University

College of Electrical and Power Engineering， Taiyuan University of Technology

College of Electronic Information and Optical Engineering， Taiyuan University of Technology

Beijing Xinghang Electromechanical Equipment Co. Ltd.

Technological Institute of Materials & Energy Science， Xijing University

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China 130033 Postal code：130033
Tel：0431-86176059 Email：yjxs@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰