Traffic sign recognition based on multi-scale feature fusion and extreme learning machine

Yong-jie MA; Shi-sheng CHENG; Yun-ting MA; Min CHEN

doi:10.3788/YJYXS20203506.0572

您当前的位置：

首页 >

文章列表页 >

Traffic sign recognition based on multi-scale feature fusion and extreme learning machine

Image Processing | 更新时间：2020-07-13

- Traffic sign recognition based on multi-scale feature fusion and extreme learning machine
- Chinese Journal of Liquid Crystals and Displays Vol. 35, Issue 6, Pages: 572-582(2020)
- 作者机构：
  
  西北师范大学物理与电子工程学院, 甘肃兰州 730070
- 作者简介：
- 基金信息：
- DOI：10.3788/YJYXS20203506.0572
  CLC： TP391.7
- Received：05 November 2019，
  
  Revised：02 February 2020，
  
  Accepted：02 February 2020，
  
  Published：05 June 2020
- 稿件说明：
移动端阅览
Yong-jie MA, Shi-sheng CHENG, Yun-ting MA, et al. Traffic sign recognition based on multi-scale feature fusion and extreme learning machine[J]. Chinese journal of liquid crystals and displays, 2020, 35(6): 572-582.
DOI：

Yong-jie MA, Shi-sheng CHENG, Yun-ting MA, et al. Traffic sign recognition based on multi-scale feature fusion and extreme learning machine[J]. Chinese journal of liquid crystals and displays, 2020, 35(6): 572-582. DOI： 10.3788/YJYXS20203506.0572.

摘要

针对卷积神经网络只能使用相同尺寸图像和卷积核的网络进行特征提取，导致提取的特征不全面，在交通标志识别中因车载摄像头与交通标志的位置不断变化影响交通标志的识别精度等问题，提出了一种基于多尺度特征融合与极限学习机结合的交通标志识别方法。首先，将预训练适应3种不同尺寸图像的网络模型作为实验的初始模型；然后，融合3个网络模型构建多尺度卷积神经网络，将3个预训练网络的参数级联到融合模型的全连接层，对融合模型的全连接层进行训练，采用随机梯度下降算法更新网络参数；最后，将融合后的模型作为特征提取器提取特征，把提取到的多尺度特征送入极限学习机，实现交通标志识别。实验采用德国交通标志数据库(GTSRB)对算法性能进行测试，实验结果显示，多尺度特征融合与极限学习机结合的网络识别精度为99.23%，识别速度为46 ms。相对于预训练的网络，网络的分类精度分别提高了2.35%，3.22%，3.74%。多尺度特征融合能够有效提取交通标志图像的特征信息，极限学习机可以提高分类精度和分类时间，该方法能满足交通标志识别的准确性和实时性的要求。

Abstract

Feature extraction is performed on a convolutional neural network using only the same size image and convolution kernel

which results in incomplete extraction features. In the traffic sign recognition

the continuous change of the position between the vehicle camera and the traffic sign affects the recognition accuracy of the traffic sign. A traffic sign recognition method based on multi-scale feature fusion and extreme learning machine is proposed. Firstly

the network model adapted to three different size images is pre-trained as the initial model of the experiment. Then

the three network models are combined to construct a multi-scale convolutional neural network

and the parameters of the three pre-trained networks are cascaded to the full connection of the fusion model to train the fully connected layer of the fusion model

and the network parameters are updated by the random gradient descent algorithm. Finally

the fusion model is used as the feature extractor to extract features

which are sent to the extreme learning machine(ELM) to realize traffic sign recognition. The experimental results show that the recognition accuracy of the multi-scale feature fusion and extreme learning machine combined network is 99.23% and the recognition speed is 46 ms. Compared with the pre-trained network

the classification accuracy of the network is increased by 2.35%

3.22% and 3.74%

respectively. Therefore

the multi-scale feature fusion method can effectively extract the feature information of traffic sign images

ELM classifier can improve classification accuracy and classification time

and meet the requirements of traffic sign recognition accuracy and real-time.

关键词

Keywords

references

DALAI N, TRIGGS B. Histograms of oriented gradients for human detection[C]// IEEE Computer Society Conference on Computer Vision & Pattern Recognition . San Diego, CA: IEEE, 2005: 886-893.

VIOLA P, JONES M. Rapid object detection using a boosted cascade of simple features[C]// Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition . Kauai, HI, USA: IEEE, 2001: 511-518.

D G LOWE . Distinctive image fearures from scale-invariant keypoints . International Journal of Computer Vision , 2004 . 60 ( 2 ): 91 - 110 .

T OJALA , M PIETIKÄINEN , D HARWOOD . A comparative study of texture measures with classification based on featured distributions . Pattern Recognition , 1996 . 29 ( 1 ): 51 - 59 .

刘占文 , 赵祥模 , 李强 , 等 . 基于图模型与卷积神经网络的交通标志识别方法 . 交通运输工程学报 , 2016 . 16 ( 5 ): 122 - 131 . DOI: 10.3969/j.issn.1671-1637.2016.05.014 http://doi.org/10.3969/j.issn.1671-1637.2016.05.014 .

Z W LIU , X M ZHAO , Q LI , 等 . Traffic sign recognition method based on graphical model and convolutional neural network . Journal of Traffic and Transportation Engineering , 2016 . 16 ( 5 ): 122 - 131 . DOI: 10.3969/j.issn.1671-1637.2016.05.014 http://doi.org/10.3969/j.issn.1671-1637.2016.05.014 .

ZENG Y, XU X, FANG Y, et al . Traffic sign recognition using deep convolutional networks and extreme learning machine[C]. 5th International Conference on Intelligent Science and Big Data Engineering. Suzhou, China: Springer, 2015: 272-280.

LIU S, QI L, QIN H F, et al . Path aggregation network for instance segmentation[C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE, 2018: 8759-8768.

LIN D, SHEN D G, SHEN S T, et al . ZigZagNet: Fusing top-down and bottom-up context for object segmentation[C]. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, CA, USA: IEEE, 2019: 7482-7491

Z QIN , Z M LI , Z N ZHANG , 等 . ThunderNet:Towards real-time generic object detection . ArXiv , 2019 . 1903 .

Q J ZHAO , T SHENG , Y T WANG , 等 . M2Det:A single-shot object detector based on multi-level feature pyramid network . ArXiv , 2019 . 1811 .

Y LIU , C ZHANG , J CHENG , 等 . A multi-scale data fusion framework for bone age assessment with convolutional neural networks . Computers in Biology and Medicine , 2019 . 108 161 - 173 . DOI: 10.1016/j.compbiomed.2019.03.015 http://doi.org/10.1016/j.compbiomed.2019.03.015 .

S XIANG , Q K LIANG , Y C HU , 等 . AMC-Net:Asymmetric and multi-scale convolutional neural network for multi-label HPA classification . Computer Methods and Programs in Biomedicine , 2019 . 178 275 - 287 . DOI: 10.1016/j.cmpb.2019.07.009 http://doi.org/10.1016/j.cmpb.2019.07.009 .

S F CHEVTCHENKO , R F VALE , V MACARIO , 等 . A convolutional neural network with feature fusion for real-time hand posture recognition . Applied Soft Computing , 2018 . 73 748 - 766 . DOI: 10.1016/j.asoc.2018.09.010 http://doi.org/10.1016/j.asoc.2018.09.010 .

张俊.基于AlexNet融合特征的图像检索研究[D].重庆: 重庆邮电大学, 2016.

ZHANG J. Image retrieval based on AlexNet fusion feature[D]. Chongqing: Chongqing University of Posts and Telecommunications, 2016. (in Chinese)

林志洁 , 罗壮 , 赵磊 , 等 . 特征金字塔多尺度全卷积目标检测算法 . 浙江大学学报:工学版 , 2019 . 53 ( 3 ): 533 - 540 . http://d.old.wanfangdata.com.cn/Periodical/zjdxxb-gx201903014 http://d.old.wanfangdata.com.cn/Periodical/zjdxxb-gx201903014 .

Z J LIN , Z LUO , L ZHAO , 等 . Multi-scale convolution target detection algorithm with feature pyramid . Journal of Zhejiang University:Engineering Science , 2019 . 53 ( 3 ): 533 - 540 . http://d.old.wanfangdata.com.cn/Periodical/zjdxxb-gx201903014 http://d.old.wanfangdata.com.cn/Periodical/zjdxxb-gx201903014 .

余烨 , 金强 , 傅云翔 , 等 . 基于Fg-CarNet的车辆型号精细分类研究 . 自动化学报 , 2018 . 44 ( 10 ): 1864 - 1875 . http://d.old.wanfangdata.com.cn/Periodical/zdhxb201810013 http://d.old.wanfangdata.com.cn/Periodical/zdhxb201810013 .

W YU , Q JIN , Y X FU , 等 . Fine-grained classification of car models using Fg-CarNet convolutional neural network . Acta Automatica Sinica , 2018 . 44 ( 10 ): 1864 - 1875 . http://d.old.wanfangdata.com.cn/Periodical/zdhxb201810013 http://d.old.wanfangdata.com.cn/Periodical/zdhxb201810013 .

G B HUANG , Q Y ZHU , C K SIEW . Extreme learning machine:Theory and applications . Neurocomputing , 2006 . 70 ( 1/3 ): 489 - 501 . http://d.old.wanfangdata.com.cn/Periodical/kzllyyy201501011 http://d.old.wanfangdata.com.cn/Periodical/kzllyyy201501011 .

欧先锋 , 向灿群 , 郭龙源 , 等 . 基于Caffe深度学习框架的车牌数字字符识别算法研究 . 四川大学学报:自然科学版 , 2017 . 54 ( 5 ): 971 - 977 . http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=scdxxb201705013 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=scdxxb201705013 .

X F OU , C Q XIANG , L Y GUO , 等 . Research of recognition of digital characters on vehicle license based on Caffe deep learning framework . Journal of Sichuan University:Natural Science Edition , 2017 . 54 ( 5 ): 971 - 977 . http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=scdxxb201705013 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=scdxxb201705013 .

孙伟 , 杜宏吉 , 张小瑞 , 等 . 基于CNN多层特征和ELM的交通标志识别 . 电子科技大学学报 , 2018 . 47 ( 3 ): 343 - 349 . DOI: 10.3969/j.issn.1001-0548.2018.03.004 http://doi.org/10.3969/j.issn.1001-0548.2018.03.004 .

W SUN , H J DU , X R ZHANG , 等 . Traffic sign recognition method based on multi-layer feature CNN and extreme learning machine . Journal of University of Electronic Science and Technology of China , 2018 . 47 ( 3 ): 343 - 349 . DOI: 10.3969/j.issn.1001-0548.2018.03.004 http://doi.org/10.3969/j.issn.1001-0548.2018.03.004 .

宋青松 , 张超 , 田正鑫 , 等 . 基于多尺度卷积神经网络的交通标志识别 . 湖南大学学报:自然科学版 , 2018 . 45 ( 8 ): 131 - 137 . http://d.old.wanfangdata.com.cn/Periodical/jsjgc202003037 http://d.old.wanfangdata.com.cn/Periodical/jsjgc202003037 .

Q S SONG , C ZHANG , Z X TIAN , 等 . Traffic sign recognition based on multi-scale convolutional neural network . Journal of Hunan University:Natural Sciences , 2018 . 45 ( 8 ): 131 - 137 . http://d.old.wanfangdata.com.cn/Periodical/jsjgc202003037 http://d.old.wanfangdata.com.cn/Periodical/jsjgc202003037 .

梁浩 , 刘克俭 , 刘康 , 等 . 引入再检测机制的孪生神经网络目标跟踪 . 光学精密工程 , 2019 . 27 ( 7 ): 1621 - 1631 . http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201907024 http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201907024 .

H LIANG , K J LIU , K LIU , 等 . Siamese network tracking with redetection mechanism . Optics and Precision Engineering , 2019 . 27 ( 7 ): 1621 - 1631 . http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201907024 http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201907024 .

王军 , 何昕 , 魏仲慧 , 等 . 基于多特征匹配的快速星图识别算法 . 光学精密工程 , 2019 . 27 ( 8 ): 1870 - 1879 .

WANG J Jun , X HE , Z H WEI , 等 . Fast star identification algorithm based on multi-feature matching . Optics and Precision Engineering , 2019 . 27 ( 8 ): 1870 - 1879 .

王广龙 , 田杰 , 朱文杰 , 等 . 特征融合和自适应权重更新相结合的运动模糊目标跟踪 . 光学精密工程 , 2019 . 27 ( 5 ): 1158 - 1166 . http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201905019 http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201905019 .

G L WANG , J TIAN , W J ZHU , 等 . Feature fusion and weight adaptive updating based motion blur object tracking . Optics and Precision Engineering , 2019 . 27 ( 5 ): 1158 - 1166 . http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201905019 http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201905019 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Design of heterogeneous FPGA hardware accelerator based on CNN

Lightweight image super-resolution combining residual learning and layer attention

Remote sensing image change detection based on CNN-Transformer structure

Wavefront correction method based on P-U-net for pyramid wavefront detector

Image rain removal algorithm based on multi-cascade progressive convolution structure

Related Author

JI Haolin

XU Wei

PIAO Yongjie

WU Xiaobin

GAO Tan

WU Difan

ZHANG Xuande

PAN Mengyang

Related Institution

Key Laboratory of Space-based Dynamic & Rapid Optical Imaging Technology， Chinese Academy of Sciences

Changchun Institute of Optics， Fine Mechanics and Physics， Chinese Academy of Sciences

University of Chinese Academy of Sciences

School of Electronic Information and Artificial Intelligence， Shaanxi University of Science & Technology， Xi′an

Key Laboratory of Electro-Optical Countermeasure Test & Evaluation Technology

AI问答

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China 130033 Postal code：130033
Tel：0431-86176059 Email：yjxs@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰