Online class incremental learning for multi-pose point cloud targets

ZHANG Run-jiang; GUO Jie-long; YU Hui; LAN Hai; WAGN Xi-hao; WEI Xian

doi:10.37188/CJLCD.2022-0419

您当前的位置：

首页 >

文章列表页 >

Online class incremental learning for multi-pose point cloud targets

Object Detection and Recognition | 更新时间：2023-11-03

- Online class incremental learning for multi-pose point cloud targets
- Chinese Journal of Liquid Crystals and Displays Vol. 38, Issue 11, Pages: 1542-1553(2023)
- 作者机构：
  
  1.福州大学电气工程与自动化学院，福建福州 350108
  2.中国科学院福建物质结构研究所，福建福州 350002
  3.中国科学院海西研究院泉州装备制造研究中心，福建泉州 362000
- 作者简介：
- 基金信息：
  
  Fujian Science and Technology Plan(2021T3003);Quanzhou Science and Technology Plan(2021C065L)
- DOI：10.37188/CJLCD.2022-0419
  CLC：
扫描看全文
ZHANG Run-jiang, GUO Jie-long, YU Hui, et al. Online class incremental learning for multi-pose point cloud targets. [J]. Chinese Journal of Liquid Crystals and Displays 38(11):1542-1553(2023)
DOI：

ZHANG Run-jiang, GUO Jie-long, YU Hui, et al. Online class incremental learning for multi-pose point cloud targets. [J]. Chinese Journal of Liquid Crystals and Displays 38(11):1542-1553(2023) DOI： 10.37188/CJLCD.2022-0419.

摘要

针对目前增量学习中所面向目标都是固定姿态这一现象，本文考虑了更严格的设定，即面向多姿态目标的在线类增量学习，并提出了无视姿态重放方法来缓解在线类增量学习中面对多姿态目标时的灾难性遗忘。首先，将2D/3D目标进行点云化处理，以方便提取目标的有效几何信息；其次，基于, $S E (d) (d = 2,3)$ ,https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=49588130&type=,https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=49588142&type=,22.77533340,3.04800010,群对网络进行平移旋转等变性改进，使网络能够提取更丰富的几何信息，从而降低模型在每个任务中受目标姿态的影响；最后，根据损失变化采样特定样本用于重放来缓解灾难性遗忘。实验结果表明，在面对固定姿态目标MNIST、CIFAR-10时，本文方法的最终平均精度分别达到了88%和42.6%，与对比方法结果相近，但最终平均遗忘率明显优于对比方法，分别降低了约3%和15%。在面对多姿态目标RotMNIST、trCIFAR-10时，本文方法依旧能很好地保持在固定姿态目标中的表现，基本不受目标姿态的影响。此外，在3D数据集ModelNet40中的表现也依旧稳定。本文所提方法在在线类增量学习中能够不受目标姿态的影响，同时能缓解灾难性遗忘，具有很好的稳定性和可塑性。

Abstract

In response to current phenomenon that all targets in incremental learning are fixed pose， this paper considers a more rigorous setting， ,i.‍e., online class incremental learning for multi-pose targets， which innovatively proposes an ignoring pose replay method to alleviate the catastrophic forgetting in facing multi-pose targets in online class incremental learning. Firstly， 2D/3D targets are point-clouded to facilitate the extraction of useful geometric information. Secondly， the network modifies for equivariance based on the , $S E (d) (d = 2,3)$ ,https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=49588850&type=,https://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=49588839&type=,26.58533096,3.55599999, group to enable the network to extract richer geometric information， thus reducing the impact of target poses on the model in each task. Finally， specific samples are sampled for replay to mitigate catastrophic forgetting based on loss variation. Experimental results show that when facing fixed posture targets MNIST and CIFAR-10， final average accuracy reaches to 88% and 42.6% respectively， which is comparable to the comparison method， and final average forgetting is significantly better than the comparison method， with a reduction of about 3% and 15% respectively. In the case of the multi-pose target RotMNIST and trCIFAR-10， the proposed method continues to perform well in fixed-pose targets， largely independent of target pose. In addition， the performance in 3D datasets ModelNet40 and trModelNet40 remains stable. The method proposed is able to be independent of the target pose in online class incremental learning， while achieving catastrophic forgetting mitigation， with excellent stability and plasticity.

关键词

在线类增量学习灾难性遗忘无视姿态重放等变性点云分类

Keywords

online class-incremental learningcatastrophic forgettingignoring pose replayequivariancepoint cloud classification

references

龚勋，张志莹，刘璐，等.人物交互检测研究进展综述［J］.西南交通大学学报，2022，57（4）：693-704. doi: 10.3969/j.issn.0258-2724.20210339http://dx.doi.org/10.3969/j.issn.0258-2724.20210339

GONG X， ZHANG Z Y， LIU L， et al. A survey of human-object interaction detection ［J］. Journal of Southwest Jiaotong University， 2022， 57（4）： 693-704. （in Chinese）. doi: 10.3969/j.issn.0258-2724.20210339http://dx.doi.org/10.3969/j.issn.0258-2724.20210339

刘飞，高红艳，卫泽刚，等.基于Res-Net深度特征的SAR图像目标识别方法［J］.液晶与显示，2021，36（4）：624-631. doi: 10.37188/CJLCD.2020-0226http://dx.doi.org/10.37188/CJLCD.2020-0226

LIU F， GAO H Y， WEI Z G， et al. SAR image target recognition method using based on Res-Net deep features ［J］. Chinese Journal of Liquid Crystals and Displays， 2021， 36（4）： 624-631. （in Chinese）. doi: 10.37188/CJLCD.2020-0226http://dx.doi.org/10.37188/CJLCD.2020-0226

李国友，李晨光，王维江，等.基于单样本学习的多特征人体姿态模型识别研究［J］.光电工程，2021，48（2）：200099. doi: 10.12086/oee.2021.200099http://dx.doi.org/10.12086/oee.2021.200099

LI G Y， LI C G， WANG W J， et al. Research on multi-feature human pose model recognition based on one-shot learning ［J］. Opto-Electronic Engineering， 2021， 48（2）： 200099. （in Chinese）. doi: 10.12086/oee.2021.200099http://dx.doi.org/10.12086/oee.2021.200099

MCCLOSKEY M， COHEN N J. Catastrophic interference in connectionist networks： the sequential learning problem ［J］. Psychology of Learning and Motivation， 1989， 24： 109-165. doi: 10.1016/s0079-7421(08)60536-8http://dx.doi.org/10.1016/s0079-7421(08)60536-8

LI Z Z， HOIEM D. Learning without forgetting ［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2018， 40（12）： 2935-2947. doi: 10.1109/tpami.2017.2773081http://dx.doi.org/10.1109/tpami.2017.2773081

MERMILLOD M， BUGAISKA A， BONIN P. The stability-plasticity dilemma： investigating the continuum from catastrophic forgetting to age-limited learning effects ［J］. Frontiers in Psychology， 2013， 4： 504. doi: 10.3389/fpsyg.2013.00504http://dx.doi.org/10.3389/fpsyg.2013.00504

ALJUNDI R， CACCIA L， BELILOVSKY E， et al. Online continual learning with maximally interfered retrieval ［J/OL］. arXiv， 2019： 1908.04742. doi: 10.1109/cvpr.2019.01151http://dx.doi.org/10.1109/cvpr.2019.01151

COHEN T， WELLING M. Group equivariant convolutional networks ［C］//Proceedings of the 33nd International Conference on Machine Learning. New York： ICML， 2016： 2990-2999.

KRIZHEVSKY A， SUTSKEVER I， HINTON G E. ImageNet classification with deep convolutional neural networks ［J］. Communications of the ACM， 2017， 60（6）： 84-90. doi: 10.1145/3065386http://dx.doi.org/10.1145/3065386

LECUN Y， BENGIO Y. Convolutional networks for images， speech， and time-series ［M］//ARBIB M A. The Handbook of Brain Theory and Neural Networks. Cambridge： MIT Press， 1995.

ESTEVES C， ALLEN-BLANCHETTE C， MAKADIA A， et al. Learning SO（3） equivariant representations with spherical CNNs ［C］//Proceedings of the 15th European Conference on Computer Vision. Munich： Springer， 2018： 54-70. doi: 10.1007/978-3-030-01261-8_4http://dx.doi.org/10.1007/978-3-030-01261-8_4

RAVANBAKHSH S， SCHNEIDER J G， PÓCZOS B. Deep learning with sets and point clouds ［C］. 5th International Conference on Learning Representations. Toulon： ICLR， 2017.

WANG Y， SUN Y B， LIU Z W， et al. Dynamic graph CNN for learning on point clouds ［J］. ACM Transactions On Graphics， 2019， 38（5）： 146. doi: 10.1145/3326362http://dx.doi.org/10.1145/3326362

CAI S S， XU Z W， HUANG Z C， et al. Enhancing CNN incremental learning capability with an expanded network ［C］//2018 IEEE International Conference on Multimedia and Expo （ICME）. San Diego： IEEE， 2018： 1-6. doi: 10.1109/icme.2018.8486457http://dx.doi.org/10.1109/icme.2018.8486457

MALLYA A， DAVIS D， LAZEBNIK S. Piggyback： Adapting a single network to multiple tasks by learning to mask weights ［C］//Proceedings of the European Conference on Computer Vision （ECCV）. Munich： Springer， 2018： 67-82. doi: 10.1007/978-3-030-01225-0_5http://dx.doi.org/10.1007/978-3-030-01225-0_5

MALLYA A， LAZEBNIK S. PackNet： Adding multiple tasks to a single network by iterative pruning ［C］//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City： IEEE， 2018： 7765-7773. doi: 10.1109/cvpr.2018.00810http://dx.doi.org/10.1109/cvpr.2018.00810

KIRKPATRICK J， PASCANU R， RABINOWITZ N， et al. Overcoming catastrophic forgetting in neural networks ［J］. Proceedings of the National Academy of Sciences of the United States of America， 2017， 114（13）： 3521-3526. doi: 10.1073/pnas.1611835114http://dx.doi.org/10.1073/pnas.1611835114

CHAUDHRY A， DOKANIA P K， AJANTHAN T， et al. Riemannian walk for incremental learning： Understanding forgetting and intransigence ［C］//Proceedings of the 15th European Conference on Computer Vision. Munich： Springer， 2018： 556-572. doi: 10.1007/978-3-030-01252-6_33http://dx.doi.org/10.1007/978-3-030-01252-6_33

ZENKE F， POOLE B， GANGULI S. Continual learning through synaptic intelligence ［C］//Proceedings of the 34th International Conference on Machine Learning. Sydney： ACM， 2017： 3987-3995. doi: 10.1007/978-1-4899-7687-1_171http://dx.doi.org/10.1007/978-1-4899-7687-1_171

REBUFFI S A， KOLESNIKOV A， SPERL G， et al. iCaRL： incremental classifier and representation learning ［C］//Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. Honolulu： IEEE， 2017： 5533-5542. doi: 10.1109/cvpr.2017.587http://dx.doi.org/10.1109/cvpr.2017.587

王伟，张志莹，郭杰龙，等.基于脑启发的类增量学习［J］.计算机应用研究，2023，40（3）：671-675，688.

WANG W， ZHANG Z Y， GUO J L， et al. Brain-inspired class incremental learning ［J］. Application Research of Computers， 2023， 40（3）： 671-675， 688. （in Chinese）

SHIN H， LEE J K， KIM J， et al. Continual learning with deep generative replay ［C］//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach： ACM， 2017： 2994-3003.

RIOS A， ITTI L. Closed-loop memory GAN for continual learning ［C］//Proceedings of the 28th International Joint Conference on Artificial Intelligence. Macao， China： ACM， 2019： 3332-3338. doi: 10.24963/ijcai.2019/462http://dx.doi.org/10.24963/ijcai.2019/462

DOUILLARD A， CORD M， OLLION C， et al. PODNet： pooled outputs distillation for small-tasks incremental learning ［C］. 16th European Conference on Computer Vision. Glasgow： Springer， 2020： 86-102. doi: 10.1007/978-3-030-58565-5_6http://dx.doi.org/10.1007/978-3-030-58565-5_6

HU X T， TANG K H， MIAO C Y， et al. Distilling causal effect of data in class-incremental learning ［C］//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville： IEEE， 2021： 3956-3965. doi: 10.1109/cvpr46437.2021.00395http://dx.doi.org/10.1109/cvpr46437.2021.00395

BANG J， KIM H， YOO Y J， et al. Rainbow memory： continual learning with a memory of diverse samples ［C］//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville： IEEE， 2021： 8214-8223. doi: 10.1109/cvpr46437.2021.00812http://dx.doi.org/10.1109/cvpr46437.2021.00812

ZHU F， CHENG Z， ZHANG X Y， et al. Class-incremental learning via dual augmentation ［C］//Proceedings of the 35th International Conference on Neural Information Processing Systems. Online： NeurIPS， 2021： 14306-14318.

HUTCHINSON M J， LE LAN C， ZAIDI S， et al. Lietransformer： Equivariant self-attention for lie groups ［C］//Proceedings of the 38th International Conference on Machine Learning. Online： ICML， 2021： 4533-4543.

FASEL B， GATICA-PEREZ D. Rotation-invariant neoperceptron ［C］. 18th International Conference on Pattern Recognition （ICPR'06）. Hong Kong， China： IEEE， 2006： 336-339. doi: 10.1109/icpr.2006.1020http://dx.doi.org/10.1109/icpr.2006.1020

LAPTEV D， SAVINOV N， BUHMANN J M， et al. TI-POOLING： transformation-invariant pooling for feature learning in convolutional neural networks ［C］//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas： IEEE， 2016： 289-297. doi: 10.1109/cvpr.2016.38http://dx.doi.org/10.1109/cvpr.2016.38

COHEN T S， GEIGER M， KÖHLER J， et al. Spherical CNNs ［C］. 6th International Conference on Learning Representations. Vancouver： ICLR， 2018.

DENG C Y， LITANY O， DUAN Y Q， et al. Vector neurons： a general framework for SO（3）‍-equivariant networks ［C］//Proceedings of the IEEE/CVF International Conference on Computer Vision. Montreal： IEEE， 2021： 12180-12189. doi: 10.1109/iccv48922.2021.01198http://dx.doi.org/10.1109/iccv48922.2021.01198

FUCHS F， WORRALL D， FISCHER V， et al. Se（3）-transformers： 3d roto-translation equivariant attention networks ［C］//Proceedings of the 34th International Conference on Neural Information Processing Systems. Online： NeurIPS， 2020： 1970-1981.

FINZI M， STANTON S， IZMAILOV P， et al. Generalizing convolutional neural networks for equivariance to lie groups on arbitrary continuous data ［C］//Proceedings of the 37th International Conference on Machine Learning. Virtual Event： ACM， 2020： 297.

SOSNOVIK I， SZMAJA M， SMEULDERS A. Scale-equivariant steerable networks ［C］. 8th International Conference on Learning Representations. Addis Ababa： ICLR， 2019.

DAI J F， QI H Z， XIONG Y W， et al. Deformable convolutional networks ［C］//Proceedings of the IEEE International Conference on Computer Vision. Venice： IEEE， 2017： 764-773. doi: 10.1109/iccv.2017.89http://dx.doi.org/10.1109/iccv.2017.89

WU W X， QI Z A， FUXIN L. PointConv： deep convolutional networks on 3d point clouds ［C］//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach： IEEE， 2019： 9613-9622. doi: 10.1109/cvpr.2019.00985http://dx.doi.org/10.1109/cvpr.2019.00985

QI C R， YI L， SU H， et al. PointNet++： Deep hierarchical feature learning on point sets in a metric space ［C］//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach： ACM， 2017： 5105-5114.

ROBINS A. Catastrophic forgetting， rehearsal and pseudorehearsal ［J］. Connection Science， 1995， 7（2）： 123-146. doi: 10.1080/09540099550039318http://dx.doi.org/10.1080/09540099550039318

JIN X S， SADHU A， DU J Y， et al. Gradient-based editing of memory examples for online task-free continual learning ［C］//Proceedings of the 35th International Conference on Neural Information Processing Systems. Online： NeurIPS， 2021： 29193-29205. doi: 10.18653/v1/2020.emnlp-main.158http://dx.doi.org/10.18653/v1/2020.emnlp-main.158

CHAUDHRY A， ROHRBACH M， ELHOSEINY M， et al. Continual learning with tiny episodic memories ［J/OL］. arXiv， 2019：1902.10486v1.

TONEVA M， SORDONI A， DES COMBES R T， et al. An empirical study of example forgetting during deep neural network learning ［C］. 7th International Conference on Learning Representations. New Orleans： ICLR， 2019.

LECUN Y， BOTTOU L， BENGIO Y， et al. Gradient-based learning applied to document recognition ［J］. Proceedings of the IEEE， 1998， 86（11）： 2278-2324. doi: 10.1109/5.726791http://dx.doi.org/10.1109/5.726791

LAROCHELLE H， ERHAN D， COURVILLE A， et al. An empirical evaluation of deep architectures on problems with many factors of variation ［C］//Proceedings of the 24th International Conference on Machine Learning. Corvalis： ACM， 2007： 473-480. doi: 10.1145/1273496.1273556http://dx.doi.org/10.1145/1273496.1273556

KRIZHEVSKY A. Learning multiple layers of features from tiny images ［D］. Toronto： University of Toronto， 2009.

WU Z R， SONG S R， KHOSLA A， et al. 3D ShapeNets： a deep representation for volumetric shapes ［C］//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston： IEEE， 2015： 1912-1920. doi: 10.1109/cvpr.2015.7298801http://dx.doi.org/10.1109/cvpr.2015.7298801

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

College of Electrical Engineering and Automation， Fuzhou University

Fujian Institute of Research on the Structure of Matter， Chinese Academy of Sciences

Quanzhou Institute of Equipment Manufacturing Haixi Institutes， Chinese Academy of Sciences

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China 130033 Postal code：130033
Tel：0431-86176059 Email：yjxs@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰