Target event classification method through thinly scattered media

YANG Bo-yu; KE Jun

doi:10.37188/CJLCD.2023-0027

您当前的位置：

首页 >

文章列表页 >

Target event classification method through thinly scattered media

Intelligent Imaging | 更新时间：2023-06-12

- Target event classification method through thinly scattered media
- Chinese Journal of Liquid Crystals and Displays Vol. 38, Issue 6, Pages: 770-777(2023)
- 作者机构：
  
  1.北京理工大学光电学院，北京 100081
- 作者简介：
- 基金信息：
  
  National Natural Science Foundation of China(62175032;U2241295)
- DOI：10.37188/CJLCD.2023-0027
  CLC：
扫描看全文
YANG Bo-yu, KE Jun. Target event classification method through thinly scattered media. [J]. Chinese Journal of Liquid Crystals and Displays 38(6):770-777(2023)
DOI：

YANG Bo-yu, KE Jun. Target event classification method through thinly scattered media. [J]. Chinese Journal of Liquid Crystals and Displays 38(6):770-777(2023) DOI： 10.37188/CJLCD.2023-0027.

摘要

现有的通过散射介质的目标分析主要针对静止目标，同时，环境或外部亮度条件对目标分析结果有着至关重要的影响，这就对图像采集设备的动态范围、时间分辨率等性能提出了极高要求。事件相机由于其高动态范围、高时间分辨率与低延迟等特点为应对上述问题提供了新的解决思路。本文针对低照度情况下运动目标透过薄散射介质探测效果差的问题，利用事件相机开展了透过薄散射介质的目标探测的研究。采用V2E算法利用灰度散斑制备“事件散斑”数据集，并采用ResNet分类网络进行目标分类，获得了94.27%的十分类精确率。实验结果表明，使用事件流在分析通过散射介质的目标信息方面有着巨大的发展潜力。

Abstract

The existing target analysis through scattering media is mainly aimed at stationary targets. At the same time， ambient or external luminance conditions have a critical impact on the target analysis results. This paper puts forward extremely high requirements for the dynamic range and time resolution of the image acquisition equipment. Due to its high dynamic range， high temporal resolution and low latency， event cameras provide new solutions to the above problems. Aiming at the problem of poor detection effect of moving targets through thin scattering medium under low illumination， event camera is adopted to carry out target detection research through thin scattering medium. V2E network is used to prepare the “event speckle” dataset by grayscale speckle， and ResNet classification network is adopted for target classification. The ten-class accuracy of 94.27% is obtained. Experimental results show that the applicaiton of event streams has great potential for analyzing target information passing through scattering media.

关键词

散斑成像事件相机神经网络

Keywords

speckle imagingevent cameraneural networks

references

BABCOCK H W. The possibility of compensating astronomical seeing ［J］. Publications of the Astronomical Society of the Pacific， 1953， 65（386）： 229-236. doi: 10.1086/126606http://dx.doi.org/10.1086/126606

HORSTMEYER R， RUAN HW， YANG C. Guidestar-assisted wavefront-shaping methods for focusing light into. doi: 10.1038/nphoton.2015.140http://dx.doi.org/10.1038/nphoton.2015.140

biological tissue ［J］. Nature Photonics， 2015， 9（9）： 563-571. doi: 10.1038/nphoton.2015.140http://dx.doi.org/10.1038/nphoton.2015.140

POPOFF S， LEROSEY G， FINK M， et al. Image transmission through an opaque material ［J］. Nature Communications， 2010， 1： 81. doi: 10.1038/ncomms1078http://dx.doi.org/10.1038/ncomms1078

KATZ O， SMALL E， SILBERBERG Y. Looking around corners and through thin turbid layers in real time with scattered incoherent light ［J］. Nature Photonics， 2012， 6（8）： 549-553. doi: 10.1038/nphoton.2012.150http://dx.doi.org/10.1038/nphoton.2012.150

朱磊，邵晓鹏.散射成像技术的研究进展［J］.光学学报，2020，40（1）：0111005. doi: 10.3788/aos202040.0111005http://dx.doi.org/10.3788/aos202040.0111005

ZHU L， SHAO X P. Research progress on scattering imaging technology ［J］. Acta Optica Sinica， 2020， 40（1）： 0111005. （in Chinese）. doi: 10.3788/aos202040.0111005http://dx.doi.org/10.3788/aos202040.0111005

孙存志，陈子阳，蒲继雄.调制散射光束的振幅实现聚焦［J］.光学学报，2014，34（8）：0829001． doi: 10.3788/aos201434.0829001http://dx.doi.org/10.3788/aos201434.0829001

SUN C Z， CHEN Z Y， PU J X. Modulating the amplitude of scattering light for focusing ［J］. Acta Optica Sinica， 2014， 34（8）： 0829001. （in Chinese）. doi: 10.3788/aos201434.0829001http://dx.doi.org/10.3788/aos201434.0829001

BERTOLOTTI J， VANPUTTEN E G， BLUM C， et al. Non-invasive imaging through opaque scattering layers ［J］. Nature， 2012， 491（7423）： 232-234. doi: 10.1038/nature11578http://dx.doi.org/10.1038/nature11578

KATZ O， HEIDMANN P， FINK M， et al. Non-invasive single-shot imaging through scattering layers and around corners via speckle correlations ［J］. Nature Photonics， 2014， 8（10）： 784-790. doi: 10.1038/nphoton.2014.189http://dx.doi.org/10.1038/nphoton.2014.189

AMITONOVA L V， MOSK A P， PINKSE P W H. Rotational memory effect of a multimode fiber ［J］. Optics Express， 2015， 23（16）： 20569-20575. doi: 10.1364/oe.23.020569http://dx.doi.org/10.1364/oe.23.020569

谢向生，刘忆琨，梁浩文，等.散斑相关成像：从点扩展函数到光场全要素［J］.光学学报，2020，40（1）：0111004. doi: 10.3788/aos202040.0111004http://dx.doi.org/10.3788/aos202040.0111004

XIE X S， LIU Y K， ZLIANG H W， et al. Speckle correlation imaging： from point spread functions to light field plenoptics ［J］. Acta Optica Sinica， 2020， 40（1）： 0111004. （in Chinese）. doi: 10.3788/aos202040.0111004http://dx.doi.org/10.3788/aos202040.0111004

JUDKEWITZ B， HORSTMEYER R， VELLEKOOP I M， et al. Translation correlations in anisotropically scattering media ［J］. Nature Physics， 2015， 11（8）： 684-689. doi: 10.1038/nphys3373http://dx.doi.org/10.1038/nphys3373

LI Y Z， XUE Y J， TIAN L. Deep speckle correlation： a deep learning approach toward scalable imaging through scattering media ［J］. Optica， 2018， 5（10）： 1181-1190. doi: 10.1364/optica.5.001181http://dx.doi.org/10.1364/optica.5.001181

郑珊珊，杨婉琴，司徒国海.计算光学成像在散射中的应用［J］.红外与激光工程，2019，48（6）：0603005. doi: 10.3788/irla201948.0603005http://dx.doi.org/10.3788/irla201948.0603005

ZHENG S S， YANG W Q， SITU G H. Application of computational optical imaging in scattering ［J］. Infrared and Laser Engineering， 2019， 48（6）： 0603005. （in Chinese）. doi: 10.3788/irla201948.0603005http://dx.doi.org/10.3788/irla201948.0603005

FENG S C， KANE C， LEE P A， et al. Correlations and fluctuations of coherent wave transmission through disordered media ［J］. Physical Review Letters， 1988， 61（7）： 834-837. doi: 10.1103/physrevlett.61.834http://dx.doi.org/10.1103/physrevlett.61.834

FREUND I， ROSENBLUH M， FENG S C. Memory effects in propagation of optical waves through disordered media ［J］. Physical Review Letters， 1988， 61（20）： 2328-2331. doi: 10.1103/physrevlett.61.2328http://dx.doi.org/10.1103/physrevlett.61.2328

LICHTSTEINER P， POSCH C， DELBRUCK T. A 128×128 120 dB 15 μs latency asynchronous temporal contrast vision sensor ［J］. IEEE Journal of Solid-State Circuits， 2008， 43（2）： 566-576. doi: 10.1109/jssc.2007.914337http://dx.doi.org/10.1109/jssc.2007.914337

LICHTSTEINER P， DELBRUCK T. A 64×64 aer logarithmic temporal derivative silicon retina ［C］//Proceedings of the Research in Microelectronics and Electronics. Lausanne， Switzerland： IEEE， 2005： 202-205. doi: 10.1109/rme.2005.1542972http://dx.doi.org/10.1109/rme.2005.1542972

LICHTSTEINER P， POSCH C， DELBRUCK T. A 64×64 aer logarithmic temporal derivative silicon retina ［J］. Research in Microelectronics and Electronics， 2006： 2060-2069.

PIATKOWSKA E， BELBACHIR A N， SCHRAML S， et al. Spatiotemporal multiple persons tracking using dynamic vision sensor ［C］//Proceedings of the 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops. Providence： IEEE， 2012： 35-40. doi: 10.1109/cvprw.2012.6238892http://dx.doi.org/10.1109/cvprw.2012.6238892

ZHU A Z， YUAN L Z， CHANEY K， et al. EV-FlowNet： self-supervised optical flow estimation for event-based cameras ［J/OL］. arXiv， 2018： 1802.06898. doi: 10.15607/rss.2018.xiv.062http://dx.doi.org/10.15607/rss.2018.xiv.062

Kim H， Handa A， Benosman R， et al. Simultaneous mosaicing and tracking with an event camera ［C］//Proceedings British Machine Vision Conference. Nottingham： BMVA Press， 2014. doi: 10.5244/c.28.26http://dx.doi.org/10.5244/c.28.26

GE Z， ZHANG P， GAO Y Z， et al. Lens-free motion analysis via neuromorphic laser speckle imaging ［J］. Optics Express， 2022， 30（2）： 2206-2218. doi: 10.1364/oe.444948http://dx.doi.org/10.1364/oe.444948

ZHOU G， GAO Y Z， LAM E Y. Event-based laser speckle correlation for micro motion estimation ［J］. Optics Letters， 2021， 46（16）： 3885-3888. doi: 10.1364/ol.430419http://dx.doi.org/10.1364/ol.430419

Various Contributors. Event-based vision resources ［EB/OL］. https：//github.com/uzh-rpg/event-based_vision_resourceshttps://github.com/uzh-rpg/event-based_vision_resources.

KATZ M L， NIKOLIC K， DELBRUCK T. Live demonstration： Behavioural emulation of event-based vision sensors ［C］//Proceedings of the 2012 IEEE International Symposium on Circuits and Systems （ISCAS）. Seoul： IEEE， 2012： 736-740. doi: 10.1109/iscas.2012.6272143http://dx.doi.org/10.1109/iscas.2012.6272143

REBECQ H， GEHRIG D， SCARAMUZZA D. ESIM： an open event camera simulator ［C］//Proceedings of the 2nd Conference on Robot Learning. Zürich： PMLR， 2018.

GEHRIG D， GEHRIG M， HIDALGO-CARRIÓ J， et al. Video to events： Recycling video datasets for event cameras ［C］//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Seattle： IEEE， 2020： 3583-3592. doi: 10.1109/cvpr42600.2020.00364http://dx.doi.org/10.1109/cvpr42600.2020.00364

HU Y H， LIU S C， DELBRUCK T. V2E： from video frames to realistic DVS events ［C］//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops （CVPRW）. Nashville： IEEE， 2021： 1312-1321. doi: 10.1109/cvprw53098.2021.00144http://dx.doi.org/10.1109/cvprw53098.2021.00144

JIANG H Z， SUN D Q， JAMPANI V， et al. Super SloMo： high quality estimation of multiple intermediate frames for video interpolation ［C］//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Salt Lake City： IEEE， 2018： 9000-9008. doi: 10.1109/cvpr.2018.00938http://dx.doi.org/10.1109/cvpr.2018.00938

SU S C， DELBRACIO M， WANG J， et al. Deep video deblurring for hand-held cameras ［C］//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Honolulu： IEEE， 2017： 237-246. doi: 10.1109/cvpr.2017.33http://dx.doi.org/10.1109/cvpr.2017.33

KATZ M L，LUTTERBECK C，NIKOLIC K. An implementation of magnocellular pathways in event-based retinomorphic systems [C］//Proceedings of the 2012 IEEE Biomedical Circuits and Systems Conference (BioCAS). Seoul：IEEE， 2012：17-20. doi: 10.1109/biocas.2012.6418480http://dx.doi.org/10.1109/biocas.2012.6418480

REBECQ H， HORSTSCHAEFER T， GALLEGO G， et al. EVO： a geometric approach to event-based 6-DOF parallel tracking and mapping in real time ［J］. IEEE Robotics and Automation Letters， 2017， 2（2）： 593-600. doi: 10.1109/lra.2016.2645143http://dx.doi.org/10.1109/lra.2016.2645143

GEHRIG D， REBECQ H， GALLEGO G， et al. EKLT： Asynchronous photometric feature tracking using events and frames ［J］. International Journal of Computer Vision， 2018， 128（3）： 601-618.

BARDOW P， DAVISON A J， LEUTENEGGER S. Simultaneous optical flow and intensity estimation from an event camera ［C］//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Las Vegas： IEEE， 2016： 884-892. doi: 10.1109/cvpr.2016.102http://dx.doi.org/10.1109/cvpr.2016.102

WANG I， MOSTAFAVI I S M， HO Y S， et al. Event-based high dynamic range image and very high frame rate video generation using conditional generative adversarial networks ［C］//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Long Beach： IEEE， 2019： 10073-10082. doi: 10.1109/cvpr.2019.01032http://dx.doi.org/10.1109/cvpr.2019.01032

BENOSMAN R， CLERCQ C， LAGORCE X， et al. Event-based visual flow ［J］. IEEE Transactions on Neural Networks and Learning Systems， 2014， 25（2）： 407-417. doi: 10.1109/tnnls.2013.2273537http://dx.doi.org/10.1109/tnnls.2013.2273537

SEKIKAWA Y， HARA K， SAITO H. EventNet： asynchronous recursive event processing ［C］//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Long Beach： IEEE， 2019： 3882-3891. doi: 10.1109/cvpr.2019.00401http://dx.doi.org/10.1109/cvpr.2019.00401

REBECQ H， RANFTL R， KOLTUN V， et al. Events-To-Video： bringing modern computer vision to event cameras ［C］.2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Long Beach： IEEE， 2019： 3852-3861. doi: 10.1109/cvpr.2019.00398http://dx.doi.org/10.1109/cvpr.2019.00398

ZHU A Z， YUAN L Z， CHANEY K， et al. Unsupervised event-based optical flow using motion compensation ［C］//Proceedings of the European Conference on Computer Vision. Munich： Springer， 2018. doi: 10.1007/978-3-030-11024-6_54http://dx.doi.org/10.1007/978-3-030-11024-6_54

HE K M， ZHANG X Y， REN S Q， et al. Deep residual learning for image recognition ［C］//Proceedings of the 2016 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Las Vegas： IEEE， 2016： 770-778. doi: 10.1109/cvpr.2016.90http://dx.doi.org/10.1109/cvpr.2016.90

National Institute of Standards and Technology. Mixed national institute of standards and technology database http：//yann.lecun.com/exdb/mnist/http://yann.lecun.com/exdb/mnist/. doi: 10.6028/nist.sp.732http://dx.doi.org/10.6028/nist.sp.732

ZUO C， CHEN Q. Exploiting optical degrees of freedom for information multiplexing in diffractive neural networks ［J］. Light： Science & Applications， 2022， 11（1）： 208. doi: 10.1038/s41377-022-00903-8http://dx.doi.org/10.1038/s41377-022-00903-8

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Underwater image enhancement method combining feature fusion and physical correction

Loading effect and solutions in gate etching process for TFT

Real-time detection model of highway vehicle based on YOLOv5s

Related Author

No data

Related Institution

Beijing Institute of Technology

School of Information， Shanghai Ocean University

Department of Applied Physics， Chongqing University

Chongqing BOE Optoelectronics Technology Co.， Ltd.

Beijing BOE Display Technology Co.， Ltd.

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China 130033 Postal code：130033
Tel：0431-86176059 Email：yjxs@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰