Development status and trend of event-based vision sensor

Ying-hong FANG; Wei XU; Yong-jie PIAO; Ru-peng FENG; Liang-liang ZHENG

doi:10.37188/CJLCD.2021-0149

您当前的位置：

首页 >

文章列表页 >

Development status and trend of event-based vision sensor

Image Processing | 更新时间：2021-12-10

- Development status and trend of event-based vision sensor
- Chinese Journal of Liquid Crystals and Displays Vol. 36, Issue 12, Pages: 1664-1673(2021)
- 作者机构：
  
  1.中国科学院长春光学精密机械与物理研究所, 吉林长春 130033
  2.中国科学院大学, 北京 100049
- 作者简介：
- 基金信息：
  
  Supported by Qian Xuesen Laboratory of Space Technology,GAST(GZZKFJJ2020003)
- DOI：10.37188/CJLCD.2021-0149
  CLC： TP391.4;TH691.9
- Received：31 May 2021，
  
  Revised：29 June 2021，
  
  Published：2021-12
- 稿件说明：
移动端阅览
Ying-hong FANG, Wei XU, Yong-jie PIAO, et al. Development status and trend of event-based vision sensor[J]. Chinese journal of liquid crystals and displays, 2021, 36(12): 1664-1673.
DOI：

Ying-hong FANG, Wei XU, Yong-jie PIAO, et al. Development status and trend of event-based vision sensor[J]. Chinese journal of liquid crystals and displays, 2021, 36(12): 1664-1673. DOI： 10.37188/CJLCD.2021-0149.

摘要

基于事件的视觉传感器是一种新型的仿生型视觉传感器，它更类似于人眼的工作机制使其广受关注。与基于帧的传统相机的工作机制和输出方式不同，基于事件的视觉传感器的像素可以单独检测光照强度对数的变化，并在变化量超过一定阈值时输出包含位置、时间、极性的事件信息，拥有低延迟、高动态范围、低功耗的优点，其独特的输出方式和工作特性使其特别适应于有高速运动、光照条件变化较大或小能耗的场合。本文介绍了事件相机的发展历程、分类和工作原理、优缺点，以及事件相机在快速运动的跟踪与监测、目标识别、即时定位与地图构建（SLAM）等领域近些年的应用情况。最后总结了事件相机在不同应用领域仍存在的挑战，并展望其未来的发展。事件相机的广泛应用将为目前传统相机仍很棘手的高速运动和高动态范围场合提供新的解决方案，在未来的不断更新和发展下，它将能够在更多复杂的应用场景发挥作用。

Abstract

Event-based vision sensor is a new type of bionic vision sensor

which is more similar to the working mechanism of human eye

and makes it widely concerned. Different from the working mechanism and output mode of traditional frame-based cameras

the pixels of event-based vision sensors can individually detect logarithmic change of light intensity

and output event information including location

time

and polarity when the amount of change exceeds a certain threshold. The event-based vision sensor has the advantages of low latency

high dynamic range

and low power consumption. Its unique output mode and working characteristics make it especially suitable for occasions with high-speed movement

large changes in lighting conditions

or low energy consumption. This article introduces the development history

classification

working principle

advantages and disadvantages of event-based cameras

as well as their applications in the fields of rapid motion tracking and monitoring

target recognition

simultaneous localization and mapping(SLAM) in recent years. Finally

it summarizes the challenges that the event-based camera still exists in different application fields

and looks forward to its future development. The wide application of event-based cameras can provide new solutions for high-speed motion and high dynamic range occasions where traditional cameras are still tricky. With continuous updates and development in the future

it will be able to play a role in more complex application scenarios.

关键词

Keywords

references

孔德磊 , 方正 . 基于事件的视觉传感器及其应用综述 . 信息与控制 , 2021 . 50 ( 1 ): 1 - 19 . DOI: 10.13976/j.cnki.xk.2021.0069 http://doi.org/10.13976/j.cnki.xk.2021.0069 .

D L KONG , Z FANG . A review of event-based vision sensors and their applications . Information and Control , 2021 . 50 ( 1 ): 1 - 19 . DOI: 10.13976/j.cnki.xk.2021.0069 http://doi.org/10.13976/j.cnki.xk.2021.0069 .

C POSCH , T SERRANO-GOTARREDONA , B LINARES-BARRANCO , 等 . Retinomorphic event-based vision sensors: bioinspired cameras with spiking output . Proceedings of the IEEE , 2014 . 102 ( 10 ): 1470 - 1484 . DOI: 10.1109/JPROC.2014.2346153 http://doi.org/10.1109/JPROC.2014.2346153 .

G CRISTÓBAL , L PERRINET , M S KEIL . Biologically Inspired Computer Vision , : Berlin Wiley-VCH , 2015 .

LICHTSTEINER P, DELBRUCK T. A 64×64 AER logarithmic temporal derivative silicon retina[C]// Research in Microelectronics and Electronics , 2005 PhD . Lausanne: IEEE, 2005: 202-205.

C MEAD . Analog VLSI and Neural Systems , : Reading, Mass. Addison-Wesley , 1989 .

C MEAD . Neuromorphic electronic systems . Proceedings of the IEEE , 1990 . 78 ( 10 ): 1629 - 1636 . DOI: 10.1109/5.58356 http://doi.org/10.1109/5.58356 .

M A C MAHER , S P DEWEERTH , M A MAHOWALD , 等 . Implementing neural architectures using analog VLSI circuits . IEEE Transactions on Circuits and Systems , 1989 . 36 ( 5 ): 643 - 652 . DOI: 10.1109/31.31311 http://doi.org/10.1109/31.31311 .

C POSCH . Bio-inspired vision . Journal of Instrumentation , 2012 . 7 C01054 DOI: 10.1088/1748-0221/7/01/C01054 http://doi.org/10.1088/1748-0221/7/01/C01054 DOI: 10.1088/1748-0221/7/01/C01054 http://doi.org/10.1088/1748-0221/7/01/C01054 .

BOAHEN K A. Communicating neuronal ensembles between neuromorphic chips[M]//LANDE T S. Neuromorphic Systems Engineering . Boston: Springer, 1998: 229-259.

K A BOAHEN . Point-to-point connectivity between neuromorphic chips using address events . IEEE Transactions on Circuits and Systems Ⅱ: Analog and Digital Signal Processing , 2000 . 47 ( 5 ): 416 - 434 . DOI: 10.1109/82.842110 http://doi.org/10.1109/82.842110 .

LICHTSTEINER P, POSCH C, DELBRUCK T. A 128×128 120 dB 30 mW asynchronous vision sensor that responds to relative intensity change[C]//2006 IEEE International Solid State Circuits Conference - Digest of Technical Papers . San Francisco: IEEE, 2006: 2060-2069.

P LICHTSTEINER , C POSCH , T DELBRUCK . A 128×128 120 dB 15 μ s latency asynchronous temporal contrast vision sensor . IEEE Journal of Solid-State Circuits , 2008 . 43 ( 2 ): 566 - 576 . DOI: 10.1109/JSSC.2007.914337 http://doi.org/10.1109/JSSC.2007.914337 .

POSCH C, WOHLGENANNT R, MATOLIN D, et al . A temporal contrast IR vision sensor[C]// Proceedings of SPIE 7100 Optical Design and Engineering Ⅲ. Glasgow: SPIE, 2008: 71002A.

C POSCH , D MATOLIN , R WOHLGENANNT . A QVGA 143 dB dynamic range frame-free PWM image sensor with lossless pixel-level video compression and time-domain CDS . IEEE Journal of Solid-State Circuits , 2011 . 46 ( 1 ): 259 - 275 . DOI: 10.1109/JSSC.2010.2085952 http://doi.org/10.1109/JSSC.2010.2085952 .

BERNER R, BRANDLI C, YANG M H, et al . A 240×180 10 mW 12 μs latency sparse-output vision sensor for mobile applications[C]//2013 Symposium on VLSI Circuits . Kyoto: IEEE, 2013: C186-C187.

C BRANDLI , R BERNER , M H YANG , 等 . A 240×180 130 dB 3 s latency global shutter spatiotemporal vision sensor . IEEE Journal of Solid-State Circuits , 2014 . 49 ( 10 ): 2333 - 2341 . DOI: 10.1109/JSSC.2014.2342715 http://doi.org/10.1109/JSSC.2014.2342715 .

D D CHO , T J LEE . A review of bioinspired vision sensors and their applications . Sensors and Materials , 2015 . 27 ( 6 ): 447 - 463 . http://smartsearch.nstl.gov.cn/paper_detail.html?id=73a033131ce7d3c5a106fa377c1f2d44 http://smartsearch.nstl.gov.cn/paper_detail.html?id=73a033131ce7d3c5a106fa377c1f2d44 .

马艳阳 , 叶梓豪 , 刘坤华 , 等 . 基于事件相机的定位与建图算法: 综述 . 自动化学报 , 2021 . 47 ( 7 ): 1484 - 1494 . DOI: 10.16383/j.aas.c190550 http://doi.org/10.16383/j.aas.c190550 .

Y Y MA , Z H YE , K H LIU , 等 . Event-based visual localization and mapping algorithms: a survey . Acta Automatica Sinica , 2021 . 47 ( 7 ): 1484 - 1494 . DOI: 10.16383/j.aas.c190550 http://doi.org/10.16383/j.aas.c190550 .

闫昌达 , 王霞 , 左一凡 , 等 . 基于事件相机的可视化及降噪算法 . 北京航空航天大学学报 , 2021 . 47 ( 2 ): 342 - 350 . DOI: 10.13700/j.bh.1001-5965.2020.0192 http://doi.org/10.13700/j.bh.1001-5965.2020.0192 .

C D YAN , X WANG , Y F ZUO , 等 . Visualization and noise reduction algorithm based on event camera . Journal of Beijing University of Aeronautics and Astronautics , 2021 . 47 ( 2 ): 342 - 350 . DOI: 10.13700/j.bh.1001-5965.2020.0192 http://doi.org/10.13700/j.bh.1001-5965.2020.0192 .

周小龙 , 刘倩倩 , 产思贤 , 等 . 基于事件相机的视觉跟踪算法综述 . 小型微型计算机系统 , 2020 . 41 ( 11 ): 2325 - 2332 . DOI: 10.3969/j.issn.1000-1220.2020.11.015 http://doi.org/10.3969/j.issn.1000-1220.2020.11.015 .

X L ZHOU , Q Q LIU , S X CHAN , 等 . Event camera-based visual tracking algorithms: a survey . Journal of Chinese Computer Systems , 2020 . 41 ( 11 ): 2325 - 2332 . DOI: 10.3969/j.issn.1000-1220.2020.11.015 http://doi.org/10.3969/j.issn.1000-1220.2020.11.015 .

桑永胜 , 李仁昊 , 李耀仟 , 等 . 神经形态视觉传感器及其应用研究 . 物联网学报 , 2019 . 3 ( 4 ): 63 - 71 . DOI: 10.11959/j.issn.2096-3750.2019.00133 http://doi.org/10.11959/j.issn.2096-3750.2019.00133 .

Y S SANG , R H LI , Y Q LI , 等 . Research on neuromorphic vision sensor and its applications . Chinese Journal on Internet of Things , 2019 . 3 ( 4 ): 63 - 71 . DOI: 10.11959/j.issn.2096-3750.2019.00133 http://doi.org/10.11959/j.issn.2096-3750.2019.00133 .

RAMESH B, ZHANG S H, LEE Z W, et al . Long-term object tracking with a moving event camera[C]// BMVC . Newcastle, 2018: 241.

TEDALDI D, GALLEGO G, MUEGGLER E, et al . Feature detection and tracking with the dynamic and active-pixel vision sensor (DAVIS)[C]//2016 Second International Conference on Event-based Control , Communication , and Signal Processing (EBCCSP). Krakow: IEEE, 2016: 1-7.

MITROKHIN A, FERMVLLER C, PARAMESHWARA C, et al . Event-based moving object detection and tracking[C]//2018 IEEE / RSJ International Conference on Intelligent Robots and Systems ( IROS ). Madrid: IEEE, 2018: 1-9.

BOETTIGER J P. A comparative evaluation of the detection and tracking capability between novel event-based and conventional frame-based sensors[D]. Ohio: The Air Force Air University, 2020.

WANG Y H, IDOUGHI R, HEIDRICH W. Stereo event-based particle tracking velocimetry for 3D fluid flow reconstruction[C]// Proceedings of 16 th European Conference on Computer Vision . Glasgow: Springer, 2020: 36-53.

DAMIEN J, HUBERT K, FREDERIC C. Convolutional neural network for detection and classification with event-based data[C]// Proceedings of the 14 th International Joint Conference on Computer Vision , Imaging and Computer Graphics Theory and Applications . Prague: VISAPP, 2019: 200-208.

MESSIKOMMER N, GEHRIG D, LOQUERCIO A, et al . Event-based asynchronous sparse convolutional networks[C]// Proceedings of 16 th European Conference on Computer Vision . Glasgow: Springer, 2020: 415-431.

PEROT E, DE TOURNEMIRE P, NITTI D, et al . Learning to detect objects with a 1 Megapixel event camera[J]. arXiv preprint arXiv : 2009.13436, 2020.

WANG Q Y, ZHANG Y X, YUAN J S, et al . Space-time event clouds for gesture recognition: from RGB cameras to event cameras[C] //2019 IEEE Winter Conference on Applications of Computer Vision ( WACV ). Waikoloa: IEEE, 2019: 1826-1835.

MONFORTE M, ARRIANDIAGA A, GLOVER A, et al . Exploiting event cameras for spatio-temporal prediction of fast-changing trajectories[C]//2020 2 nd IEEE International Conference on Artificial Intelligence Circuits and Systems ( AICAS ). Genova: IEEE, 2020: 108-112.

ZHU A Z, ATANASOV N, DANIILIDIS K. Event-based visual inertial odometry[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition ( CVPR ). Honolulu: IEEE, 2017: 5816-5824.

CENSI A, STRUBEL J, BRANDLI C, et al . Low-latency localization by active LED markers tracking using a dynamic vision sensor[C]//2013 IEEE / RSJ International Conference on Intelligent Robots and Systems . Tokyo: IEEE, 2013: 891-898.

HUANG J. Asynchronous high-speed feature extraction image sensor[D]. Singapore: Nanyang Technological University, 2018.

Views

750

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

No data

Related Author

Ying-hong FANG

Wei XU

Yong-jie PIAO

Ru-peng FENG

Liang-liang ZHENG

Related Institution

Changchun institute of Optics， Fine Mechanics and Physics， Chinese Academy of Sciences.， University of Chinese Academy of Sciences

AI问答

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China 130033 Postal code：130033
Tel：0431-86176059 Email：yjxs@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰