Fusion method based on FPGA image overlay and cross-screen stitching

Quan WAN; Shao-fu LI

doi:10.37188/YJYXS20203510.1066

您当前的位置：

首页 >

文章列表页 >

Fusion method based on FPGA image overlay and cross-screen stitching

Display & Image Processing | 更新时间：2020-09-28

- Fusion method based on FPGA image overlay and cross-screen stitching
- Chinese Journal of Liquid Crystals and Displays Vol. 35, Issue 10, Pages: 1066-1072(2020)
- 作者机构：
  
  西南科技大学信息工程学院, 四川绵阳 621010
- 作者简介：
- 基金信息：
- DOI：10.37188/YJYXS20203510.1066
  CLC： TP391
- Received：23 March 2020，
  
  Revised：20 April 2020，
  
  Accepted：20 April 2020，
  
  Published：05 October 2020
- 稿件说明：
移动端阅览
Quan WAN, Shao-fu LI. Fusion method based on FPGA image overlay and cross-screen stitching[J]. Chinese journal of liquid crystals and displays, 2020, 35(10): 1066-1072.
DOI：

Quan WAN, Shao-fu LI. Fusion method based on FPGA image overlay and cross-screen stitching[J]. Chinese journal of liquid crystals and displays, 2020, 35(10): 1066-1072. DOI： 10.37188/YJYXS20203510.1066.

摘要

针对目前视频显示领域中常见的图像叠加和跨屏拼接显示方式，两种功能往往不能同时在一个系统中实现，导致应用的场景受到局限问题，提出一种多功能视频拼接系统的硬件结构设计。该系统基于双线性插值图像算法，采用流水线设计思路对算法进行优化，同时采用单片机完成视频处理前后，缩放倍数的计算与显示参数的传递，通过IIC总线发送计算出的参数到FPGA芯片执行，整个过程减少了硬件资源上的消耗，提高了算法的效率，并在Lattice的开发环境Diamond中对设计进行验证。实验结果表明，FPGA能够同时处理4路输入视频流的存储操作，输出4路任意分辨率的视频信号，在2×2的拼接矩阵中，同时实现图像叠加与跨屏拼接。该拼接系统支持最高1 920×1 080的分辨率输入，最小50×50的分辨率输出，达到预期结果。

Abstract

Aiming at the common display methods such as image superimposition and cross-screen stitching in the current video display field

the two functions often cannot be implemented in the same system at the same time

resulting in limitation of application scenarios. A hardware structure design of a multifunctional video stitching system is proposed. The system is based on a bilinear interpolation image algorithm. The pipeline design ideas are used to optimize the algorithm. At the same time

a single-chip computer is used to complete the calculation of the zoom factor and the transmission of display parameters before and after video processing. The whole process reduces the consumption of hardware resources

and also improves the efficiency of the algorithm. The design is verified in Diamond that is the development environment of Lattice. The results show that the FPGA can simultaneously process four input video stream storage operations and output four video signals of any resolution. In a 2×2 stitching matrix

the image overlay and cross-screen stitching can be achieved simultaneously. The stitching system supports a maximum resolution of 1 920×1 080 and an output with a minimum resolution of 50×50

achieving the expected result.

关键词

Keywords

references

王明宇.基于FPGA的大屏幕多投影无缝拼接系统的设计研究[D].杭州: 浙江大学, 2015.

WANGM Y. The research and design of FPGA-based large-screen Multi-projection seamless tiled display system[D]. Hangzhou: Zhejiang University, 2015. (in Chinese)

严飞 , 陆宝毅 , 刘银萍 , 等 . 实时视频流缩放系统设计 . 液晶与显示 , 2019 . 34 ( 11 ): 1124 - 1130 . http://cjlcd.lightpublishing.cn/thesisDetails?columnId=1385234&Fpath=&index=-1&l=zh http://cjlcd.lightpublishing.cn/thesisDetails?columnId=1385234&Fpath=&index=-1&l=zh .

NF YA , B Y LU , Y P LIU , 等 . Design of real-time video stream scaling system . Chinese Journal of Liquid Crystals and Displays , 2019 . 34 ( 11 ): 1124 - 1130 . http://cjlcd.lightpublishing.cn/thesisDetails?columnId=1385234&Fpath=&index=-1&l=zh http://cjlcd.lightpublishing.cn/thesisDetails?columnId=1385234&Fpath=&index=-1&l=zh .

吴以凯 , 喻金华 , 肖铁军 . 基于FPGA的视频缩放设计与实现 . 软件导刊 , 2017 . 16 ( 8 ): 83 - 85 . http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=rjdk201708025 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=rjdk201708025 .

Y K WU , J H YU , T J XIAO . Design and implementation of video scaling based on FPGA . Software Guide , 2017 . 16 ( 8 ): 83 - 85 . http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=rjdk201708025 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=rjdk201708025 .

李渊渊 , 孟伟 . 一种用硬件实现的Bayer格式图像恢复算法 . 中国光学 , 2009 . 2 ( 6 ): 538 - 542 . http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=zggxyyygxwz200906012 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=zggxyyygxwz200906012 .

Y Y LI , W MENG . Real-time hardware Bayer color image restoration algorithm and its implementation . Chinese Optics , 2009 . 2 ( 6 ): 538 - 542 . http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=zggxyyygxwz200906012 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=zggxyyygxwz200906012 .

罗潇 , 孙海江 , 陈秋萍 , 等 . Bayer格式图像的实时彩色复原 . 中国光学 , 2010 . 3 ( 2 ): 182 - 187 . http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=zggxyyygxwz201002014 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=zggxyyygxwz201002014 .

X LUO , H J SUN , Q P CHEN , 等 . Real-time demosaicing of Bayer pattern images . Chinese Optics , 2010 . 3 ( 2 ): 182 - 187 . http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=zggxyyygxwz201002014 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=zggxyyygxwz201002014 .

YEGANLI F, NAZZAL M, OZKARAMANLI H. Super-resolution using multiple structured dictionaries based on the gradient operator and bicubic interpolation[C]// Proceedings of 2016 24th Signal Processing And Communication Application Conference . Zonguldak, Turkey: IEEE, 2016.

RANGSIKUNPUM A, LEELARASMEE E, PUMRIN S. A design of sign video image expander for HDMI source using bicubic interpolation[C]// Proceedings of 2017 14th International Conference on Electrical Engineering/Electronics , Computer , Telecommunications and Information Technology . Phuket, Thailand: IEEE, 2017.

R G ZHOU , Y CHENG , D Q LIU . Quantum image scaling based on bilinear interpolation with arbitrary scaling ratio . Quantum Information Processing , 2019 . 18 ( 9 ): 267 DOI: 10.1007/s11128-019-2377-4 http://doi.org/10.1007/s11128-019-2377-4 .

罗军 , 黄启俊 , 常胜 , 等 . H.264中整数变换与量化的FPGA实现 . 中国图象图形学报 , 2011 . 16 ( 5 ): 740 - 745 . http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=zgtxtxxb-a201105007 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=zgtxtxxb-a201105007 .

J LUO , Q J HUANG , S CHANG , 等 . . A FPGA implementation of integer discrete consine transform and quantification for H.264 compression . Journal of Image and Graphics , 2011 . 16 ( 5 ): 740 - 745 . http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=zgtxtxxb-a201105007 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=zgtxtxxb-a201105007 .

刘海鹰 , 张兆杨 , 沈礼权 . 基于FPGA的H.264变换量化的高性能的硬件实现 . 中国图象图形学报 , 2006 . 11 ( 11 ): 1636 - 1639 . http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=zgtxtxxb-a200611031 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=zgtxtxxb-a200611031 .

H Y LIU , Z Y ZHANG , L Q SHEN . A high-performance hardware implementation of the h. 264 transformation and quantization oriented to FPGA . Journal of Image and Graphics , 2006 . 11 ( 11 ): 1636 - 1639 . http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=zgtxtxxb-a200611031 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=zgtxtxxb-a200611031 .

崔鹏 , 燕天天 . 融合YCbCr肤色模型与改进的Adaboost算法的人脸检测 . 哈尔滨理工大学学报 , 2018 . 23 ( 2 ): 91 - 96 . http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=heblgdxxb201802016 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=heblgdxxb201802016 .

P CUI , T T YAN . Face detection combining the YCbCr skin color model with improved Adaboost algorithm . Journal of Harbin University of Science and Technology , 2018 . 23 ( 2 ): 91 - 96 . http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=heblgdxxb201802016 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=heblgdxxb201802016 .

SHI M, YI Q M, YANG H M. Implementation of video surveillance systems based on the Linux platform[C]// Proceedings of 2017 Third Asian Conference on Defence Technology . Phuket, Thailand: IEEE, 2017.

熊文 , 王莉 , 肖健 , 等 . 一种RS485串口通信电路的高可靠性设计 . 自动化与仪器仪表 , 2017 . ( 3 ): 43 - 45 . http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=zdhyyqyb201703016 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=zdhyyqyb201703016 .

W XIONG , L WANG , J XIAO , 等 . A RS485 serial communication circuit of the high reliability design . Automation & Instrumentation , 2017 . ( 3 ): 43 - 45 . http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=zdhyyqyb201703016 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=zdhyyqyb201703016 .

卢宇峰 , 罗小华 , 俞淼 , 等 . 基于LUT实时图像矫正的行缓存优化 . 复旦学报:自然科学版 , 2016 . 55 ( 4 ): 418 - 424 . http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=fdxb201604003 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=fdxb201604003 .

Y F LU , X H LUO , M YU , 等 . Line buffer reduction for LUT-based real-time image rectification . Journal of Fudan University:Natural Science , 2016 . 55 ( 4 ): 418 - 424 . http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=fdxb201604003 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=fdxb201604003 .

肖建波 . 电视信号传输和处理延时的一种测量方法 . 现代电视技术 , 2016 . ( 9 ): 142 - 144 . http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=xddsjs201609030 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=xddsjs201609030 .

J B XIAO . A method for measuring the delay of TV signal transmission and processing . Advanced Television Engineering , 2016 . ( 9 ): 142 - 144 . http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=xddsjs201609030 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=xddsjs201609030 .

赵书杰, 孙书为, 雷元武.一种基于BT.1120协议的数字视频输入接口设计[C]//第二十一届计算机工程与工艺年会暨第七届微处理器技术论坛论文集.2017.

ZHAO S J, SUN S W, LEI Y W. Design of a digital video input interface based on BT.1120 protocol[C]// Proceedings of the 21st Annual Computer Engineering and Technology Conference and the 7th Microprocessor Technology Forum. 2017. (in Chinese)

NIU Y H, YU J Y, GU M, et al . A high speed virtual channel data transfer method for spacecraft application[C]// Proceedings of 2016 IEEE 13th International Conference on Signal Processing . Chengdu: IEEE, 2016.

CHEN S Y, WANG D H, SHAN R, et al . An innovative design of the DDR/DDR2 SDRAM compatible controller[C]// Proceedings of 2005 6th International Conference on ASIC . Shanghai: IEEE, 2005.

冉峰 , 李天 , 季渊 , 等 . 基于FPGA的图像缩放算法设计 . 光电子技术 , 2017 . 37 ( 1 ): 21 - 26 . http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=gdzjs201701004 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=gdzjs201701004 .

F RAN , T LI , Y JI , 等 . Design of image scaling algorithm based on FPGA . Optoelectronic Technology , 2017 . 37 ( 1 ): 21 - 26 . http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=gdzjs201701004 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=gdzjs201701004 .

谭燕林.基于FPGA的DDR2 SDRAM控制器设计[D].南京: 东南大学, 2016. http://cdmd.cnki.com.cn/Article/CDMD-10286-1016246964.htm

TAN Y L. Design of DDR2 SDRAM controller based on FPGA[D]. Nanjing: Southeast University, 2016. (in Chinese)

陈平 , 张春 , 张一山 , 等 . DDR2 SDRAM控制器IP功能测试与FPGA验证 . 微电子学 , 2016 . 46 ( 2 ): 251 - 254 . http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=wdzx201602025 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=wdzx201602025 .

P CHEN , C ZHANG , Y S ZHANG , 等 . Function simulation and FPGA verification of DDR2 SDRAM controller IP . Microelectronics , 2016 . 46 ( 2 ): 251 - 254 . http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=wdzx201602025 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=wdzx201602025 .

张秋林 , 夏靖波 , 邱婧 , 等 . 基于ARM和FPGA的双路远程视频监控系统设计 . 液晶与显示 , 2011 . 26 ( 6 ): 780 - 784 . http://cjlcd.lightpublishing.cn/thesisDetails?columnId=1692903&Fpath=&index=-1&l=zh http://cjlcd.lightpublishing.cn/thesisDetails?columnId=1692903&Fpath=&index=-1&l=zh .

Q L ZHANG , J B XIA , J QIU , 等 . Design of dual remote video monitoring system based on ARM and FPGA . Chinese Journal of Liquid Crystals and Displays , 2011 . 26 ( 6 ): 780 - 784 . http://cjlcd.lightpublishing.cn/thesisDetails?columnId=1692903&Fpath=&index=-1&l=zh http://cjlcd.lightpublishing.cn/thesisDetails?columnId=1692903&Fpath=&index=-1&l=zh .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Design of multi-channel real-time video processing system based on FPGA

Adaptive real-time video trapezoidal correction system design and implementation based on FPGA

Registration method of large field view and multi-sensor images of TDICCD cameras

FPGA Realization of Video Image Zooming Display

Related Author

Jia LIU

Wei CHEN

Yin-ping LIU

Zhao-dong DING

Fei YAN

Guan-mao ZHANG

Han-qing ZHAO

Xin QIAO

Related Institution

College of Automation， Nanjing University of Information Science & Technology

Jiangsu Collaborative Innovation Center of Atmospheric Environment and Equipment Technology （CICAEET）

College of atmospheric physics， Nanjing University of Information Science & Technology

Institute of Optoelectronics and Electromagnetic Information, School of Information Science and Engineering, Lanzhou University

Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences

AI问答

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China 130033 Postal code：130033
Tel：0431-86176059 Email：yjxs@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰