1.中国科学院 长春光学精密机械与物理研究所, 吉林 长春 130033
2.中国科学院大学, 北京 100049
3.拉彭兰塔-拉赫蒂理工大学 工程科学学院 计算机视觉与模式识别实验室, 芬兰 拉赫蒂15210
扫 描 看 全 文
SONG Dong-han, WANG Bin, ZHU You-qiang, et al. Fourier ptychography based on multi-scale feature fusion network. [J]. Chinese Journal of Liquid Crystals and Displays 37(11):1476-1487(2022)
SONG Dong-han, WANG Bin, ZHU You-qiang, et al. Fourier ptychography based on multi-scale feature fusion network. [J]. Chinese Journal of Liquid Crystals and Displays 37(11):1476-1487(2022) DOI: 10.37188/CJLCD.2022-0094.
傅里叶叠层成像是一种实现光学系统高分辨率、大视场成像的技术。传统FP方法的高分辨率重建过程需要较高的孔径重叠率,导致采集图像数量较多,采样效率低。此外,FP重建算法的复杂度高,重建时间长。针对以上问题,本文结合深度学习,提出一种基于多尺度特征融合网络的傅里叶叠层成像算法,通过改进的特征金字塔卷积神经网络,能够从稀疏采样的低分辨振幅图像中提取特征信息并进行融合,实现超分辨的复图像重建。实验结果表明,在相同采样条件下,与传统方法相比,本文提出的深度学习算法提高了图像重建的质量,减少了90%以上的重建时间,并且对高斯噪声的鲁棒性较高。所提出的方法能够将相邻频域子孔径间的重叠率从50%降低至25%,减少50%的采集图像数量,大幅提高采样效率。
Fourier Ptychography (FP) is a technology of achieving high-resolution, large field-of-view imaging of optical system. However, the high-resolution reconstruction based on traditional FP methods requires a high aperture overlap ratio, resulting in a large number of captured images and low sampling efficiency. In addition, the FP reconstruction algorithm has high complexity and long reconstruction time. Aiming at solving these problems of the FP, this paper proposes a deep learning algorithm based on multi-scale feature fusion network. Through the improved feature pyramid module, the feature information can be extracted from multiple low-resolution images captured by the FP imaging system, and the information is fused to achieve super-resolution reconstruction. Experimental results show that compared with traditional methods, the deep learning algorithm proposed in this paper improves the quality of image reconstruction, reduces the reconstruction time by 90%, and is more robust to Gaussian noise. In addition, the proposed method can reduce the overlap ratio between sub-apertures from 50% to 25% in frequency domain, and reduce the number of captured images by 50%, greatly improving the sampling efficiency.
计算成像傅里叶叠层成像特征金字塔稠密连接通道注意力
computational imagingfourier ptychographyfeature pyramiddense connectivitychannel attention
LOHMANN A W, DORSCH R G, MENDLOVIC D, et al. Space-bandwidth product of optical signals and systems [J]. JOSA A, 1996, 13(3): 470-473. doi: 10.1364/josaa.13.000470http://dx.doi.org/10.1364/josaa.13.000470
孙佳嵩,张玉珍,陈钱,等.傅里叶叠层显微成像技术:理论、发展和应用[J].光学学报,2016,36(10):1011005. doi: 10.3788/aos201636.1011005http://dx.doi.org/10.3788/aos201636.1011005
SUN J S, ZHANG Y Z, CHEN Q, et al. Fourier ptychographic microscopy: theory, advances, and applications [J]. Acta Optica Sinica, 2016, 36(10): 1011005. (in Chinese). doi: 10.3788/aos201636.1011005http://dx.doi.org/10.3788/aos201636.1011005
张韶辉,周国城,崔柏岐,等.傅里叶叠层显微成像模型、算法及系统研究综述[J].激光与光电子学进展,2021,58(14):1400001.
ZHANG S H, ZHOU G C, CUI B Q, et al. Review of Fourier ptychographic microscopy: models, algorithms, and systems [J]. Laser & Optoelectronics Progress, 2021, 58(14): 1400001. (in Chinese)
ZHENG G A, HORSTMEYER R, YANG C. Wide-field, high-resolution Fourier ptychographic microscopy [J]. Nature Photonics, 2013, 7(9): 739-745. doi: 10.1038/nphoton.2013.187http://dx.doi.org/10.1038/nphoton.2013.187
ZHENG G A. Fourier Ptychographic Imaging: A MATLAB Tutorial [M]. San Rafael: Morgan & Claypool Publishers, 2016: 10-22. doi: 10.1088/978-1-6817-4273-1http://dx.doi.org/10.1088/978-1-6817-4273-1
CHUNG J, OU X Z, KULKARNI R P, et al. Counting white blood cells from a blood smear using Fourier ptychographic microscopy [J]. PLoS One, 2015, 10(7): e0133489. doi: 10.1371/journal.pone.0133489http://dx.doi.org/10.1371/journal.pone.0133489
WILLIAMS A J, CHUNG J, OU X Z, et al. Fourier ptychographic microscopy for filtration-based circulating tumor cell enumeration and analysis [J]. Journal of Biomedical Optics, 2014, 19(6): 066007. doi: 10.1117/1.jbo.19.6.066007http://dx.doi.org/10.1117/1.jbo.19.6.066007
HORSTMEYER R, OU X Z, ZHENG G A, et al. Digital pathology with Fourier ptychography [J]. Computerized Medical Imaging and Graphics, 2015, 42: 38-43. doi: 10.1016/j.compmedimag.2014.11.005http://dx.doi.org/10.1016/j.compmedimag.2014.11.005
ZHENG G A, SHEN C, JIANG S W, et al. Concept, implementations and applications of Fourier ptychography [J]. Nature Reviews Physics, 2021, 3(3): 207-223. doi: 10.1038/s42254-021-00280-yhttp://dx.doi.org/10.1038/s42254-021-00280-y
HOLLOWAY J, ASIF M S, SHARMA M K, et al. Toward long-distance subdiffraction imaging using coherent camera arrays [J]. IEEE Transactions on Computational Imaging, 2016, 2(3): 251-265. doi: 10.1109/tci.2016.2557067http://dx.doi.org/10.1109/tci.2016.2557067
TIAN L, LI X, RAMCHANDRAN K, et al. Multiplexed coded illumination for Fourier ptychography with an LED array microscope [J]. Biomedical Optics Express, 2014, 5(7): 2376-2389. doi: 10.1364/boe.5.002376http://dx.doi.org/10.1364/boe.5.002376
KAPPELER A, GHOSH S, HOLLOWAY J, et al. Ptychnet: CNN based Fourier ptychography [C]//2017 IEEE International Conference on Image Processing (ICIP). Beijing, China: IEEE, 2017: 1712-1716. doi: 10.1109/icip.2017.8296574http://dx.doi.org/10.1109/icip.2017.8296574
BIAN L H, SUO J L, ZHENG G A, et al. Fourier ptychographic reconstruction using Wirtinger flow optimization [J]. Optics Express, 2015, 23(4): 4856-4866. doi: 10.1364/oe.23.004856http://dx.doi.org/10.1364/oe.23.004856
ZHANG J Z, XU T F, SHEN Z Y, et al. Fourier ptychographic microscopy reconstruction with multiscale deep residual network [J]. Optics Express, 2019, 27(6): 8612-8625. doi: 10.1364/oe.27.008612http://dx.doi.org/10.1364/oe.27.008612
JIANG S W, GUO K K, LIAO J, et al. Solving Fourier ptychographic imaging problems via neural network modeling and TensorFlow [J]. Biomedical Optics Express, 2018, 9(7): 3306-3319. doi: 10.1364/boe.9.003306http://dx.doi.org/10.1364/boe.9.003306
陈献明,王阿川,王春艳.基于深度学习的木材表面缺陷图像检测[J].液晶与显示,2019,34(9):879-887. doi: 10.3788/yjyxs20193409.0879http://dx.doi.org/10.3788/yjyxs20193409.0879
CHEN X M, WANG A C, WANG C Y. Image detection of wood surface defects based on deep learning [J]. Chinese Journal of Liquid Crystals and Displays, 2019, 34(9): 879-887. (in Chinese). doi: 10.3788/yjyxs20193409.0879http://dx.doi.org/10.3788/yjyxs20193409.0879
刘杨帆,曹立华,李宁,等.基于YOLOv4的空间红外弱目标检测[J].液晶与显示,2021,36(4):615-623. doi: 10.37188/CJLCD.2020-0227http://dx.doi.org/10.37188/CJLCD.2020-0227
LIU Y F, CAO L H, LI N, et al. Detection of space infrared weak target based on YOLOv4 [J]. Chinese Journal of Liquid Crystals and Displays, 2021, 36(4): 615-623. (in Chinese). doi: 10.37188/CJLCD.2020-0227http://dx.doi.org/10.37188/CJLCD.2020-0227
LEDIG C, THEIS L, HUSZÁR F, et al. Photo-realistic single image super-resolution using a generative adversarial network [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA: IEEE, 2017: 4681-4690. doi: 10.1109/cvpr.2017.19http://dx.doi.org/10.1109/cvpr.2017.19
LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA: IEEE, 2017: 2117-2125. doi: 10.1109/cvpr.2017.106http://dx.doi.org/10.1109/cvpr.2017.106
HUANG G, LIU Z, VAN DER MAATEN L, et al. Densely connected convolutional networks [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA: IEEE, 2017: 4700-4708. doi: 10.1109/cvpr.2017.243http://dx.doi.org/10.1109/cvpr.2017.243
HU J, SHEN L, SUN G. Squeeze-and-excitation networks [C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake City, UT, USA: IEEE, 2018: 7132-7141. doi: 10.1109/cvpr.2018.00745http://dx.doi.org/10.1109/cvpr.2018.00745
SHI W Z, CABALLERO J, HUSZÁR F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network [C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA: IEEE, 2016: 1874-1883. doi: 10.1109/cvpr.2016.207http://dx.doi.org/10.1109/cvpr.2016.207
BEVILACQUA M, ROUMY A, GUILLEMOT C, et al. Low-complexity single-image super-resolution based on nonnegative neighbor embedding [C]//British Machine Vision Conference. Surrey, UK: BMVC, 2012. doi: 10.5244/c.26.135http://dx.doi.org/10.5244/c.26.135
ZEYDE R, ELAD M, PROTTER M. On single image scale-up using sparse-representations [C]//Proceedings of the 7th International Conference on Curves and Surfaces. Avignon, France: Springer, 2010: 711-730.
MARTIN D, FOWLKES C, TAL D, et al. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics [C]//Proceedings Eighth IEEE International Conference on Computer Vision. Vancouver, BC, Canada: IEEE, 2001: 416-423. doi: 10.1109/iccv.2001.937568http://dx.doi.org/10.1109/iccv.2001.937568
HUANG J B, SINGH A, AHUJA N. Single image super-resolution from transformed self-exemplars [C]//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, MA, USA: IEEE, 2015: 5197-5206. doi: 10.1109/cvpr.2015.7299156http://dx.doi.org/10.1109/cvpr.2015.7299156
DONG S Y, BIAN Z C, SHIRADKAR R, et al. Sparsely sampled Fourier ptychography [J]. Optics Express, 2014, 22(5): 5455-5464. doi: 10.1364/oe.22.005455http://dx.doi.org/10.1364/oe.22.005455
沙浩,刘阳哲,张永兵.基于深度学习的傅里叶叠层成像技术[J].激光与光电子学进展,2021,58(18):1811020.
SHA H, LIU Y Z, ZHANG Y B. Fourier ptychography based on deep learning [J]. Laser & Optoelectronics Progress, 2021, 58(18): 1811020. (in Chinese)
0
Views
111
下载量
0
CSCD
Publicity Resources
Related Articles
Related Author
Related Institution