A lightweight FPGA image preprocessing accelerator scheme for visual navigation

doi:10.7523/j.ucas.2024.063

Abstract

Abstract: An image preprocessing accelerator scheme based on lightweight and low-cost FPGA has been proposed in this paper to meet the accelerated processing requirements of the visual navigation image frontend. Through efficient pipeline design and parallel processing technology, the designed accelerator integrates key functions such as histogram equalization, FAST feature points detection, and multi-source sensor data time synchronization. This solution solves technical difficulties such as achieving multifunctional integration, meeting real-time requirements, balancing cost and performance, synchronizing multi-source sensor information time, and achieving software hardware collaborative design under limited hardware resources. The proposed solution is based on Xilinx's Zynq-7000 series lightweight FPGA implementation, which greatly reduces image processing latency while achieving low cost. When the FPGA operates at a frequency of 160MHz, it achieves a processing speed of 150 frames per second for 1280×720 images, providing a low-cost and high-performance visual navigation image front-end acceleration solution.

Key words: Key Words: image accelerator, histogram equalization, feature point extraction, time synchronization

CLC Number:

TP911.73

XUE Renkui, ZHANG Jie, LI Bin, LI Meng, WU Yang. A lightweight FPGA image preprocessing accelerator scheme for visual navigation[J]. Journal of University of Chinese Academy of Sciences, DOI: 10.7523/j.ucas.2024.063.

References

[1] Floreano D, Wood R J.Science, technology and the future of small autonomous drones[J]. Nature, 2015, 521(7553): 460-466. DOI: 10.1038/nature14542.
[2] 房德国, 王伟, 李自然, 等. VIO-SLAM综述[J]. 电光与控制, 2020, 27(12): 58-62, 100. DOI: 10.3969/j.issn.1671-637X.2020.12.013.
[3] 田野, 陈宏巍, 王法胜, 等. 室内移动机器人的SLAM算法综述[J]. 计算机科学, 2021, 48(9): 223-234. DOI: 10.11896/jsjkx.200700152.
[4] Peng T, Zhang D N, Liu R X, et al.Evaluating the power efficiency of visual SLAM on embedded GPU systems[C]//2019 IEEE National Aerospace and Electronics Conference (NAECON). Dayton, OH, USA. IEEE, 2019: 117-121. DOI: 10.1109/NAECON46414.2019.9058059.
[5] Zhao Y, Xiong Z, Duan S Q, et al.Improved ORB based image registration acceleration algorithm in visual-inertial navigation system[C]//2020 Chinese Automation Congress (CAC). Shanghai, China. IEEE, 2020: 5714-5718. DOI: 10.1109/CAC51589.2020.9326928.
[6] 郑泰皓, 方子彬. 视觉SLAM特征与提取算法综述[J]. 计算机时代, 2020(7): 16-21. DOI: 10.16644/j.cnki.cn33-1094/tp.2020.07.005.
[7] Vemulapati V, Chen D M.FSLAM: An efficient and accurate SLAM accelerator on SoC FPGAs[C]//2022 International Conference on Field-Programmable Technology (ICFPT). Hong Kong, China. IEEE, 2022: 1-9. DOI: 10.1109/ICFPT56656.2022.9974562.
[8] Taranco R, Arnau J M, González A.A low-power hardware accelerator for ORB feature extraction in self-driving cars[C]//2021 IEEE 33rd International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD). Belo Horizonte, Brazil. IEEE, 2021: 11-21. DOI: 10.1109/SBAC-PAD53543.2021.00013.
[9] Liu Y J, Wu X F.Parallel and pipelining design of SLAM feature detection algorithm in hardware[C]//2021 IEEE 16th Conference on Industrial Electronics and Applications (ICIEA). Chengdu, China. IEEE, 2021: 1388-1393. DOI: 10.1109/ICIEA51954.2021.9516430.
[10] Li J Y, Liu Y, Huang K, et al.An FPGA-based high-throughput keypoint detection accelerator using convolutional neural network for mobile robot applications[C]//2022 IEEE Asia Pacific Conference on Postgraduate Research in Microelectronics and Electronics (PrimeAsia). Shenzhen, China. IEEE, 2022: 81-84. DOI: 10.1109/PrimeAsia56064.2022.10104021.
[11] Xu Z L, Yu J C, Yu C, et al.CNN-based feature-point extraction for real-time visual SLAM on embedded FPGA[C]//2020 IEEE 28th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM). Fayetteville, AR, USA. IEEE, 2020: 33-37. DOI: 10.1109/FCCM48280.2020.00014.
[12] 丁畅, 董丽丽, 许文海. “直方图” 均衡化图像增强技术研究综述[J]. 计算机工程与应用, 2017, 53(23): 12-17. DOI: 10.3778/j.issn.1002-8331.1710-0031.
[13] Gauch J M.Investigations of image contrast space defined by variations on histogram equalization[J]. CVGIP: Graphical Models and Image Processing, 1992, 54(4): 269-280. DOI: 10.1016/1049-9652(92)90074-8.
[14] Zuiderveld K.Contrast limited adaptive histogram equalization[M]//Graphics Gems. Amsterdam: Elsevier, 1994: 474-485. DOI: 10.1016/b978-0-12-336156-1.50061-6.
[15] 彭波, 王一鸣. 低照度图像增强算法的研究与实现[J]. 计算机应用, 2007, 27(8): 2001-2003. DOI: 10.1360/crad20071222.
[16] Rosten E, Porter R, Drummond T.Faster and better: a machine learning approach to corner detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(1): 105-119. DOI: 10.1109/TPAMI.2008.275.
[17] Rosten E, Drummond T.Machine learning for high-speed corner detection[M]//Leonardis A, Bischof H, Pinz A, eds. Computer Vision - ECCV 2006. Berlin, Heidelberg: Springer Berlin Heidelberg, 2006: 430-443. DOI: 10.1007/11744023_34.
[18] 程彪, 黄鲁. 自适应阈值FAST特征点检测算法的FPGA实现[J]. 信息技术与网络安全, 2018, 37(10): 82-86. DOI: 10.19358/j.issn.2096-5133.2018.10.019.
[19] Defossen M. XAPP894(v1.0.1) D-PHY solutions[EB/OL]. Xilinx, (2021-02-01)[2024-04-20]. https://download.csdn.net/download/weixin_42681774/27489049.
[20] UG871(v2014.1) Vivado design suite tutorial: high-level synthesis[EB/OL]. Xilinx,(2014-05-06)[2024-04-20]. https://www.doc88.com/p-3127859166761.html.
[21] 刘延飞, 杨铁阡, 李琪, 等. 多总线地面测试设备中双SDRAM控制器的设计[J]. 电讯技术, 2011, 51(4): 31-34. DOI: 10.3969/j.issn.1001-893x.2011.04.007.
[22] 何凯, 梁蓓, 杨发顺. 基于Vivado HLS的求取特征点图像坐标的设计[J]. 电子科技, 2018, 31(4): 87-90. DOI: 10.16180/j.cnki.issn1007-7820.2018.04.023.