一种应用于视觉导航的轻量级FPGA图像预处理加速器方案*

doi:10.7523/j.ucas.2024.063

中国科学院大学学报

一种应用于视觉导航的轻量级FPGA图像预处理加速器方案^*

薛仁魁¹, 张杰, 李斌³, 李萌¹, 吴洋¹

1 北京东方计量测试研究所,北京 100086;
2 中国科学院国家天文台,北京 100101; 3 北京眸星科技有限公司,北京 100015

收稿日期:2024-02-23 修回日期:2024-06-06 发布日期:2024-06-24
通讯作者: ^† E-mail:zhangjie05@mails. ucas. ac. cn
基金资助:
^* 国家自然科学基金项目（12273074）资助

A lightweight FPGA image preprocessing accelerator scheme for visual navigation

XUE Renkui¹, ZHANG Jie², LI Bin³, LI Meng¹, WU Yang¹

1 Beijing Orient Institute of Measurement and Test, Beijing 100086, China;
2 National Astronomical Observatories, Chinese Academy of Sciences, Beijing 100101, China;
3 Beijing Eyestar Technology Co., Ltd, Beijing 100015, China

Received:2024-02-23 Revised:2024-06-06 Published:2024-06-24

摘要/Abstract

摘要： 本文针对视觉导航图像前端的加速处理需求,提出了一种基于轻量级、低成本FPGA的图像预处理加速器方案。通过高效的流水线设计以及并行处理技术,所设计的加速器集成了直方图均衡化、FAST特征点检测以及多源传感器数据时间同步等关键功能。该方案解决了有限硬件资源下实现多功能集成、满足实时性要求、平衡成本与性能、多源传感器信息时间同步,以及实现软硬件协同设计等技术难点。本文提出的方案基于Xilinx公司Zynq-7000系列轻量级FPGA实现,在实现低成本的同时大大降低了图像处理延迟。当FPGA以160MHz的频率运行时,对于1280×720的图像实现了150帧/秒的处理速度,提供了一种低成本、高性能的视觉导航图像前端加速解决方案。

关键词: 图像加速器, 直方图均衡化, 特征点提取, 时间同步

Abstract: An image preprocessing accelerator scheme based on lightweight and low-cost FPGA has been proposed in this paper to meet the accelerated processing requirements of the visual navigation image frontend. Through efficient pipeline design and parallel processing technology, the designed accelerator integrates key functions such as histogram equalization, FAST feature points detection, and multi-source sensor data time synchronization. This solution solves technical difficulties such as achieving multifunctional integration, meeting real-time requirements, balancing cost and performance, synchronizing multi-source sensor information time, and achieving software hardware collaborative design under limited hardware resources. The proposed solution is based on Xilinx's Zynq-7000 series lightweight FPGA implementation, which greatly reduces image processing latency while achieving low cost. When the FPGA operates at a frequency of 160MHz, it achieves a processing speed of 150 frames per second for 1280×720 images, providing a low-cost and high-performance visual navigation image front-end acceleration solution.

Key words: Key Words: image accelerator, histogram equalization, feature point extraction, time synchronization

中图分类号:

TP911.73

薛仁魁, 张杰, 李斌, 李萌, 吴洋. 一种应用于视觉导航的轻量级FPGA图像预处理加速器方案^*[J]. 中国科学院大学学报, DOI: 10.7523/j.ucas.2024.063.

XUE Renkui, ZHANG Jie, LI Bin, LI Meng, WU Yang. A lightweight FPGA image preprocessing accelerator scheme for visual navigation[J]. Journal of University of Chinese Academy of Sciences, DOI: 10.7523/j.ucas.2024.063.

参考文献

[1] Floreano D, Wood R J.Science, technology and the future of small autonomous drones[J]. Nature, 2015, 521(7553): 460-466. DOI: 10.1038/nature14542.
[2] 房德国, 王伟, 李自然, 等. VIO-SLAM综述[J]. 电光与控制, 2020, 27(12): 58-62, 100. DOI: 10.3969/j.issn.1671-637X.2020.12.013.
[3] 田野, 陈宏巍, 王法胜, 等. 室内移动机器人的SLAM算法综述[J]. 计算机科学, 2021, 48(9): 223-234. DOI: 10.11896/jsjkx.200700152.
[4] Peng T, Zhang D N, Liu R X, et al.Evaluating the power efficiency of visual SLAM on embedded GPU systems[C]//2019 IEEE National Aerospace and Electronics Conference (NAECON). Dayton, OH, USA. IEEE, 2019: 117-121. DOI: 10.1109/NAECON46414.2019.9058059.
[5] Zhao Y, Xiong Z, Duan S Q, et al.Improved ORB based image registration acceleration algorithm in visual-inertial navigation system[C]//2020 Chinese Automation Congress (CAC). Shanghai, China. IEEE, 2020: 5714-5718. DOI: 10.1109/CAC51589.2020.9326928.
[6] 郑泰皓, 方子彬. 视觉SLAM特征与提取算法综述[J]. 计算机时代, 2020(7): 16-21. DOI: 10.16644/j.cnki.cn33-1094/tp.2020.07.005.
[7] Vemulapati V, Chen D M.FSLAM: An efficient and accurate SLAM accelerator on SoC FPGAs[C]//2022 International Conference on Field-Programmable Technology (ICFPT). Hong Kong, China. IEEE, 2022: 1-9. DOI: 10.1109/ICFPT56656.2022.9974562.
[8] Taranco R, Arnau J M, González A.A low-power hardware accelerator for ORB feature extraction in self-driving cars[C]//2021 IEEE 33rd International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD). Belo Horizonte, Brazil. IEEE, 2021: 11-21. DOI: 10.1109/SBAC-PAD53543.2021.00013.
[9] Liu Y J, Wu X F.Parallel and pipelining design of SLAM feature detection algorithm in hardware[C]//2021 IEEE 16th Conference on Industrial Electronics and Applications (ICIEA). Chengdu, China. IEEE, 2021: 1388-1393. DOI: 10.1109/ICIEA51954.2021.9516430.
[10] Li J Y, Liu Y, Huang K, et al.An FPGA-based high-throughput keypoint detection accelerator using convolutional neural network for mobile robot applications[C]//2022 IEEE Asia Pacific Conference on Postgraduate Research in Microelectronics and Electronics (PrimeAsia). Shenzhen, China. IEEE, 2022: 81-84. DOI: 10.1109/PrimeAsia56064.2022.10104021.
[11] Xu Z L, Yu J C, Yu C, et al.CNN-based feature-point extraction for real-time visual SLAM on embedded FPGA[C]//2020 IEEE 28th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM). Fayetteville, AR, USA. IEEE, 2020: 33-37. DOI: 10.1109/FCCM48280.2020.00014.
[12] 丁畅, 董丽丽, 许文海. “直方图” 均衡化图像增强技术研究综述[J]. 计算机工程与应用, 2017, 53(23): 12-17. DOI: 10.3778/j.issn.1002-8331.1710-0031.
[13] Gauch J M.Investigations of image contrast space defined by variations on histogram equalization[J]. CVGIP: Graphical Models and Image Processing, 1992, 54(4): 269-280. DOI: 10.1016/1049-9652(92)90074-8.
[14] Zuiderveld K.Contrast limited adaptive histogram equalization[M]//Graphics Gems. Amsterdam: Elsevier, 1994: 474-485. DOI: 10.1016/b978-0-12-336156-1.50061-6.
[15] 彭波, 王一鸣. 低照度图像增强算法的研究与实现[J]. 计算机应用, 2007, 27(8): 2001-2003. DOI: 10.1360/crad20071222.
[16] Rosten E, Porter R, Drummond T.Faster and better: a machine learning approach to corner detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(1): 105-119. DOI: 10.1109/TPAMI.2008.275.
[17] Rosten E, Drummond T.Machine learning for high-speed corner detection[M]//Leonardis A, Bischof H, Pinz A, eds. Computer Vision - ECCV 2006. Berlin, Heidelberg: Springer Berlin Heidelberg, 2006: 430-443. DOI: 10.1007/11744023_34.
[18] 程彪, 黄鲁. 自适应阈值FAST特征点检测算法的FPGA实现[J]. 信息技术与网络安全, 2018, 37(10): 82-86. DOI: 10.19358/j.issn.2096-5133.2018.10.019.
[19] Defossen M. XAPP894(v1.0.1) D-PHY solutions[EB/OL]. Xilinx, (2021-02-01)[2024-04-20]. https://download.csdn.net/download/weixin_42681774/27489049.
[20] UG871(v2014.1) Vivado design suite tutorial: high-level synthesis[EB/OL]. Xilinx,(2014-05-06)[2024-04-20]. https://www.doc88.com/p-3127859166761.html.
[21] 刘延飞, 杨铁阡, 李琪, 等. 多总线地面测试设备中双SDRAM控制器的设计[J]. 电讯技术, 2011, 51(4): 31-34. DOI: 10.3969/j.issn.1001-893x.2011.04.007.
[22] 何凯, 梁蓓, 杨发顺. 基于Vivado HLS的求取特征点图像坐标的设计[J]. 电子科技, 2018, 31(4): 87-90. DOI: 10.16180/j.cnki.issn1007-7820.2018.04.023.

一种应用于视觉导航的轻量级FPGA图像预处理加速器方案^*

A lightweight FPGA image preprocessing accelerator scheme for visual navigation

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 1

编辑推荐

Metrics

本文评价

访问统计

联系我们