基于dropout算法的卷积神经网络单粒子翻转容错方法

doi:10.7523/j.issn.2095-6134.2021.05.016

中国科学院大学学报 ›› 2021, Vol. 38 ›› Issue (5): 712-719.DOI: 10.7523/j.issn.2095-6134.2021.05.016

• 计算机科学 • 上一篇

基于dropout算法的卷积神经网络单粒子翻转容错方法

钱欢, 谢卓辰, 梁旭文

中国科学院微小卫星创新研究院中国科学院微小卫星重点实验室, 上海 201203;中国科学院大学, 北京 100049

收稿日期:2019-12-11 修回日期:2020-04-08 发布日期:2021-09-13
通讯作者: 梁旭文
基金资助:
上海市青年科技英才扬帆计划项目（17YF1418200）和国家自然科学基金（91738201）资助

Single-event upsets fault tolerance of convolutional neural networks based on dropout algorithm

QIAN Huan, XIE Zhuochen, LIANG Xuwen

Key Laboratory of Microsatellites of CAS, Innovation Academy for Microsatellites, Chinese Academy of Sciences, Shanghai 201203, China;University of Chinese Academy of Sciences, Beijing 100049, China

Received:2019-12-11 Revised:2020-04-08 Published:2021-09-13

摘要/Abstract

摘要： 空间辐照干扰尤其是单粒子翻转（SEU）效应对神经网络芯片的正常稳定运行造成很大影响，它会导致存储在芯片SRAM存储器的权重参数随机发生比特位翻转，进而神经元的权重参数值发生变化，直接影响神经网络芯片输出的准确度。在分析现有的一些抗辐照干扰方法基础上，针对芯片硬件开销、恢复时间与处理速度的问题，利用软件仿真研究在不同比例权重参数出错的情况下神经网络的测试准确度，就结果准确度下降的情况，采用dropout算法构建新的网络框架，以一定概率屏蔽受到SEU影响的神经元。仿真实验结果表明，该方法可以提升受SEU干扰神经网络的准确度。

关键词: SEU干扰, SRAM存储器, 神经网络芯片, 权重参数

Abstract: Space radiation interference especially the single event upset (SEU) effect can make a great impact on the normal and stable operation of the neural network chips, which can lead to the random bit flips of the weight parameters stored in the SRAM, and thus change the values of the neural weight parameters and the accuracy of the neural network chip outputs. In this paper, we analyze the commonly used radiation resistant methods, and try to overcome the problem of hardware consumption, recovery time, and processing speed. After the software simulation of the neural network accuracy with the different ratios of weight parameter errors, we adopt the dropout algorithm to construct the novel network framework to avoid the decline of accuracy. The algorithm can mask the neurons affected by SEU with some probability. The simulation experiment results show that this method can promote the accuracy of the neural network affected by SEU.

Key words: SEU interference, SRAM memory, neural network chip, weight parameter

中图分类号:

TP183

钱欢, 谢卓辰, 梁旭文. 基于dropout算法的卷积神经网络单粒子翻转容错方法[J]. 中国科学院大学学报, 2021, 38(5): 712-719.

QIAN Huan, XIE Zhuochen, LIANG Xuwen. Single-event upsets fault tolerance of convolutional neural networks based on dropout algorithm[J]. Journal of University of Chinese Academy of Sciences, 2021, 38(5): 712-719.

参考文献

[1] Akopyan F, Sawada J, Cassidy A, et al. Truenorth:design and tool flow of a 65 mW 1 million neuron programmable neurosynaptic chip[J]. IEEE Transactions on Computer-aided Design of Integrated Circuits and Systems, 2015, 34(10):1537-1557.
[2] Xu L, Yu R, Wang L, et al. Memway:in-memorywaylaying acceleration for practical rowhammer attacks against binaries[J]. Tsinghua Science and Technology, 2019, 24(5):535-545.
[3] Kim Y, Daly R, Kim J, et al. Flipping bits in memory without accessing them:an experimental study of DRAM disturbance errors[J]. ACM SIGARCH Computer Architecture News, 2014, 42(3):361-372.
[4] 王润泽,王颖,杨栋毅.大规模FFT并行计算中2维SRAM的设计[J].中国科学院研究生院学报, 2008, 25(1):123-128.
[5] Zhu M, Song N, Pan X. Mitigation and experiment on neutron induced single-event upsets in SRAM-based FPGAs[J]. IEEE Transactions on Nuclear Science, 2013, 60(4):3063-3073.
[6] Abraham J A, Siewiorek D P. An algorithm for the accurate reliability evaluation of triple modular redundancy networks[J]. IEEE Transactions on Computers, 1974, 100(7):682-692.
[7] Gils V. A triple modular redundancy technique providing multiple-bit error protection without using extra redundancy[J]. IEEE Transactions on Computers, 1986, 100(7):623-631.
[8] Katsarou K, Tsiatouhas Y. Soft error interception latch:double node charge sharing SEU tolerant design[J]. Electronics Letters, 2015, 51(4):330-332.
[9] 桂江华,徐睿,卓琳.基于三模冗余架构的集成电路加固设计[J].中国电子科学研究院学报,2013,8(6):643-646.
[10] Binder D, Smith E C, Holman A B. Satellite anomalies from galactic cosmic rays[J]. IEEE Transactions on Nuclear Science, 1975, 22(6):2675-2680.
[11] 薛玉雄, 曹洲, 杨世宇, 等. IDT6116单粒子敏感性评估试验技术研究[J]. 原子能科学技术, 2008, 42(1):22-27.
[12] Arechiga A P, Michaels A J. The effect of weight errors on neural networks[C]//2018 IEEE 8th Annual Computing and Communication Workshop and Conference (CCWC). IEEE, 2018:190-196.
[13] Kwon S, Lee K, Kim Y, et al. Measuring error-tolerance in SRAM architecture on hardware accelerated neural network[C]//2016 IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia). IEEE, 2016:1-4.
[14] Mukherjee S S, Emer J, Reinhardt S K. The soft error problem:an architectural perspective[C]//11th International Symposium on High-Performance Computer Architecture. IEEE, 2005:243-247.
[15] Wirthlin M J, Keller A M, McCloskey C, et al. SEU mitigation and validation of the LEON3 soft processor using triple modular redundancy for space processing[C]//Proceedings of the 2016 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays. 2016:205-214.
[16] Saleh A M, Serrano J J, Patel J H. Reliability of scrubbing recovery-techniques for memory systems[J]. IEEE Transactions on Reliability, 1990, 39(1):114-122.
[17] Agarwal A, Negahban S, Wainwright M J. A simple way to prevent neural networks from overfitting[J]. Ann Stat, 2012, 40(2):1171-1197.
[18] Xiao T, Li H, Ouyang W, et al. Learning deep feature representations with domain guided dropout for person re-identification[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016:1249-1258.
[19] Wager S, Wang S, Liang P S. Dropout training as adaptive regularization[C]//Advances in Neural Information Processing Systems. 2013:351-359.
[20] Pham V, Bluche T, Kermorvant C, et al. Dropout improves recurrent neural networks for handwriting recognition[C]//2014 14th International Conference on Frontiers in Handwriting Recognition. IEEE, 2014:285-290.
[21] Yu N, Jiao P, Zheng Y. Handwritten digits recognition base on improved LeNet5[C]//The 27th Chinese Control and Decision Conference (2015 CCDC). IEEE, 2015:4871-4875.
[22] Karlik B, Olgac A V. Performance analysis of various activation functions in generalized MLP architectures of neural networks[J]. International Journal of Artificial Intelligence and Expert Systems, 2011, 1(4):111-122.

基于dropout算法的卷积神经网络单粒子翻转容错方法

Single-event upsets fault tolerance of convolutional neural networks based on dropout algorithm

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 7

编辑推荐

Metrics

本文评价

访问统计

联系我们

[1]	王慧玲, 谢卓辰, 梁旭文. 单粒子翻转对神经网络的影响分析与优化[J]. 中国科学院大学学报, 2021, 38(6): 832-840.
[2]	庄子俊, 袁晓兵, 裴俊, 王国辉, 刘建坡. 基于无监督表达学习的森林地貌特征建模及林火易发性评估[J]. 中国科学院大学学报, 0, (): 1-1.
[3]	朱思捷, 雷斌, 吴一戎. 基于稠密卷积神经网络的遥感图像自动色彩校正[J]. 中国科学院大学学报, 2019, 36(1): 93-100.
[4]	于凤玲, 周扬, 陈建宏, 周汉陵. DELTA势阱改进QPSO优化BP算法及其应用[J]. 中国科学院大学学报, 2015, 32(3): 416-421.
[5]	周尚明, 刘定生. 模糊控制和神经网络在现场总线控制系统中的应用[J]. 中国科学院大学学报, 2000, 17(2): 81-88.
[6]	霍鑫怡, 李焱磊, 陈龙永, 张福博, 孙巍. 基于卷积注意力和胶囊网络的SAR少样本目标识别方法^*[J]. 中国科学院大学学报, , (): 2-2.
[7]	李翔, 王艳, 李宝清. 基于FVC-CNN模型的野外车辆声信号分类^*[J]. 中国科学院大学学报, , (): 18-18.