Single-event upsets fault tolerance of convolutional neural networks based on dropout algorithm

doi:10.7523/j.issn.2095-6134.2021.05.016

Abstract

Abstract: Space radiation interference especially the single event upset (SEU) effect can make a great impact on the normal and stable operation of the neural network chips, which can lead to the random bit flips of the weight parameters stored in the SRAM, and thus change the values of the neural weight parameters and the accuracy of the neural network chip outputs. In this paper, we analyze the commonly used radiation resistant methods, and try to overcome the problem of hardware consumption, recovery time, and processing speed. After the software simulation of the neural network accuracy with the different ratios of weight parameter errors, we adopt the dropout algorithm to construct the novel network framework to avoid the decline of accuracy. The algorithm can mask the neurons affected by SEU with some probability. The simulation experiment results show that this method can promote the accuracy of the neural network affected by SEU.

Key words: SEU interference, SRAM memory, neural network chip, weight parameter

CLC Number:

TP183

QIAN Huan, XIE Zhuochen, LIANG Xuwen. Single-event upsets fault tolerance of convolutional neural networks based on dropout algorithm[J]. Journal of University of Chinese Academy of Sciences, 2021, 38(5): 712-719.

References

[1] Akopyan F, Sawada J, Cassidy A, et al. Truenorth:design and tool flow of a 65 mW 1 million neuron programmable neurosynaptic chip[J]. IEEE Transactions on Computer-aided Design of Integrated Circuits and Systems, 2015, 34(10):1537-1557.
[2] Xu L, Yu R, Wang L, et al. Memway:in-memorywaylaying acceleration for practical rowhammer attacks against binaries[J]. Tsinghua Science and Technology, 2019, 24(5):535-545.
[3] Kim Y, Daly R, Kim J, et al. Flipping bits in memory without accessing them:an experimental study of DRAM disturbance errors[J]. ACM SIGARCH Computer Architecture News, 2014, 42(3):361-372.
[4] 王润泽,王颖,杨栋毅.大规模FFT并行计算中2维SRAM的设计[J].中国科学院研究生院学报, 2008, 25(1):123-128.
[5] Zhu M, Song N, Pan X. Mitigation and experiment on neutron induced single-event upsets in SRAM-based FPGAs[J]. IEEE Transactions on Nuclear Science, 2013, 60(4):3063-3073.
[6] Abraham J A, Siewiorek D P. An algorithm for the accurate reliability evaluation of triple modular redundancy networks[J]. IEEE Transactions on Computers, 1974, 100(7):682-692.
[7] Gils V. A triple modular redundancy technique providing multiple-bit error protection without using extra redundancy[J]. IEEE Transactions on Computers, 1986, 100(7):623-631.
[8] Katsarou K, Tsiatouhas Y. Soft error interception latch:double node charge sharing SEU tolerant design[J]. Electronics Letters, 2015, 51(4):330-332.
[9] 桂江华,徐睿,卓琳.基于三模冗余架构的集成电路加固设计[J].中国电子科学研究院学报,2013,8(6):643-646.
[10] Binder D, Smith E C, Holman A B. Satellite anomalies from galactic cosmic rays[J]. IEEE Transactions on Nuclear Science, 1975, 22(6):2675-2680.
[11] 薛玉雄, 曹洲, 杨世宇, 等. IDT6116单粒子敏感性评估试验技术研究[J]. 原子能科学技术, 2008, 42(1):22-27.
[12] Arechiga A P, Michaels A J. The effect of weight errors on neural networks[C]//2018 IEEE 8th Annual Computing and Communication Workshop and Conference (CCWC). IEEE, 2018:190-196.
[13] Kwon S, Lee K, Kim Y, et al. Measuring error-tolerance in SRAM architecture on hardware accelerated neural network[C]//2016 IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia). IEEE, 2016:1-4.
[14] Mukherjee S S, Emer J, Reinhardt S K. The soft error problem:an architectural perspective[C]//11th International Symposium on High-Performance Computer Architecture. IEEE, 2005:243-247.
[15] Wirthlin M J, Keller A M, McCloskey C, et al. SEU mitigation and validation of the LEON3 soft processor using triple modular redundancy for space processing[C]//Proceedings of the 2016 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays. 2016:205-214.
[16] Saleh A M, Serrano J J, Patel J H. Reliability of scrubbing recovery-techniques for memory systems[J]. IEEE Transactions on Reliability, 1990, 39(1):114-122.
[17] Agarwal A, Negahban S, Wainwright M J. A simple way to prevent neural networks from overfitting[J]. Ann Stat, 2012, 40(2):1171-1197.
[18] Xiao T, Li H, Ouyang W, et al. Learning deep feature representations with domain guided dropout for person re-identification[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016:1249-1258.
[19] Wager S, Wang S, Liang P S. Dropout training as adaptive regularization[C]//Advances in Neural Information Processing Systems. 2013:351-359.
[20] Pham V, Bluche T, Kermorvant C, et al. Dropout improves recurrent neural networks for handwriting recognition[C]//2014 14th International Conference on Frontiers in Handwriting Recognition. IEEE, 2014:285-290.
[21] Yu N, Jiao P, Zheng Y. Handwritten digits recognition base on improved LeNet5[C]//The 27th Chinese Control and Decision Conference (2015 CCDC). IEEE, 2015:4871-4875.
[22] Karlik B, Olgac A V. Performance analysis of various activation functions in generalized MLP architectures of neural networks[J]. International Journal of Artificial Intelligence and Expert Systems, 2011, 1(4):111-122.