Analysis and optimization of single event upset on neural network

doi:10.7523/j.issn.2095-6134.2021.06.014

Abstract

Abstract: When the DNN (deep neural network) chip is used in a satellite system as a space-borne chip, it will be affected by space radiation. The interference of single event upset (SEU) on the storage unit will cause the parameters of the memory unit to be wrong. The error mapping to the neural network will affect the output results. This paper analyzes the accuracy of the network inference which combines the SEU probability model to inject the error on the network weight parameters. From the analysis of the nonlinear characteristics of the activation function and experimental verification, it is found that the activation function with bilateral inhibition is more fault-tolerant. Furthermore, we add the BN layer after the network convolution layer and consider L2 regularization during training to improve the network's fault tolerance, and verify its feasibility through experiments.

Key words: single event upset error model, DNN, activation function, fault tolerance

CLC Number:

TP183

WANG Huiling, XIE Zhuochen, LIANG Xuwen. Analysis and optimization of single event upset on neural network[J]. Journal of University of Chinese Academy of Sciences, 2021, 38(6): 832-840.

References

[1] Krizhevsky A, Sutskever I, Hinton G. Imagenet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6):84-90.
[2] Graves A, Mohamed A R, Hinton G. Speech recognition with deep recurrent neural networks[C]//2013 IEEE International Conference on Acoustics, Speech and Signal Processing. May 26-31, 2013, Vancouver, BC, Canada. IEEE, 2013:6645-6649.
[3] Hauser S C, Headley W C, Michaels A J. Signal detection effects on deep neural networks utilizing raw IQ for modulation classification[C]//2017 Military Communications Conference (MILCOM). October 23-25, 2017. Baltimore, MD, USA. IEEE, 2017:121-127.
[4] Duzellier S. Radiation effects on electronic devices in space[J].Aerospace Science and Technology, 2005, 9(1):93-99.
[5] 刘永杰.SOI FinFET器件与组合逻辑电路单粒子效应研究[D].西安:西安电子科技大学,2015.
[6] Akkerman A, Barak J, Yitzhak N M. Role of elastic scattering of protons, muons, and electrons in inducing single-event upsets[J]. IEEE Transactions on Nuclear Science,2017,64(10):2648-2660.
[7] Caron P, Inguimbert C, Artola L, et al. Physical mechanisms inducing electron single-event upset[J].IEEE Transactions on Nuclear Science,2018,65(8):1759-1767.
[8] Jin Y,Huan Y X,Chu H M,et al.TMR group coding method for optimized SEU and MBU tolerant memory design[C]//2018 IEEE International Symposium on Circuits and Systems (ISCAS). May 27-30, 2018, Florence, Italy. IEEE, 2018:1-5.
[9] 汪波,王佳,刘伟鑫,等. 星载ASIC芯片单粒子效应检测及在轨翻转率预估[J].半导体技术, 2019,44(8):728-734.
[10] Chen Y J, Chen T S, Xu Z W, et al. Diannao family:energy-efficient hardware accelerators for machine learning[J].Communications of the ACM, 2016,59(11):105-112.
[11] Chen T S, Du Z D, Sun N H. et al. DianNao:a small footprint high-throughput accelerator for ubiquitous machine learning[C]//Proceedings of the 19th International Conference on Architectural Support for Programming Languages and Operating Systems. Salt Lake City, Utah, USA. New York, NY, USA:ACM, 2014:269-283.
[12] 张建华,王娜,王琦,等.基于SRAM型FPGA抗单粒子措施研究[J].空间电子技术,2015,12(2):83-85, 91.
[13] 杨阳,陶建中,万书芹,等.基于抗辐照技术的DDS电路设计与实现[J].电子与封装,2019,19(8):24-28.
[14] Lee M, Hwang K, Sung W. Fault tolerance analysis of digital feed-forward deep neural networks[C]//2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). May 4-9, 2014, Florence, Italy. IEEE, 2014:5031-5035.
[15] Assoum A, Radi N E, Velazco R, et al. Robustness against SEU of an artificial neural network space application[J].IEEE Transactions on Nuclear Science, 1996,43(3):973-978.
[16] Arechiga A P, Michaels A J. The robustness of modern deep learning architectures against single event upset errors[C]//2018 IEEE High Performance extreme Computing Conference (HPEC). Spptember 25-27, 2018, Waltham, MA, USA. IEEE, 2018:1-6.
[17] Kwon S, Lee K, Kim Y, et al. Measuring error-tolerance in SRAM architecture on hardware accelerated neural network[C]//2016 IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia). October 26-28, 2016, Seoul, koreal (South). IEEE, 2016:1-4.
[18] Arechiga A P,Michaels A J. The effect of weight errors on neural networks[C]//2018 IEEE 8th Annual Computing and Communication Workshop and Conference (CCWC). January 8-10, 2018, Las Vegas, NV, USA. IEEE, 2018:190-196.
[19] Barraza N B, Cernuschi-Frías B, Cernuschi F. A probabilistic model for grouped events analysis[C]//1995 IEEE International Conference on Systems, Man and Cybernetics, October 22-25, 1995, Vancouver, BC, Canada. IEEE, 1995:3386-3390.
[20] Baeg S, Wen S J, Wong R. SRAM interleaving distance selection with a soft error failure model[J]. IEEE Transactions on Nuclear Science,2009,56(4):2111-2118.
[21] Vinod N, Hinton G. Rectified linear units improve restricted Boltzmann machines[C]//International Conference on Machine Learning, 2010:807-814.
[22] Devlin J,Chang M W,Lee K, et al. BERT:pre-training of deep bidirectional transformers for language understanding[J].Association for Computational Linguistics,2019,1:4171-4186.
[23] 李航.统计学习方法[M].北京:清华大学出版社,2012.
[24] Bishop C M. Training with noise is equivalent to Tikhonov regularization[J]. Neural Computation, 1995,7(1):108-116.
[25] Alwani M, Chen H, Ferdman M, et al. Fused-layer CNN accelerators[C]//2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO). October 15-19, 2016, Taipei, Taiwan, China. IEEE, 2016:1-12.