单粒子翻转对神经网络的影响分析与优化

doi:10.7523/j.issn.2095-6134.2021.06.014

中国科学院大学学报 ›› 2021, Vol. 38 ›› Issue (6): 832-840.DOI: 10.7523/j.issn.2095-6134.2021.06.014

单粒子翻转对神经网络的影响分析与优化

王慧玲, 谢卓辰, 梁旭文

中国科学院微小卫星创新研究院, 上海 201203;中国科学院大学, 北京 100049

收稿日期:2019-12-23 修回日期:2020-05-05 发布日期:2021-11-16
通讯作者: 梁旭文
基金资助:
上海市青年科技英才扬帆计划项目（17YF1418200）和国家自然科学基金（91738201）资助

Analysis and optimization of single event upset on neural network

WANG Huiling, XIE Zhuochen, LIANG Xuwen

Innovation Academy for Microsatellites, Chinese Academy of Sciences, Shanghai 201203, China;University of Chinese Academy of Sciences, Beijing 100049, China

Received:2019-12-23 Revised:2020-05-05 Published:2021-11-16

摘要/Abstract

摘要： DNN芯片作为星载芯片应用到卫星系统中时会受到太空辐照的影响，其中单粒子翻转对存储单元的干扰会使得存储器单元的参数出现错误，该错误映射到神经网络中会造成神经网络最后的输出结果出现偏差。结合单粒子翻转概率模型，对用于网络推断的神经网络的权值参数进行注错后分析实验结果准确率，从激活函数的非线性特性分析并通过实验验证具有双边抑制效果的函数容错能力更强。进一步在网络卷积层后加入BN层和在训练过程中考虑L2正则化提高网络的容错能力，并通过实验验证其可行性。

关键词: 单粒子翻转错误概率模型, 深度神经网络, 激活函数, 网络容错

Abstract: When the DNN (deep neural network) chip is used in a satellite system as a space-borne chip, it will be affected by space radiation. The interference of single event upset (SEU) on the storage unit will cause the parameters of the memory unit to be wrong. The error mapping to the neural network will affect the output results. This paper analyzes the accuracy of the network inference which combines the SEU probability model to inject the error on the network weight parameters. From the analysis of the nonlinear characteristics of the activation function and experimental verification, it is found that the activation function with bilateral inhibition is more fault-tolerant. Furthermore, we add the BN layer after the network convolution layer and consider L2 regularization during training to improve the network's fault tolerance, and verify its feasibility through experiments.

Key words: single event upset error model, DNN, activation function, fault tolerance

中图分类号:

TP183

王慧玲, 谢卓辰, 梁旭文. 单粒子翻转对神经网络的影响分析与优化[J]. 中国科学院大学学报, 2021, 38(6): 832-840.

WANG Huiling, XIE Zhuochen, LIANG Xuwen. Analysis and optimization of single event upset on neural network[J]. Journal of University of Chinese Academy of Sciences, 2021, 38(6): 832-840.

参考文献

[1] Krizhevsky A, Sutskever I, Hinton G. Imagenet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6):84-90.
[2] Graves A, Mohamed A R, Hinton G. Speech recognition with deep recurrent neural networks[C]//2013 IEEE International Conference on Acoustics, Speech and Signal Processing. May 26-31, 2013, Vancouver, BC, Canada. IEEE, 2013:6645-6649.
[3] Hauser S C, Headley W C, Michaels A J. Signal detection effects on deep neural networks utilizing raw IQ for modulation classification[C]//2017 Military Communications Conference (MILCOM). October 23-25, 2017. Baltimore, MD, USA. IEEE, 2017:121-127.
[4] Duzellier S. Radiation effects on electronic devices in space[J].Aerospace Science and Technology, 2005, 9(1):93-99.
[5] 刘永杰.SOI FinFET器件与组合逻辑电路单粒子效应研究[D].西安:西安电子科技大学,2015.
[6] Akkerman A, Barak J, Yitzhak N M. Role of elastic scattering of protons, muons, and electrons in inducing single-event upsets[J]. IEEE Transactions on Nuclear Science,2017,64(10):2648-2660.
[7] Caron P, Inguimbert C, Artola L, et al. Physical mechanisms inducing electron single-event upset[J].IEEE Transactions on Nuclear Science,2018,65(8):1759-1767.
[8] Jin Y,Huan Y X,Chu H M,et al.TMR group coding method for optimized SEU and MBU tolerant memory design[C]//2018 IEEE International Symposium on Circuits and Systems (ISCAS). May 27-30, 2018, Florence, Italy. IEEE, 2018:1-5.
[9] 汪波,王佳,刘伟鑫,等. 星载ASIC芯片单粒子效应检测及在轨翻转率预估[J].半导体技术, 2019,44(8):728-734.
[10] Chen Y J, Chen T S, Xu Z W, et al. Diannao family:energy-efficient hardware accelerators for machine learning[J].Communications of the ACM, 2016,59(11):105-112.
[11] Chen T S, Du Z D, Sun N H. et al. DianNao:a small footprint high-throughput accelerator for ubiquitous machine learning[C]//Proceedings of the 19th International Conference on Architectural Support for Programming Languages and Operating Systems. Salt Lake City, Utah, USA. New York, NY, USA:ACM, 2014:269-283.
[12] 张建华,王娜,王琦,等.基于SRAM型FPGA抗单粒子措施研究[J].空间电子技术,2015,12(2):83-85, 91.
[13] 杨阳,陶建中,万书芹,等.基于抗辐照技术的DDS电路设计与实现[J].电子与封装,2019,19(8):24-28.
[14] Lee M, Hwang K, Sung W. Fault tolerance analysis of digital feed-forward deep neural networks[C]//2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). May 4-9, 2014, Florence, Italy. IEEE, 2014:5031-5035.
[15] Assoum A, Radi N E, Velazco R, et al. Robustness against SEU of an artificial neural network space application[J].IEEE Transactions on Nuclear Science, 1996,43(3):973-978.
[16] Arechiga A P, Michaels A J. The robustness of modern deep learning architectures against single event upset errors[C]//2018 IEEE High Performance extreme Computing Conference (HPEC). Spptember 25-27, 2018, Waltham, MA, USA. IEEE, 2018:1-6.
[17] Kwon S, Lee K, Kim Y, et al. Measuring error-tolerance in SRAM architecture on hardware accelerated neural network[C]//2016 IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia). October 26-28, 2016, Seoul, koreal (South). IEEE, 2016:1-4.
[18] Arechiga A P,Michaels A J. The effect of weight errors on neural networks[C]//2018 IEEE 8th Annual Computing and Communication Workshop and Conference (CCWC). January 8-10, 2018, Las Vegas, NV, USA. IEEE, 2018:190-196.
[19] Barraza N B, Cernuschi-Frías B, Cernuschi F. A probabilistic model for grouped events analysis[C]//1995 IEEE International Conference on Systems, Man and Cybernetics, October 22-25, 1995, Vancouver, BC, Canada. IEEE, 1995:3386-3390.
[20] Baeg S, Wen S J, Wong R. SRAM interleaving distance selection with a soft error failure model[J]. IEEE Transactions on Nuclear Science,2009,56(4):2111-2118.
[21] Vinod N, Hinton G. Rectified linear units improve restricted Boltzmann machines[C]//International Conference on Machine Learning, 2010:807-814.
[22] Devlin J,Chang M W,Lee K, et al. BERT:pre-training of deep bidirectional transformers for language understanding[J].Association for Computational Linguistics,2019,1:4171-4186.
[23] 李航.统计学习方法[M].北京:清华大学出版社,2012.
[24] Bishop C M. Training with noise is equivalent to Tikhonov regularization[J]. Neural Computation, 1995,7(1):108-116.
[25] Alwani M, Chen H, Ferdman M, et al. Fused-layer CNN accelerators[C]//2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO). October 15-19, 2016, Taipei, Taiwan, China. IEEE, 2016:1-12.

单粒子翻转对神经网络的影响分析与优化

Analysis and optimization of single event upset on neural network

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 2

编辑推荐

Metrics

本文评价

访问统计

联系我们

[1]	王健飞, 张卫强, 刘加. 基于多状态跳转模型的场景独立音频事件检测方法[J]. 中国科学院大学学报, 2019, 36(2): 218-225.
[2]	杨建斌, 张卫强, 刘加. 深度神经网络自适应中基于身份认证向量的归一化方法[J]. 中国科学院大学学报, 2017, 34(5): 633-639.