基于记忆增强和扩散模型的骨架视频异常检测*

doi:10.7523/j.ucas.2025.017

摘要/Abstract

摘要： 视频异常检测（VAD）是一种旨在识别视频中异常事件的技术,在公共安全和视频内容理解方面有广泛应用价值。传统的VAD方法主要依赖于从视频帧的整体或特定区域提取像素特征。为了降低非结构化噪音的影响,基于人体骨架VAD方法备受关注。然而在开集场景下,此类方法面临2大挑战：对异常行为的错误重构及对正常行为拓展能力的不足。针对这2问题,本文提出一种基于记忆增强与扩散模型的骨架视频异常检测框架（MEDM-SVAD）。该框架通过引入记忆增强模块扩展模型对正常样本的记忆能力,避免异常行为因重构误差较小而被误判;此外,利用扩散模型显著提高对域外正常行为的泛化能力。在HR-STC、HR-Avenue和HR-UBnormal这3个公开数据集上的实验结果表明,相较于现有最先进的MoCoDAD算法,所提方法MEDM-SVAD在AUC指标上分别取得78.5、90.1和69.7的结果,都实现不同程度的性能提升,验证了其在多种场景下的有效性与优越性。

关键词: 视频异常检测, 骨架特征, 扩散模型, 记忆增强

Abstract: Video Anomaly Detection (VAD) is a technique aimed at identifying abnormal events within videos and has significant applications in public safety and video content understanding. Traditional VAD methods primarily rely on extracting pixel features from either the entire video frame or specific regions. To reduce the impact of unstructured noise, VAD methods based on human skeletal data have garnered considerable attention. However, in open-set scenarios, these methods face two major challenges: erroneous reconstruction of abnormal behaviors and insufficient generalization to diverse normal behaviors. To address these issues, this paper proposes a skeleton-based video anomaly detection framework with memory enhancement and diffusion model (MEDM-SVAD). This framework incorporates a memory enhancement module to expand the model's memory capacity for normal samples, preventing abnormal behaviors from being misclassified due to minimal reconstruction error. Additionally, the diffusion model significantly improves the framework's ability to generalize to out-of-domain normal behaviors. Experimental results on three public datasets, HR-STC, HR-Avenue, and HR-UBnormal, show that the proposed method MEDM-SVAD achieves AUC scores of 78.5, 90.1, and 69.7, respectively, demonstrating various levels of performance improvement over the current state-of-the-art MoCoDAD algorithm, thus verifying its effectiveness and superiority across multiple scenarios.

Key words: video anomaly detection, skeleton-based features, diffusion model, memory-enhanced

中图分类号:

TP181

张栖, 李元, 周彦钊, 焦建彬. 基于记忆增强和扩散模型的骨架视频异常检测^*[J]. 中国科学院大学学报, DOI: 10.7523/j.ucas.2025.017.

ZHANG Qi, LI Yuan, ZHOU Yanzhao, JIAO Jianbin. Skeleton-based video anomaly detection with memory enhancement and diffusion model[J]. Journal of University of Chinese Academy of Sciences, DOI: 10.7523/j.ucas.2025.017.

参考文献

[1] Sultani W, Chen C, Shah M.Real-world anomaly detection in surveillance videos[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). June 18-23, 2018, Salt Lake City, UT, USA. IEEE, 2018: 6479-6488. DOI: 10.1109/CVPR.2018.00678.
[2] Hasan M, Choi J, Neumann J, et al.Learning temporal regularity in video sequences[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). June 27-30, 2016. Las Vegas, NV, USA. IEEE, 2016: 733-742. DOI: 10.1109/CVPR.2016.86.
[3] Liu W, Luo W X, Lian D Z, et al.Future frame prediction for anomaly detection - A new baseline[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). June 18-23, 2018, Salt Lake City, UT, USA. IEEE, 2018: 6536-6545. DOI: 10.1109/CVPR.2018.00684.
[4] Luo W X, Liu W, Gao S H.A revisit of sparse coding based anomaly detection in stacked RNN framework[C]//2017 IEEE International Conference on Computer Vision(ICCV). October 22-29, 2017, Venice, Italy. IEEE, 2017: 341-349. DOI: 10.1109/ICCV.2017.45.
[5] Chong Y S, Tay Y H.Abnormal event detection in videos using spatiotemporal autoencoder[C]//Advances in Neural Networks-ISNN 2017. Cham: Springer International Publishing, 2017: 189-196. DOI: 10.1007/978-3-319-59081-3_23.
[6] Xu D, Yan Y, Ricci E, et al.Detecting anomalous events in videos by learning deep representations of appearance and motion[J]. Computer Vision and Image Understanding, 2017, 156: 117-127. DOI: 10.1016/j.cviu.2016.10.010.
[7] Hinami R, Mei T, Satoh S.Joint detection and recounting of abnormal events by learning deep generic knowledge[C]//2017 IEEE International Conference on Computer Vision(ICCV). October 22-29, 2017, Venice, Italy. IEEE, 2017: 3639-3647. DOI: 10.1109/ICCV.2017.391.
[8] Coşar S, Donatiello G, Bogorny V, et al.Toward abnormal trajectory and event detection in video surveillance[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2017, 27(3): 683-695. DOI: 10.1109/TCSVT.2016.2589859.
[9] Zimek A, Schubert E, Kriegel H P.A survey on unsupervised outlier detection in high‐dimensional numerical data[J]. Statistical Analysis and Data Mining: the ASA Data Science Journal, 2012, 5(5): 363-387. DOI: 10.1002/sam.11161.
[10] Morais R, Le V, Tran T, et al.Learning regularity in skeleton trajectories for anomaly detection in videos[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). June 15-20, 2019, Long Beach, CA, USA. IEEE, 2019: 11988-11996. DOI: 10.1109/CVPR.2019.01227.
[11] Flaborea A, Collorone L, D’ Amely Di Melendugno G M, et al. Multimodal motion conditioned diffusion model for skeleton-based video anomaly detection[C]//2023 IEEE/CVF International Conference on Computer Vision(ICCV). October 1-6, 2023, Paris, France. IEEE, 2023: 10284-95. DOI: 10.1109/ICCV51070.2023.00947.
[12] Jain Y, Sharma A K, Velmurugan R, et al.PoseCVAE: anomalous human activity detection[C]//2020 25th International Conference on Pattern Recognition(ICPR). January 10-15, 2021, Milan, Italy. IEEE, 2021: 2927-2934. DOI: 10.1109/ICPR48806.2021.9412132.
[13] Markovitz A, Sharir G, Friedman I, et al.Graph embedded pose clustering for anomaly detection[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). June 13-19, 2020, Seattle, WA, USA. IEEE, 2020: 10536-10544. DOI: 10.1109/CVPR42600.2020.01055.
[14] Rodrigues R, Bhargava N, Velmurugan R, et al.Multi-timescale trajectory prediction for abnormal human activity detection[C]//2020 IEEE Winter Conference on Applications of Computer Vision(WACV). March 1-5, 2020, Snowmass Village, CO, USA. IEEE, 2020: 2626-2634. DOI: 10.1109/WACV45572.2020.9093633.
[15] Barbalau A, Ionescu R T, Georgescu M I, et al.SSMTL++: revisiting self-supervised multi-task learning for video anomaly detection[J]. Computer Vision and Image Understanding, 2023, 229: 103656. DOI: 10.1016/j.cviu.2023.103656.
[16] Flaborea A, D’ Amely di Melendugno G M, D’Arrigo S, et al. Contracting skeletal kinematics for human-related video anomaly detection[J]. Pattern Recognition, 2024, 156: 110817. DOI: 10.1016/j.patcog.2024.110817.
[17] Kanu-Asiegbu A M, Vasudevan R, Du X X. BiPOCO: Bi-directional trajectory prediction with pose constraints for pedestrian anomaly detection[EB/OL]. arXiv:2207.02281v1. (2022-07-05) [2025-03-18]. https://arxiv.org/abs/2207.02281v1.
[18] Li N J, Chang F L, Liu C S.Human-related anomalous event detection via spatial-temporal graph convolutional autoencoder with embedded long short-term memory network[J]. Neurocomputing, 2022, 490: 482-494. DOI: 10.1016/j.neucom.2021.12.023.
[19] Luo W X, Liu W, Gao S H.Normal graph: spatial temporal graph convolutional networks based prediction network for skeleton based video anomaly detection[J]. Neurocomputing, 2021, 444: 332-337. DOI: 10.1016/j.neucom.2019.12.148.
[20] 刘禹含, 吉根林, 张红苹. 基于骨架图与混合注意力的视频行人异常检测方法[J]. 计算机应用, 2024, 44(8): 2551-2557. DOI: 10.11772/j.issn.1001-9081.2023081157.
[21] 杨学存, 李杰华, 陈丽媛, 等. 基于人体骨架的扶梯乘客异常行为识别方法[J]. 安全与环境学报, 2024, 24(2): 636-643. DOI: 10.13637/j.issn.1009-6094.2022.2404.
[22] 朱高伟. 基于骨架数据的列车司机异常行为检测[J]. 计算机科学与应用, 2024, 14(7): 42-50. DOI: 10.12677/csa.2024.147162.
[23] Acsintoae A, Florescu A, Georgescu M I, et al.UBnormal: new benchmark for supervised open-set video anomaly detection[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 18-24, 2022, New Orleans, LA, USA. IEEE, 2022: 20111-20121. DOI: 10.1109/CVPR52688.2022.01951.
[24] Gong D, Liu L Q, Le V, et al.Memorizing normality to detect anomaly: memory-augmented deep autoencoder for unsupervised anomaly detection[C]//2019 IEEE/CVF International Conference on Computer Vision(ICCV). October 27-November 2, 2019, Seoul, Korea. IEEE, 2019: 1705-1714. DOI: 10.1109/ICCV.2019.00179.
[25] Nguyen T N, Meunier J.Anomaly detection in video sequence with appearance-motion correspondence[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV). October 27-November 2, 2019, Seoul, Korea. IEEE, 2019: 1273-1283. DOI: 10.1109/ICCV.2019.00136.
[26] Maas A L, Hannun A Y, Ng A Y.Rectifier nonlinearities improve neural network acoustic models[C]//2013 International Conference on Machine Learning (ICML). June 16-21, 2013, Atlanta, Georgia, USA. PMLR, 2013, 30(1): 3.
[27] Fan Y X, Wen G J, Li D R, et al.Video anomaly detection and localization via gaussian mixture fully convolutional variational autoencoder[J]. Computer Vision and Image Understanding, 2020, 195: 102920. DOI: 10.1016/j.cviu.2020.102920.
[28] Lu Y W, Kumar K M, Nabavi S S, et al.Future frame prediction using convolutional VRNN for anomaly detection[C]//2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance(AVSS). September 18-21, 2019, Taipei, China. IEEE, 2019: 1-8. DOI: 10.1109/avss.2019.8909850.
[29] Yu G, Wang S Q, Cai Z P, et al.Cloze test helps: effective video anomaly detection via learning to complete video events[C]//Proceedings of the 28th ACM International Conference on Multimedia. October 12-16, 2020, Seattle, WA, USA. ACM, 2020: 583-591. DOI: 10.1145/3394171.3413973.
[30] Liu Z A, Nie Y W, Long C J, et al.A hybrid video anomaly detection framework via memory-augmented flow reconstruction and flow-guided frame prediction[C]//2021 IEEE/CVF International Conference on Computer Vision(ICCV). October 10-17, 2021, Montreal, QC, Canada. IEEE, 2021: 13568-13577. DOI: 10.1109/ICCV48922.2021.01333.
[31] Zhang X Y, Li N Q, Li J W, et al.Unsupervised surface anomaly detection with diffusion probabilistic model[C]//2023 IEEE/CVF International Conference on Computer Vision(ICCV). October 1-6, 2023, Paris, France. IEEE, 2023: 6759-6768. DOI: 10.1109/ICCV51070.2023.00624.
[32] Ho J, Jain A, Abbeel P. Denoising diffusion probabilistic models[EB/OL]. arXiv:2006.11239v2. (2020-12-16) [2025-03-18]. https://arxiv.org/abs/2006.11239.
[33] Xiao Z S, Kreis K, Vahdat A. Tackling the generative learning trilemma with denoising diffusion GANs[EB/OL]. arXiv:2112.07804v2. (2022-04-04) [2025-03-18]. https://arxiv.org/abs/2112.07804v2.
[34] Krizhevsky A, Sutskever I, Hinton G E.ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90. DOI: 10.1145/3065386.
[35] Sofianos T, Sampieri A, Franco L, et al.Space-time-separable graph convolutional network for pose forecasting[C]//2021 IEEE/CVF International Conference on Computer Vision(ICCV). October 10-17, 2021, Montreal, QC, Canada. IEEE, 2021: 11189-11198. DOI: 10.1109/ICCV48922.2021.01102.
[36] He K M, Zhang X Y, Ren S Q, et al.Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). June 27-30, 2016, Las Vegas, NV, USA. IEEE, 2016: 770-778. DOI: 10.1109/CVPR.2016.90.
[37] Ronneberger O, Fischer P, Brox T.U-Net: convolutional networks for biomedical image segmentation[C]// Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015. October 5-9, 2015, Munich, Germany. Cham: Springer International Publishing, 2015: 234-241. DOI: 10.1007/978-3-319-24574-4_28.
[38] Shi L S, Wang L, Zhou S P, et al.Trajectory unified transformer for pedestrian trajectory prediction[C]//2023 IEEE/CVF International Conference on Computer Vision(ICCV). October 1-6, 2023, Paris, France. IEEE, 2023: 9641-9650. DOI: 10.1109/ICCV51070.2023.00887.
[39] 余保玲, 虞松坤, 孙耀然, 等. 基于DeepPose和Faster RCNN的多目标人体骨骼节点检测算法[J]. 中国科学院大学学报, 2020, 37(6): 828-834. DOI:10.7523/j.issn.2095-6134.2020.06.015.
[40] Zaremba W, Sutskever I, Vinyals O. Recurrent neural network regularization[EB/OL]. arXiv:1409.2329v5. (2015-02-19) [2025-03-18]. https://arxiv.org/abs/1409.2329.

基于记忆增强和扩散模型的骨架视频异常检测^*

Skeleton-based video anomaly detection with memory enhancement and diffusion model

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 11

编辑推荐

Metrics

本文评价

访问统计

联系我们

[1]	李元, 张栖, 朱建明, 焦建彬. 基于多层次深度模型的社交网络核心谣言传播节点识别[J]. 中国科学院大学学报, 2024, 41(1): 136-144.
[2]	朱嘉桐, 卿来云, 黄庆明. 基于双流LSTM与自监督学习的在线动作检测算法[J]. 中国科学院大学学报, 2022, 39(6): 827-835.
[3]	张知明, 李国荣, 黄庆明. 基于多专家和MDNet的视觉目标跟踪方法[J]. 中国科学院大学学报, 2022, 39(6): 836-844.
[4]	赵敏钧, 赵亚伟, 赵雅捷, 罗刚. 一种新的基于深度学习的重叠关系联合抽取模型（英文）[J]. 中国科学院大学学报, 2022, 39(2): 240-251.
[5]	吴园园, 申立勇. 基于类重叠度欠采样的不平衡模糊多类支持向量机[J]. 中国科学院大学学报, 2018, 35(4): 536-543.
[6]	王喜顺, 刘曦, 史忠植, 隋红建. 一种基于特征整合理论的物体识别模型[J]. 中国科学院大学学报, 2012, (3): 399-405.
[7]	陈肃, 罗铁坚, 许延祥. 基于信任的推荐算法的鲁棒性分析[J]. 中国科学院大学学报, 2011, 28(2): 253-261.
[8]	刘伟, 罗铁坚, 陈肃. 一种监控网络程序的信息收集方法[J]. 中国科学院大学学报, 2009, 26(6): 850-854.
[9]	夏鲁宁, 荆继武. SA-DBSCAN:一种自适应基于密度聚类算法[J]. 中国科学院大学学报, 2009, 26(4): 530-538.
[10]	王晶, 夏鲁宁, 荆继武. 一种基于密度最大值的聚类算法[J]. 中国科学院大学学报, 2009, 26(4): 539-548.
[11]	刘务华; 罗铁坚; 王文杰. 文本聚类算法的质量评价[J]. 中国科学院大学学报, 2006, 23(5): 640-646.