基于记忆增强和扩散模型的骨架视频异常检测
收稿日期: 2024-11-14
修回日期: 2025-04-02
网络出版日期: 2025-04-16
基金资助
中国科学院战略先导A项目(XDA27000000)
Skeleton-based video anomaly detection with memory enhancement and diffusion model
Received date: 2024-11-14
Revised date: 2025-04-02
Online published: 2025-04-16
传统的视频异常检测（VAD）方法主要依赖于从视频帧的整体或特定区域提取像素特征。为了降低非结构化噪声的影响，基于人体骨架的VAD方法备受关注。然而在开集场景下，此类方法面临2大挑战：对异常行为的错误重构及对正常行为拓展能力的不足。针对这个问题，提出一种基于记忆增强与扩散模型的骨架视频异常检测框架（MEDM-SVAD）。该框架通过引入记忆增强模块扩展模型对正常样本的记忆能力，避免异常行为因重构误差较小而被误判；此外，利用扩散模型显著提高对域外正常行为的泛化能力。在HR-STC、HR-Avenue和HR-UBnormal这3个公开数据集上的实验结果表明，相较于现有最先进的MoCoDAD算法，MEDM-SVAD在AUC指标上分别取得78.5%、90.1%和69.7%的结果，均有不同程度的性能提升，验证了其在多种场景下的有效性与优越性。
张栖 , 李元 , 周彦钊 , 焦建彬 . 基于记忆增强和扩散模型的骨架视频异常检测[J]. 中国科学院大学学报, 2026 , 43(5) : 677 -686 . DOI: 10.7523/j.ucas.2025.017
Video anomaly detection （VAD） is a technique aimed at identifying abnormal events within videos and has significant applications in public safety and video content understanding. Traditional VAD methods primarily rely on extracting pixel features from either the entire video frame or specific regions. To reduce the impact of unstructured noise， VAD methods based on human skeletal data have garnered considerable attention. However， in open-set scenarios， these methods face two major challenges： erroneous reconstruction of abnormal behaviors and insufficient generalization to diverse normal behaviors. To address these issues， this paper proposes a skeleton-based video anomaly detection framework with memory enhancement and diffusion model （MEDM-SVAD）. This framework incorporates a memory enhancement module to expand the model’s memory capacity for normal samples， preventing abnormal behaviors from being misclassified due to minimal reconstruction error. Additionally， the diffusion model significantly improves the framework’s ability to generalize to out-of-domain normal behaviors. Experimental results on three public datasets， HR-STC， HR-Avenue， and HR-UBnormal， show that the proposed method MEDM-SVAD achieves AUC scores of 78.5%， 90.1%， and 69.7%， respectively， demonstrating various levels of performance improvement over the current state-of-the-art MoCoDAD algorithm， thus verifying its effectiveness and superiority across multiple scenarios.
| [1] | Sultani W， Chen C， Shah M. Real-world anomaly detection in surveillance videos［C］//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR）. June 18-23， 2018， Salt Lake City， UT， USA. IEEE， 2018： 6479-6488. DOI： 10.1109/CVPR.2018.00678 . |
| [2] | Hasan M， Choi J， Neumann J， et al. Learning temporal regularity in video sequences［C］//2016 IEEE Conference on Computer Vision and Pattern Recognition（CVPR）. June 27-30， 2016. Las Vegas， NV， USA. IEEE， 2016： 733-742. DOI： 10.1109/CVPR.2016.86 . |
| [3] | Liu W， Luo W X， Lian D Z， et al. Future frame prediction for anomaly detection： a new baseline［C］//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR）. June 18-23， 2018， Salt Lake City， UT， USA. IEEE， 2018： 6536-6545. DOI： 10.1109/CVPR.2018.00684 . |
| [4] | Luo W X， Liu W， Gao S H. A revisit of sparse coding based anomaly detection in stacked RNN framework［C］//2017 IEEE International Conference on Computer Vision（ICCV）. October 22-29， 2017， Venice， Italy. IEEE， 2017： 341-349. DOI： 10.1109/ICCV.2017.45 . |
| [5] | Chong Y S， Tay Y H. Abnormal event detection in videos using spatiotemporal autoencoder［C］//Advances in Neural Networks-ISNN 2017. Cham： Springer International Publishing， 2017： 189-196. DOI： 10.1007/978-3-319-59081-3_23 . |
| [6] | Xu D， Yan Y， Ricci E， et al. Detecting anomalous events in videos by learning deep representations of appearance and motion［J］. Computer Vision and Image Understanding， 2017， 156： 117-127. DOI： 10.1016/j.cviu.2016.10.010 . |
| [7] | Hinami R， Mei T， Satoh S. Joint detection and recounting of abnormal events by learning deep generic knowledge［C］//2017 IEEE International Conference on Computer Vision（ICCV）. October 22-29， 2017， Venice， Italy. IEEE， 2017： 3639-3647. DOI： 10.1109/ICCV.2017.391 . |
| [8] | Co?ar S， Donatiello G， Bogorny V， et al. Toward abnormal trajectory and event detection in video surveillance［J］. IEEE Transactions on Circuits and Systems for Video Technology， 2017， 27（3）： 683-695. DOI： 10.1109/TCSVT.2016.2589859 . |
| [9] | Zimek A， Schubert E， Kriegel H P. A survey on unsupervised outlier detection in high‐dimensional numerical data［J］. Statistical Analysis and Data Mining： the ASA Data Science Journal， 2012， 5（5）： 363-387. DOI： 10.1002/sam.11161 . |
| [10] | Morais R， Le V， Tran T， et al. Learning regularity in skeleton trajectories for anomaly detection in videos［C］//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR）. June 15-20， 2019， Long Beach， CA， USA. IEEE， 2019： 11988-11996. DOI： 10.1109/CVPR.2019.01227 . |
| [11] | Flaborea A， Collorone L， D’ Amely Di Melendugno G M， et al. Multimodal motion conditioned diffusion model for skeleton-based video anomaly detection［C］//2023 IEEE/CVF International Conference on Computer Vision（ICCV）. October 1-6， 2023， Paris， France. IEEE， 2023： 10284-10295. DOI： 10.1109/ICCV51070.2023.00947 . |
| [12] | Jain Y， Sharma A K， Velmurugan R， et al. PoseCVAE： anomalous human activity detection［C］//2020 25th International Conference on Pattern Recognition（ICPR）. January 10-15， 2021， Milan， Italy. IEEE， 2021： 2927-2934. DOI： 10.1109/ICPR48806.2021.9412132 . |
| [13] | Markovitz A， Sharir G， Friedman I， et al. Graph embedded pose clustering for anomaly detection［C］//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR）. June 13-19， 2020， Seattle， WA， USA. IEEE， 2020： 10536-10544. DOI： 10.1109/CVPR42600.2020.01055 . |
| [14] | Rodrigues R， Bhargava N， Velmurugan R， et al. Multi-timescale trajectory prediction for abnormal human activity detection［C］//2020 IEEE Winter Conference on Applications of Computer Vision（WACV）. March 1-5， 2020， Snowmass Village， CO， USA. IEEE， 2020： 2626-2634. DOI： 10.1109/WACV45572.2020.9093633 . |
| [15] | Barbalau A， Ionescu R T， Georgescu M I， et al. SSMTL++： revisiting self-supervised multi-task learning for video anomaly detection［J］. Computer Vision and Image Understanding， 2023， 229： 103656. DOI： 10.1016/j.cviu.2023.103656 . |
| [16] | Flaborea A， D’ Amely di Melendugno G M， D’Arrigo S， et al. Contracting skeletal kinematics for human-related video anomaly detection［J］. Pattern Recognition， 2024， 156： 110817. DOI： 10.1016/j.patcog.2024.110817 . |
| [17] | Kanu-Asiegbu A M， Vasudevan R， Du X X. BiPOCO： Bi-directional trajectory prediction with pose constraints for pedestrian anomaly detection［EB/OL］. arXiv 2022：2207.02281v1. （2022-07-05） ［2025-03-18］. . |
| [18] | Li N J， Chang F L， Liu C S. Human-related anomalous event detection via spatial-temporal graph convolutional autoencoder with embedded long short-term memory network［J］. Neurocomputing， 2022， 490： 482-494. DOI： 10.1016/j.neucom.2021.12.023 . |
| [19] | Luo W X， Liu W， Gao S H. Normal graph： spatial temporal graph convolutional networks based prediction network for skeleton based video anomaly detection［J］. Neurocomputing， 2021， 444： 332-337. DOI： 10.1016/j.neucom.2019.12.148 . |
| [20] | 刘禹含， 吉根林， 张红苹. 基于骨架图与混合注意力的视频行人异常检测方法［J］. 计算机应用， 2024， 44（8）： 2551-2557. DOI： 10.11772/j.issn.1001-9081.2023081157 . |
| [21] | 杨学存， 李杰华， 陈丽媛， 等. 基于人体骨架的扶梯乘客异常行为识别方法［J］. 安全与环境学报， 2024， 24（2）： 636-643. DOI： 10.13637/j.issn.1009-6094.2022.2404 . |
| [22] | 朱高伟. 基于骨架数据的列车司机异常行为检测［J］. 计算机科学与应用， 2024， 14（7）： 42-50. DOI： 10.12677/csa.2024.147162 . |
| [23] | Acsintoae A， Florescu A， Georgescu M I， et al. UBnormal： new benchmark for supervised open-set video anomaly detection［C］//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. June 18-24， 2022， New Orleans， LA， USA. IEEE， 2022： 20111-20121. DOI： 10.1109/CVPR52688.2022.01951 . |
| [24] | Gong D， Liu L Q， Le V， et al. Memorizing normality to detect anomaly： memory-augmented deep autoencoder for unsupervised anomaly detection［C］//2019 IEEE/CVF International Conference on Computer Vision（ICCV）. October 27-November 2， 2019， Seoul， Korea. IEEE， 2019： 1705-1714. DOI： 10.1109/ICCV.2019.00179 . |
| [25] | Nguyen T N， Meunier J. Anomaly detection in video sequence with appearance-motion correspondence［C］//2019 IEEE/CVF International Conference on Computer Vision （ICCV）. October 27-November 2， 2019， Seoul， Korea. IEEE， 2019： 1273-1283. DOI： 10.1109/ICCV.2019.00136 . |
| [26] | Maas A L， Hannun A Y， Ng A Y. Rectifier nonlinearities improve neural network acoustic models［C］//2013 International Conference on Machine Learning （ICML）. June 16-21， 2013， Atlanta， Georgia， USA. PMLR， 2013， 30（1）： 3. |
| [27] | Fan Y X， Wen G J， Li D R， et al. Video anomaly detection and localization via gaussian mixture fully convolutional variational autoencoder［J］. Computer Vision and Image Understanding， 2020， 195： 102920. DOI： 10.1016/j.cviu.2020.102920 . |
| [28] | Lu Y W， Kumar K M， Nabavi S S， et al. Future frame prediction using convolutional VRNN for anomaly detection［C］//2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance（AVSS）. September 18-21， 2019， Taipei， China. IEEE， 2019： 1-8. DOI： 10.1109/avss.2019.8909850 . |
| [29] | Yu G， Wang S Q， Cai Z P， et al. Cloze test helps： effective video anomaly detection via learning to complete video events［C］//Proceedings of the 28th ACM International Conference on Multimedia. October 12-16， 2020， Seattle， WA， USA. ACM， 2020： 583-591. DOI： 10.1145/3394171.3413973 . |
| [30] | Liu Z A， Nie Y W， Long C J， et al. A hybrid video anomaly detection framework via memory-augmented flow reconstruction and flow-guided frame prediction［C］//2021 IEEE/CVF International Conference on Computer Vision（ICCV）. October 10-17， 2021， Montreal， QC， Canada. IEEE， 2021： 13568-13577. DOI： 10.1109/ICCV48922.2021.01333 . |
| [31] | Zhang X Y， Li N Q， Li J W， et al. Unsupervised surface anomaly detection with diffusion probabilistic model［C］//2023 IEEE/CVF International Conference on Computer Vision（ICCV）. October 1-6， 2023， Paris， France. IEEE， 2023： 6759-6768. DOI： 10.1109/ICCV51070.2023.00624 . |
| [32] | Ho J， Jain A， Abbeel P. Denoising diffusion probabilistic models［EB/OL］. arXiv 2020： 2006.11239v2. （2020-12-16） ［2025-03-18］. . |
| [33] | Xiao Z S， Kreis K， Vahdat A. Tackling the generative learning trilemma with denoising diffusion GANs［EB/OL］. arXiv 2022： 2112.07804v2. （2022-04-04） ［2025-03-18］. . |
| [34] | Krizhevsky A， Sutskever I， Hinton G E. ImageNet classification with deep convolutional neural networks［J］. Communications of the ACM， 2017， 60（6）： 84-90. DOI： 10.1145/3065386 . |
| [35] | Sofianos T， Sampieri A， Franco L， et al. Space-time-separable graph convolutional network for pose forecasting［C］//2021 IEEE/CVF International Conference on Computer Vision（ICCV）. October 10-17， 2021， Montreal， QC， Canada. IEEE， 2021： 11189-11198. DOI： 10.1109/ICCV48922.2021.01102 . |
| [36] | He K M， Zhang X Y， Ren S Q， et al. Deep residual learning for image recognition［C］//2016 IEEE Conference on Computer Vision and Pattern Recognition（CVPR）. June 27-30， 2016， Las Vegas， NV， USA. IEEE， 2016： 770-778. DOI： 10.1109/CVPR.2016.90 . |
| [37] | Ronneberger O， Fischer P， Brox T. U-Net： convolutional networks for biomedical image segmentation［C］// Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015. October 5-9， 2015， Munich， Germany. Cham： Springer International Publishing， 2015： 234-241. DOI： 10.1007/978-3-319-24574-4_28 . |
| [38] | Shi L S， Wang L， Zhou S P， et al. Trajectory unified transformer for pedestrian trajectory prediction［C］//2023 IEEE/CVF International Conference on Computer Vision（ICCV）. October 1-6， 2023， Paris， France. IEEE， 2023： 9641-9650. DOI： 10.1109/ICCV51070.2023.00887 . |
| [39] | 余保玲， 虞松坤， 孙耀然， 等. 基于DeepPose和Faster RCNN的多目标人体骨骼节点检测算法［J］. 中国科学院大学学报， 2020， 37（6）： 828-834. DOI：10.7523/j.issn.2095-6134.2020.06.015 . |
| [40] | Zaremba W， Sutskever I， Vinyals O. Recurrent neural network regularization［EB/OL］. arXiv 2015： 1409.2329v5. （2015-02-19） ［2025-03-18］. . |
/
| 〈 |
|
〉 |
