用于细粒度图像分类的层级注意力双重网络

doi:10.7523/j.ucas.2024.008

摘要/Abstract

摘要： 本文提出了一种用于细粒度图像分类的层级注意力双重网络。双重网络可以从数据集中随机选择成对的输入,通过层次注意力特征学习比较它们之间的差异,这有利于消除噪声的同时保留显著特征。在损失函数设计中,根据类内差异和类间差异,考虑了成对图像之间差异损失的计算。除此之外,我们通过遥感图像搜集了灾害场景数据集,这些数据集包含了各种复杂场景和多种灾害类型。论文将设计的方法应用在该灾害场景分类中。与其他方法相比,实验结果显示层级注意力双重网络在不同的数据集上具有较好的鲁棒性,取得了更好的性能指标。

关键词: 双重网络, 细粒度图像分类, 层级注意力特征

Abstract: In this paper, we propose hierarchical attention dual network for fine-grained image classification. The dual network can randomly select pairs of inputs from dataset and compare the differences between them through hierarchical attention features learning, which are used simultaneously to remove noise and retain salient features. In the loss function, it considers the losses of difference in paired images according to the intra-variance and inter-variance. In addition, we also collect the disaster scene dataset from remote sensing images and apply the proposed method to disaster scene classification, which contain complex scenes and multiple types of disasters. Compared to other methods, experimental results show that the dual network with hierarchical attention is robust to different datasets and performs better.

Key words: Dual network, Fine-grained image classification, Hierarchical attention features

中图分类号:

TP183

杨涛, 王改华. 用于细粒度图像分类的层级注意力双重网络[J]. 中国科学院大学学报, DOI: 10.7523/j.ucas.2024.008.

YANG Tao, WANG Gaihua. Dual networks with hierarchical attention for fine-grained image classification^*[J]. Journal of University of Chinese Academy of Sciences, DOI: 10.7523/j.ucas.2024.008.

参考文献

[1] Jagiello Z, Reynolds S J, Nagy J, et al.Why do some bird species incorporate more anthropogenic materials into their nests than others?[J]. Philosophical Transactions of the Royal Society B: Biological Sciences, 2023, 378(1884). DOI: 10.1098/rstb.2022.0156.
[2] Yang X D, Su J S, Qu Y X, et al.Dissecting the inheritance pattern of the anemone flower type and tubular floral traits of chrysanthemum in segregating F-1 populations[J]. Euphytica, 2023, 219(1): 16.DOI:10.1007/s10681-022-03141-6.
[3] Park C, Moon N.Dog-Species Classification through CycleGAN and Standard Data Augmentation[J]. Journal of Information Processing Systems, 2023, 19(1): 67-79.DOI: 10.3745/JIPS.02.0190.
[4] Lam M, Mahasseni B, Todorovic S.Fine-grained recognition as HSnet search for informative image parts[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA. IEEE, 2017: 6497-6506. DOI: 10.1109/CVPR.2017.688.
[5] He X T, Peng Y X. Weakly supervised learning of part selection model with spatial constraints for fine-grained image classification[C]//Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence. February 4 - 9, 2017, San Francisco, California, USA. ACM,2017: 4075-4081. https://dl.acm.org/doi/10.5555/3298023.3298160
[6] He X T, Peng Y X, Zhao J J.Fine-grained discriminative localization via saliency-guided faster R-CNN[C]//Proceedings of the 25th ACM international conference on Multimedia. October 23 - 27, 2017, Mountain View, California, USA. ACM, 2017: 627-635. DOI: 10.1145/3123266.3123319.
[7] 周龙,王伟强,吕科. FACR:一种快速且准确的车辆识别器[J]. 中国科学院大学学报,2021,38(1):130-136. DOI:10.7523/j.issn.2095-6134.2021.01.016.
[8] Lin T Y, RoyChowdhury A, Maji S. Bilinear CNNs for fine-grained visual recognition[EB/OL]. 2015: arXiv: 1504.07889. http://arxiv.org/abs/1504.07889.
[9] Gao Y, Beijbom O, Zhang N, et al.Compact bilinear pooling[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA. IEEE, 2016: 317-326. DOI: 10.1109/CVPR.2016.41.
[10] Kong S, Fowlkes C.Low-rank bilinear pooling for fine-grained classification[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA. IEEE, 2017: 7025-7034. DOI: 10.1109/CVPR.2017.743.
[11] Nie X, Chai B S, Wang L Y, et al.Learning enhanced features and inferring twice for fine-grained image classification[J]. Multimedia Tools and Applications, 2023, 82(10): 14799-14813. DOI: 10.1007/s11042-022-13619-z.
[12] Özdemir C.Avg-topk: A new pooling method for convolutional neural networks[J]. Expert Systems with Applications,2023,223:119892.DOI:10.1016/j.eswa.2023.119892.
[13] Li X J, Yang C, Chen S L, et al.Semantic bilinear pooling for fine-grained recognition[C]//2020 25th International Conference on Pattern Recognition (ICPR). Milan, Italy. IEEE, 2021: 3660-3666. DOI: 10.1109/ICPR48806.2021.9412252.
[14] Ji R Y, Li J Y, Zhang L B.Siamese self-supervised learning for fine-grained visual classification[J]. Computer Vision and Image Understanding, 2023, 229: 103658. DOI: 10.1016/j.cviu.2023.103658.
[15] Pan W Y, Yang S Y, Qian X H, et al.Learn more: sub-significant area learning for fine-grained visual classification[C]//2023 IEEE International Conference on Image Processing (ICIP). Kuala Lumpur, Malaysia. IEEE, 2023: 485-489. DOI: 10.1109/ICIP49359.2023.10222241.
[16] Jung Y, Syazwany N S, Kim S, et al.Fine-grained classification via hierarchical feature covariance attention module[J]. IEEE Access, 2023, 11: 35670-35679. DOI: 10.1109/ACCESS.2023.3265472.
[17] Xie J J, Zhong Y J, Zhang J G, et al.A weakly supervised spatial group attention network for fine-grained visual recognition[J]. Applied Intelligence, 2023,53(20):23301-23315.DOI:10.1007/s10489-023-04627-z.
[18] Yu C J, Zhao X Y, Zheng Q, et al.Hierarchical bilinear pooling for fine-grained visual recognition[C]//Computer Vision - ECCV 2018: 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part XVI. ACM, 2018: 595-610. DOI: 10.1007/978-3-030-01270-0_35.
[19] Zeng Z Y, Wang J P, Chen B, et al.Pyramid hybrid pooling quantization for efficient fine-grained image retrieval[J]. Pattern Recognition Letters, 2024, 178: 106-114. DOI: 10.1016/j.patrec.2023.12.022.
[20] Otholt J, Meinel C, Yang H J. Guided cluster aggregation: a hierarchical approach to generalized category discovery[C/OL]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision(WACV).2024:2618-2627.[2024-03-12]https://openaccess.thecvf.com/content/WACV2024/html/Otholt_Guided_Cluster_Aggregation_A_Hierarchical_Approach_to_Generalized_Category_Discovery_WACV_2024_paper.html.