提出一种用于细粒度图像分类的层级注意力双重网络。双重网络可以从数据集中随机选择成对的输入，通过层次注意力特征学习比较它们之间的差异，这有利于消除噪声的同时保留显著特征。在损失函数设计中，根据类内差异和类间差异，考虑了成对图像之间差异损失的计算。此外，通过遥感图像搜集了灾害场景数据集，这些数据集包含各种复杂场景和多种灾害类型。在该灾害场景分类中的验证实验表明，与其他方法相比，层级注意力双重网络在不同的数据集上具有较好的鲁棒性，取得了更好的性能指标。
In this paper, we propose hierarchical attention dual network (DNet) for fine-grained image classification. The DNet can randomly select pairs of inputs from the dataset and compare the differences between them through hierarchical attention feature learning, which are used simultaneously to remove noise and retain salient features. In the loss function, it considers the losses of difference in paired images according to the intra-variance and inter-variance. In addition, we also collect the disaster scene dataset from remote sensing images and apply the proposed method to disaster scene classification, which contains complex scenes and multiple types of disasters. Compared to other methods, experimental results show that the DNet with hierarchical attention is robust to different datasets and performs better.
[1] Jagiello Z, Reynolds S J, Nagy J, et al. Why do some bird species incorporate more anthropogenic materials into their nests than others?[J]. Philosophical Transactions of the Royal Society B: Biological Sciences, 2023, 378(1884). DOI: 10.1098/rstb.2022.0156.
[2] Yang X D, Su J S, Qu Y X, et al. Dissecting the inheritance pattern of the anemone flower type and tubular floral traits of chrysanthemum in segregating F-1 populations[J]. Euphytica, 2023, 219(1): 16.DOI:10.1007/s10681-022-03141-6.
[3] Park C, Moon N. Dog-species classification through CycleGAN and standard data augmentation[J]. Journal of Information Processing Systems, 2023, 19(1): 67-79.DOI: 10.3745/JIPS.02.0190.
[4] Lam M, Mahasseni B, Todorovic S. Fine-grained recognition as HSnet search for informative image parts[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA. IEEE, 2017: 6497-6506. DOI: 10.1109/CVPR.2017.688.
[5] He X T, Peng Y X. Weakly supervised learning of part selection model with spatial constraints for fine-grained image classification[C]//Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence. February 4-9, 2017, San Francisco, California, USA. ACM, 2017: 4075-4081. https://dl.acm.org/doi/10.5555/3298023.3298160.
[6] He X T, Peng Y X, Zhao J J. Fine-grained discriminative localization via saliency-guided faster R-CNN[C]//Proceedings of the 25th ACM international conference on Multimedia. October 23-27, 2017, Mountain View, California, USA. ACM, 2017: 627-635. DOI: 10.1145/3123266.3123319.
[7] Zhou L, Wang W Q, Lyu K. FACR: a fast and accurate car recognizer[J]. Journal of University of Chinese Academy of Sciences, 2021, 38(1): 130-136. DOI:10.7523/j.issn.2095-6134.2021.01.016(in Chinese).
[8] Lin T Y, RoyChowdhury A, Maji S. Bilinear CNNs for fine-grained visual recognition[EB/OL]. 2015: arXiv: 1504.07889. (2015-04-29)[2023-12-02].http://arxiv.org/abs/1504.07889.
[9] Gao Y, Beijbom O, Zhang N, et al. Compact bilinear pooling[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA. IEEE, 2016: 317-326. DOI: 10.1109/CVPR.2016.41.
[10] Kong S, Fowlkes C. Low-rank bilinear pooling for fine-grained classification[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA. IEEE, 2017: 7025-7034. DOI: 10.1109/CVPR.2017.743.
[11] Nie X, Chai B S, Wang L Y, et al. Learning enhanced features and inferring twice for fine-grained image classification[J]. Multimedia Tools and Applications, 2023, 82(10): 14799-14813. DOI: 10.1007/s11042-022-13619-z.
[12] Özdemir C. Avg-topk: new pooling method for convolutional neural networks[J]. Expert Systems with Applications,2023,223:119892.DOI:10.1016/j.eswa.2023.119892.
[13] Li X J, Yang C, Chen S L, et al. Semantic bilinear pooling for fine-grained recognition[C]//2020 25th International Conference on Pattern Recognition (ICPR). Milan, Italy. IEEE, 2021: 3660-3666. DOI: 10.1109/ICPR48806.2021.9412252.
[14] Ji R Y, Li J Y, Zhang L B. Siamese self-supervised learning for fine-grained visual classification[J]. Computer Vision and Image Understanding, 2023, 229: 103658. DOI: 10.1016/j.cviu.2023.103658.
[15] Pan W Y, Yang S Y, Qian X H, et al. Learn more: sub-significant area learning for fine-grained visual classification[C]//2023 IEEE International Conference on Image Processing (ICIP). Kuala Lumpur, Malaysia. IEEE, 2023: 485-489. DOI: 10.1109/ICIP49359.2023.10222241.
[16] Jung Y, Syazwany N S, Kim S, et al. Fine-grained classification via hierarchical feature covariance attention module[J]. IEEE Access, 2023, 11: 35670-35679. DOI: 10.1109/ACCESS.2023.3265472.
[17] Xie J J, Zhong Y J, Zhang J G, et al. A weakly supervised spatial group attention network for fine-grained visual recognition[J]. Applied Intelligence, 2023,53(20):23301-23315.DOI:10.1007/s10489-023-04627-z.
[18] Yu C J, Zhao X Y, Zheng Q, et al. Hierarchical bilinear pooling for fine-grained visual recognition[C]//Computer Vision-ECCV 2018: 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part XVI. ACM, 2018: 595-610. DOI: 10.1007/978-3-030-01270-0_35.
[19] Zeng Z Y, Wang J P, Chen B, et al. Pyramid hybrid pooling quantization for efficient fine-grained image retrieval[J]. Pattern Recognition Letters, 2024, 178: 106-114. DOI: 10.1016/j.patrec.2023.12.022.
[20] Otholt J, Meinel C, Yang H J. Guided cluster aggregation: a hierarchical approach to generalized category discovery[C/OL]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision(WACV).2024:2618-2627. [2024-03-12]https://openaccess.thecvf.com/content/WACV2024/html/Otholt_Guided_Cluster_Aggregation_A_Hierarchical_Approach_to_Generalized_Category_Discovery_WACV_2024_paper.html.
