In this paper, we propose hierarchical attention dual network (DNet) for fine-grained image classification. The DNet can randomly select pairs of inputs from the dataset and compare the differences between them through hierarchical attention feature learning, which are used simultaneously to remove noise and retain salient features. In the loss function, it considers the losses of difference in paired images according to the intra-variance and inter-variance. In addition, we also collect the disaster scene dataset from remote sensing images and apply the proposed method to disaster scene classification, which contains complex scenes and multiple types of disasters. Compared to other methods, experimental results show that the DNet with hierarchical attention is robust to different datasets and performs better.
YANG Tao
,
WANG Gaihua
. Dual networks with hierarchical attention for fine-grained image classification[J]. Journal of University of Chinese Academy of Sciences, 2025
, 42(6)
: 806
-813
.
DOI: 10.7523/j.ucas.2024.008
[1] Jagiello Z, Reynolds S J, Nagy J, et al. Why do some bird species incorporate more anthropogenic materials into their nests than others?[J]. Philosophical Transactions of the Royal Society B: Biological Sciences, 2023, 378(1884). DOI: 10.1098/rstb.2022.0156.
[2] Yang X D, Su J S, Qu Y X, et al. Dissecting the inheritance pattern of the anemone flower type and tubular floral traits of chrysanthemum in segregating F-1 populations[J]. Euphytica, 2023, 219(1): 16.DOI:10.1007/s10681-022-03141-6.
[3] Park C, Moon N. Dog-species classification through CycleGAN and standard data augmentation[J]. Journal of Information Processing Systems, 2023, 19(1): 67-79.DOI: 10.3745/JIPS.02.0190.
[4] Lam M, Mahasseni B, Todorovic S. Fine-grained recognition as HSnet search for informative image parts[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA. IEEE, 2017: 6497-6506. DOI: 10.1109/CVPR.2017.688.
[5] He X T, Peng Y X. Weakly supervised learning of part selection model with spatial constraints for fine-grained image classification[C]//Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence. February 4-9, 2017, San Francisco, California, USA. ACM, 2017: 4075-4081. https://dl.acm.org/doi/10.5555/3298023.3298160.
[6] He X T, Peng Y X, Zhao J J. Fine-grained discriminative localization via saliency-guided faster R-CNN[C]//Proceedings of the 25th ACM international conference on Multimedia. October 23-27, 2017, Mountain View, California, USA. ACM, 2017: 627-635. DOI: 10.1145/3123266.3123319.
[7] Zhou L, Wang W Q, Lyu K. FACR: a fast and accurate car recognizer[J]. Journal of University of Chinese Academy of Sciences, 2021, 38(1): 130-136. DOI:10.7523/j.issn.2095-6134.2021.01.016(in Chinese).
[8] Lin T Y, RoyChowdhury A, Maji S. Bilinear CNNs for fine-grained visual recognition[EB/OL]. 2015: arXiv: 1504.07889. (2015-04-29)[2023-12-02].http://arxiv.org/abs/1504.07889.
[9] Gao Y, Beijbom O, Zhang N, et al. Compact bilinear pooling[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA. IEEE, 2016: 317-326. DOI: 10.1109/CVPR.2016.41.
[10] Kong S, Fowlkes C. Low-rank bilinear pooling for fine-grained classification[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA. IEEE, 2017: 7025-7034. DOI: 10.1109/CVPR.2017.743.
[11] Nie X, Chai B S, Wang L Y, et al. Learning enhanced features and inferring twice for fine-grained image classification[J]. Multimedia Tools and Applications, 2023, 82(10): 14799-14813. DOI: 10.1007/s11042-022-13619-z.
[12] Özdemir C. Avg-topk: new pooling method for convolutional neural networks[J]. Expert Systems with Applications,2023,223:119892.DOI:10.1016/j.eswa.2023.119892.
[13] Li X J, Yang C, Chen S L, et al. Semantic bilinear pooling for fine-grained recognition[C]//2020 25th International Conference on Pattern Recognition (ICPR). Milan, Italy. IEEE, 2021: 3660-3666. DOI: 10.1109/ICPR48806.2021.9412252.
[14] Ji R Y, Li J Y, Zhang L B. Siamese self-supervised learning for fine-grained visual classification[J]. Computer Vision and Image Understanding, 2023, 229: 103658. DOI: 10.1016/j.cviu.2023.103658.
[15] Pan W Y, Yang S Y, Qian X H, et al. Learn more: sub-significant area learning for fine-grained visual classification[C]//2023 IEEE International Conference on Image Processing (ICIP). Kuala Lumpur, Malaysia. IEEE, 2023: 485-489. DOI: 10.1109/ICIP49359.2023.10222241.
[16] Jung Y, Syazwany N S, Kim S, et al. Fine-grained classification via hierarchical feature covariance attention module[J]. IEEE Access, 2023, 11: 35670-35679. DOI: 10.1109/ACCESS.2023.3265472.
[17] Xie J J, Zhong Y J, Zhang J G, et al. A weakly supervised spatial group attention network for fine-grained visual recognition[J]. Applied Intelligence, 2023,53(20):23301-23315.DOI:10.1007/s10489-023-04627-z.
[18] Yu C J, Zhao X Y, Zheng Q, et al. Hierarchical bilinear pooling for fine-grained visual recognition[C]//Computer Vision-ECCV 2018: 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part XVI. ACM, 2018: 595-610. DOI: 10.1007/978-3-030-01270-0_35.
[19] Zeng Z Y, Wang J P, Chen B, et al. Pyramid hybrid pooling quantization for efficient fine-grained image retrieval[J]. Pattern Recognition Letters, 2024, 178: 106-114. DOI: 10.1016/j.patrec.2023.12.022.
[20] Otholt J, Meinel C, Yang H J. Guided cluster aggregation: a hierarchical approach to generalized category discovery[C/OL]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision(WACV).2024:2618-2627. [2024-03-12]https://openaccess.thecvf.com/content/WACV2024/html/Otholt_Guided_Cluster_Aggregation_A_Hierarchical_Approach_to_Generalized_Category_Discovery_WACV_2024_paper.html.
