中国科学院大学学报 ›› 2026, Vol. 43 ›› Issue (5): 657-666.DOI: 10.7523/j.ucas.2024.056
收稿日期:2024-03-24
修回日期:2024-05-23
发布日期:2024-06-11
通讯作者:
肖俊
基金资助:
Yutao JIANG, Junqi DIAO, Jun XIAO(
), Ying WANG
Received:2024-03-24
Revised:2024-05-23
Published:2024-06-11
Contact:
Jun XIAO
摘要:
非刚性三维模型的配准在动画驱动、纹理传递、语义理解等不同领域具有广泛的应用,是计算机图形学的重要研究问题。然而基于骨架变形的非刚性物体,如人体、动物等,往往按照关节的运动变化带动整体的形状变形,这种复杂的相互作用给三维配准任务带来了困难和挑战。尽管近期的研究利用深度学习技术在重建具有骨架结构的三维形状时可以取得不错的效果,但这些方法往往严重依赖于大型带标注的数据集。在无监督的情况下,依然容易生成过度拉伸和部分交叉的模型网格。针对三维模型的配准问题,提出一种基于生成式对抗网络的无监督网络框架,利用三维生成器生成拟合扫描点云的模型网格,同时使用2个新型的判别器对变形网格进行正则化约束,该方法在公开数据集上取得了较好的实验效果。
中图分类号:
蒋宇涛, 刁俊淇, 肖俊, 王颖. 基于形状和边判别的三维模型无监督配准[J]. 中国科学院大学学报, 2026, 43(5): 657-666.
Yutao JIANG, Junqi DIAO, Jun XIAO, Ying WANG. Unsupervised 3D registration of articulated shapes by shape and edge regularization[J]. Journal of University of Chinese Academy of Sciences, 2026, 43(5): 657-666.
| 方法 | FAUST | SMAL | FAUST-inter | SMAL-Reg |
|---|---|---|---|---|
| 3D-CODED | 0.961 | 0.19 | 10.46 | 3.23 |
| Deep-Cage | 2.160 | * | 21.65 | * |
| 本文方法 | 0.844 | 0.16 | 7.64 | 2.66 |
表1 重建与配准结果评价
Table 1 Evaluation of reconstruction and registration results
| 方法 | FAUST | SMAL | FAUST-inter | SMAL-Reg |
|---|---|---|---|---|
| 3D-CODED | 0.961 | 0.19 | 10.46 | 3.23 |
| Deep-Cage | 2.160 | * | 21.65 | * |
| 本文方法 | 0.844 | 0.16 | 7.64 | 2.66 |
| 方法 | FAUST-chamfer | FAUST-inter |
|---|---|---|
| Baseline | 0.961 | 10.46 |
| 仅含形状判别 | 0.904 | 8.27 |
| 本文方法 | 0.844 | 7.64 |
表2 2个判别器的消融实验
Table 2 Ablation studies on two discriminators
| 方法 | FAUST-chamfer | FAUST-inter |
|---|---|---|
| Baseline | 0.961 | 10.46 |
| 仅含形状判别 | 0.904 | 8.27 |
| 本文方法 | 0.844 | 7.64 |
| [1] | Romero J, Tzionas D, Black M J. Embodied hands: modeling and capturing hands and bodies together[J]. ACM Trans Graph,2017,36(6): 1-17. DOI:10.1145/3130800.3130883 . |
| [2] | Loper M, Mahmood N, Romero J, et al. SMPL: a skinned multi-person linear model[M]//Seminal Graphics Papers: Pushing the Boundaries, Vol 2. 2023: 851-866. DOI:10.1145/3596711.3596800 . |
| [3] | Yang H T, Zhu H, Wang Y R, et al. FaceScape: a large-scale high quality 3D face dataset and detailed riggable 3D face prediction[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, WA, USA. IEEE, 2020: 598-607. DOI: 10.1109/CVPR42600.2020.00068 . |
| [4] | Sumner R W, Popović J. Deformation transfer for triangle meshes[J]. ACM Transactions on Graphics, 2004, 23(3): 399-405. DOI: 10.1145/1015706.1015736 . |
| [5] | Amberg B, Romdhani S, Vetter T. Optimal step nonrigid ICP algorithms for surface registration[C]//2007 IEEE Conference on Computer Vision and Pattern Recognition. Minneapolis, MN, USA. IEEE, 2007: 1-8. DOI: 10.1109/CVPR.2007.383165 . |
| [6] | Feng A W, Casas D, Shapiro A. Avatar reshaping and automatic rigging using a deformable model[C]//Proceedings of the 8th ACM SIGGRAPH Conference on Motion in Games. 2015: 57-64. DOI:10.1145/2822013.2822017 . |
| [7] | Zuffi S, Black M J. The stitched puppet: a graphical model of 3d human shape and pose[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015: 3537-3546. DOI:10.1109/cvpr.2015.7298976 . |
| [8] | Halimi O, Imanuel I, Litany O, et al. The whole is greater than the sum of its nonrigid parts[EB/OL]. arXiv 2020: 2001.09650.(2020-01-27)[2024-05-09].. |
| [9] | Deprelle T, Groueix T, Fisher M, et al. Learning elementary structures for 3D shape generation and matching[J]. Advances in Neural Information Processing Systems 32, 2019: 7433-7443. |
| [10] | Zhou B, Franco J S, Bogo F, et al. Reconstructing human body mesh from point clouds by adversarial gp network[C]//Proceedings of the Asian Conference on Computer Vision. 2020. DOI:10.1007/978-3-030-69525-5_8 . |
| [11] | Groueix T, Fisher M, Kim V G, et al. 3d-coded: 3d correspondences by deep deformation[C]//Proceedings of the european conference on computer vision (ECCV). 2018: 230-246. DOI:10.1007/978-3-030-01216-8_15 . |
| [12] | Li C L, Simon T, Saragih J, et al. Lbs autoencoder: self-supervised fitting of articulated meshes to point clouds[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 11967-11976. DOI:10.1109/cvpr.2019.01224 . |
| [13] | Wang Y F, Aigerman N, Kim V G, et al. Neural cages for detail-preserving 3D deformations[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, WA, USA. IEEE, 2020: 72-80. DOI: 10.1109/CVPR42600.2020.00015 . |
| [14] | Bhatnagar B L, Sminchisescu C, Theobalt C, et al. Combining implicit function learning and parametric models for 3D human reconstruction[C]//Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16. Springer International Publishing, 2020: 311-329. DOI:10.1007/978-3-030-58536-5_19 . |
| [15] | Ginzburg D, Raviv D. Cyclic functional mapping: self-supervised correspondence between non-isometric deformable shapes[C]//Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part V 16. Springer International Publishing, 2020: 36-52. DOI:10.1007/978-3-030-58558-7_3 . |
| [16] | Eisenberger M, Lahner Z, Cremers D. Smooth shells: multi-scale shape registration with functional maps[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 12265-12274. DOI:10.1109/CVPR42600.2020.01228 . |
| [17] | Varol G, Romero J, Martin X, et al. Learning from synthetic humans[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 109-117. DOI:10.1109/CVPR.2017.492 . |
| [18] | Bogo F, Romero J, Loper M, et al. FAUST: dataset and evaluation for 3D mesh registration[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2014: 3794-3801. DOI:10.1109/CVPR.2014.491 . |
| [19] | Zuffi S, Kanazawa A, Jacobs D W, et al. 3D menagerie: modeling the 3D shape and pose of animals[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 6365-6373. DOI:10.1109/CVPR.2017.586 . |
| [20] | Varol G, Ceylan D, Russell B, et al. Bodynet: volumetric inference of 3D human body shapes[C]//Proceedings of the European conference on computer vision (ECCV). 2018: 20-38. DOI:10.1007/978-3-030-01234-2_2 . |
| [21] | Rakotosaona M J, Ovsjanikov M. Intrinsic point cloud interpolation via dual latent space navigation[C]//European Conference on Computer Vision. Cham: Springer, 2020: 655-672.DOI:10.1007/978-3-030-58536-5_39 . |
| [22] | Ranjan A, Bolkart T, Sanyal S, et al. Generating 3D faces using convolutional mesh autoencoders[C]//Proceedings of the European conference on computer vision (ECCV). 2018: 704-720. DOI:10.1007/978-3-030-01219-9_43 . |
| [23] | Kanazawa A, Tulsiani S, Efros A A, et al. Learning category-specific mesh reconstruction from image collections[C]//Proceedings of the European Conference on Computer Vision (ECCV). 2018: 386-402. DOI:10.1007/978-3-030-01267-0_23 . |
| [24] | Xu H Y, Bazavan E G, Zanfir A, et al. GHUM & GHUML: generative 3D human shape and articulated pose models[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, WA, USA. IEEE, 2020: 6183-6192. DOI: 10.1109/CVPR42600.2020.00622 . |
| [25] | Chibane J, Alldieck T, Pons-Moll G. Implicit functions in feature space for 3D shape reconstruction and completion[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020: 6970-6981. DOI:10.1109/CVPR42600.2020.00700 . |
| [26] | Saito S, Huang Z, Natsume R, et al. Pifu: pixel-aligned implicit function for high-resolution clothed human digitization[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019: 2304-2314. DOI:10.1109/ICCV.2019.00239 . |
| [27] | Sun J, Ovsjanikov M, Guibas L. A concise and provably informative multi‐scale signature based on heat diffusion[C]//Computer graphics forum. Oxford, UK: Blackwell Publishing Ltd, 2009, 28(5): 1383-1392. DOI:10.1111/J.1467-8659.2009.01515.X . |
| [28] | Marin R, Melzi S, Rodol E, et al. FARM: functional automatic registration method for 3D human bodies[J]. Computer Graphics Forum, 2020, 39(1): 160-173. DOI: 10.1111/cgf.13751 . |
| [29] | Ovsjanikov M, Corman E, Bronstein M, et al. Computing and processing correspondences with functional maps[M]//SIGGRAPH ASIA 2016 Courses. 2016: 1-60. DOI:10.1145/2988458.2988494 . |
| [30] | Sharma A, Ovsjanikov M. Weakly supervised deep functional maps for shape matching[C]//Proceedings of the 34th International Conference on Neural Information Processing Systems, ACM. 2020: 19264-19275. |
| [31] | Hirshberg D A, Loper M, Rachlin E, et al. Evaluating the automated alignment of 3D human body scans[C]//2nd International Conference on 3D Body Scanning Technologies. Hometrica Consulting, 2011: 76-85. DOI:10.15221/11.076 . |
| [32] | Hirshberg D A, Loper M, Rachlin E, et al. Coregistration: simultaneous alignment and modeling of articulated 3D shape[C]//Computer Vision–ECCV 2012: 12th European Conference on Computer Vision, Florence, Italy, October 7-13, 2012, Proceedings, Part VI 12. Springer Berlin Heidelberg, 2012: 242-255. DOI:10.1007/978-3-642-33783-3_18 . |
| [33] | Goodfellow I J, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets[EB/OL]. arXiv 2014:1406.2661.(2014-06-10)[2024-05-09]. . |
| [34] | Abrevaya V F, Boukhayma A, Wuhrer S, et al. A decoupled 3d facial shape model by adversarial training[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019: 9419-9428. DOI:10.1109/ICCV.2019.00951 . |
| [35] | Kanazawa A, Black M J, Jacobs D W, et al. End-to-end recovery of human shape and pose[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018: 7122-7131. DOI:10.1109/CVPR.20 18.00744 . |
| [36] | Ma Q L, Yang J L, Ranjan A, et al. Learning to dress 3D people in generative clothing[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, WA, USA. IEEE, 2020: 6468-6477. DOI: 10.1109/CVPR42600.2020.00650 . |
| [37] | Charles R Q, Hao S, Mo K C, et al. PointNet: deep learning on point sets for 3D classification and segmentation[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA. IEEE, 2017: 77-85. DOI: 10.1109/CVPR.2017.16 . |
| [38] | Isola P, Zhu J Y, Zhou T H, et al. Image-to-image translation with conditional adversarial networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA. IEEE, 2017: 5967-5976. DOI: 10.1109/CVPR.2017.632 . |
| [39] | Mao X D, Li Q, Xie H R, et al. Least squares generative adversarial networks[C]//2017 IEEE International Conference on Computer Vision (ICCV). Venice, Italy. IEEE, 2017: 2813-2821. DOI: 10.1109/ICCV.2017.304 . |
| [1] | 翁永椿, 马勇, 陈甫, 尚二萍, 姚武韬, 仉淑艳, 杨进, 刘建波. 基于双时相特征的SAR生成光学影像方法[J]. 中国科学院大学学报, 2025, 42(6): 769-780. |
| [2] | 张冰玉, 潘志刚, 姚锴, 董旭彬. 基于生成对抗网络的SAR解压缩图像重建算法[J]. 中国科学院大学学报, 2025, 42(5): 666-676. |
| [3] | 喻永生, 罗铁坚. 基于GAN反演的无缝图像补全技术[J]. 中国科学院大学学报, 2024, 41(5): 705-714. |
| [4] | 陈汉钦, 秦进, 赵彤, 阎瑶. 面向识别的人脸三维信息估计[J]. 中国科学院大学学报, 2023, 40(2): 268-279. |
| [5] | 王正晟, 李亚林, 张洪群. 基于GAN序列的卫星数据接收系统故障定位[J]. 中国科学院大学学报, 2023, 40(1): 109-118. |
| [6] | 胡兵兵, 唐华, 吴幼龙. 基于互信息约束的生成对抗网络分类模型[J]. 中国科学院大学学报, 2022, 39(4): 551-560. |
| [7] | 赵心驰, 姜策, 何为. 基于注意力生成对抗网络的图像强光去除[J]. 中国科学院大学学报, 2022, 39(4): 524-531. |
| [8] | 陈国炜, 刘磊, 郭嘉逸, 潘宗序, 胡文龙. 基于生成对抗网络的半监督遥感图像飞机检测[J]. 中国科学院大学学报, 2020, 37(4): 539-546. |
| [9] | 毛明旭, 叶佳敏, 王海刚, 张娇龙. 三维电容层析成像多电极激励模式的稳定性分析[J]. 中国科学院大学学报, 2017, 34(2): 186-190. |
| [10] | 胡继伟, 洪峻. 基于三维重建的分布式卫星SAR干涉测高模型及误差分析[J]. 中国科学院大学学报, 2011, 28(4): 489-497. |
| 阅读次数 | ||||||
|
全文 |
|
|||||
|
摘要 |
|
|||||