Implicit-Expression-based 3D Reconstruction of Clothing

doi:10.19678/j.issn.1000-3428.0067724

Abstract

Abstract: Owing to the rapid development of Internet shopping in recent years, clothing items have appeared increasingly on major platforms. Generating Three-Dimensional(3D) models of garments using 3D reconstruction technology allows consumers to better understand the gestural information of garments. This study examines the 3D reconstruction technology of clothing and proposes a 3D clothing-reconstruction model based on implicit expressions. An occupancy function obtained via neural-network learning is used as the implicit expression of the garment 3D model, and mapping is performed between the 3D coordinates and model shape. Existing 3D reconstruction methods must fit complex surface models, which consumes a significant amount of resource, whereas implicit-expression-based 3D reconstruction algorithms require neither parameterization nor meshing, which accelerates the operation of the algorithm. To further improve the 3D reconstruction effect, the current best- performing PointMetaBase-L network model and the offset attention module are used as the feature-extraction network of the model. Between them, the PointMeta-Base-L network model proposes the PointMeta meta-architecture of the Set Abstraction layer based on the existing point cloud feature-extraction network and selects the best practices of four modules in the PointMeta meta-architecture to constitute the set of the PointMetaBase-L network model. This is performed by analyzing the abstraction layer while introducing a planar feature-projection module to enhance the local information of the features. In the feature-decoding stage, the occupancy probabilities of sampled points in 3D space are obtained using a weighted averaging algorithm via a feature-weighting network. A high-precision mesh-reconstruction model is extracted from the occupancy probabilities of the sampled points using the March Cubes algorithm based on regional growth. Experimental results show that compared with the occupancy network, the improved model improves the intersection ratio, chamfer distance, normal consistency, and F1 value by 48.83%, 55.17%, 4.27%, and 79.10%, respectively.

Key words: implicit expression, 3D reconstruction, PointMetaBase-L network model, offset attention, feature weight network, region growth, Marching Cubes algorithm

摘要： 随着近年来互联网购物的快速发展,在各大平台上出现了越来越多的服装商品。通过三维重建技术生成服装的三维模型可以帮助消费者更好地了解服装的姿态信息。针对服装的三维重建技术进行研究,提出基于隐式表达的服装三维重建模型。使用神经网络学习获得的占用函数作为服装三维模型的隐式表达,从而建立三维坐标和模型形状的映射。目前已有的三维重建算法需要拟合复杂的曲面模型,但资源消耗量大,而基于隐式表达的三维重建算法不需要进行参数化和网格化,能够提高算法的运行速度。为了进一步提高三维重建效果,采用目前性能最好的PointMetaBase-L网络模型和偏移注意力模块作为模型的特征提取网络。其中PointMetaBase-L网络模型基于现有的点云特征提取网络提出Set Abstraction层的元架构PointMeta,并通过分析选择PointMeta元架构中4个模块的最佳实践构成PointMetaBase-L网络模型的Set Abstraction层,同时引入平面特征投影模块加强特征的局部信息。在特征解码阶段,利用特征权重网络通过加权平均算法获取三维空间中采样点的占用概率。根据这些采样点的占用概率,通过基于区域增长的Marching Cubes算法提取高精度网格重建模型。实验结果表明,与占用网络相比,改进模型在交并比、倒角距离、法线一致性和F1值上分别提升了48.83%,55.17%、4.27%和79.10%。

关键词: 隐式表达, 三维重建, PointMetaBase-L网络模型, 偏移注意力, 特征权重网络, 区域增长, Marching Cubes算法

CLC Number:

TP391

FEI Yuzhe, CAI Xin, ZHAO Mingbo, YANG Shenghao. Implicit-Expression-based 3D Reconstruction of Clothing[J]. Computer Engineering, 2024, 50(5): 220-228.

费煜哲, 蔡欣, 赵鸣博, 杨圣豪. 基于隐式表达的服装三维重建[J]. 计算机工程, 2024, 50(5): 220-228.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0067724

https://www.ecice06.com/EN/Y2024/V50/I5/220

References

[1] SCHLEMPER J, CABALLERO J, HAJNAL J V, et al. A deep cascade of convolutional neural networks for MR image reconstruction[C]//Proceedings of International Conference on Information Processing in Medical Imaging. Berlin, Germany:Springer, 2017:647-658.
[2] VARLEY J, DECHANT C, RICHARDSON A, et al. Shape completion enabled robotic grasping[C]//Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems. New York, USA:ACM, 2017:2442-2447.
[3] RUSU R B, MARTON Z C, BLODOW N, et al. Towards 3D point cloud based object maps for household environments[J]. Robotics and Autonomous Systems, 2008, 56(11):927-941.
[4] HUANG Z, XU Y L, LASSNER C, et al. ARCH:animatable reconstruction of clothed humans[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, USA:IEEE Press, 2020:3093-3102.
[5] ZHU F, LI S B, GUO X H. A 3D reconstruction method based on RGB-D camera[J]. Journal of Physics, 2021, 18 (4):042048.
[6] 毛方儒, 王磊. 三维激光扫描测量技术[J]. 宇航计测技术, 2005, 25(2):1-6. MAO F R, WANG L. Measurement technology of 3D laser scanning[J]. Journal of Astronautic Metrology and Measurement, 2005, 25(2):1-6.(in Chinese)
[7] HARTLEY R, ZISSERMAN A. Multiple view geometry in computer vision[M]. Cambridge, UK:Cambridge University Press,2003.
[8] 赵碧霞, 张华, 王姮, 等. 基于Bayes理论的散斑三维重建方法[J]. 计算机工程, 2017, 43(12):211-215. ZHAO B X, ZHANG H, WANG H, et al. 3D reconstruction method of speckle based on Bayes theory[J]. Computer Engineering, 2017, 43(12):211-215.(in Chinese)
[9] 陈坤, 刘新国. 基于光线的全局优化多视图三维重建方法[J]. 计算机工程, 2013, 39(11):235-239. CHEN K, LIU X G. Global optimized multi-view 3D reconstruction method based on rays[J]. Computer Engineering, 2013, 39(11):235-239.(in Chinese)
[10] ZHANG X M, ZHANG Z T, ZHANG C K, et al. Learning to reconstruct shapes from unseen classes[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems. New York,USA:ACM Press, 2018:2263-2274.
[11] KANAZAWA A, BLACK M J, JACOBS D W, et al. End-to-end recovery of human shape and pose[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA:IEEE Press, 2018:7122-7131.
[12] KONG C, LIN C H, LUCEY S. Using locally corresponding CAD models for dense 3D reconstructions from a single image[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA:IEEE Press, 2017:4857-4865.
[13] LORENSEN W E, CLINE H E. Marching Cubes:a high resolution 3D surface construction algorithm[J]. ACM SIGGRAPH Computer Graphics, 1987, 21(4):163-169.
[14] MESCHEDER L, OECHSLE M, NIEMEYER M, et al. Occupancy networks:learning 3D reconstruction in function space[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, USA:IEEE Press, 2019:4460-4470.
[15] PENG S Y, NIEMEYER M, MESCHEDER L, et al. Convolutional occupancy networks[C]//Proceedings of European Conference on Computer Vision. Berlin, Germany:Springer, 2020:523-540.
[16] LIONAR S, EMTSEV D, SVILARKOVIC D, et al. Dynamic plane convolutional occupancy networks[C]//Proceedings of IEEE Winter Conference on Applications of Computer Vision. Waikoloa, USA:IEEE Press, 2021:1829-1838.
[17] TANG J P, LEI J B, XU D, et al. SA-ConvONet:sign-agnostic optimization of convolutional occupancy networks[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D. C.,USA:IEEE Press, 2021:6504-6513.
[18] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C.,USA:IEEE Press, 2016:770-778.
[19] CHARLES R Q, HAO S, MO K C, et al. PointNet:deep learning on point sets for 3D classification and segmentation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA:IEEE Press, 2017:652-660.
[20] DE VRIES H, STRUB F, MARY J, et al. Modulating early visual processing by language[C]//Proceedings of NIPS'17. Cambridge,USA:MIT Press, 2017:6594-6604.
[21] NAIR V, HINTON G E. Rectified linear units improve restricted boltzmann machines[C]//Proceedings of the 27th International Conference on International Conference on Machine Learning. New York,USA:ACM Press, 2010:807-814.
[22] GARLAND M, HECKBERT P S. Simplifying surfaces with color and texture using quadric error metrics[C]//Proceedings Visualization'98. Washington D. C.,USA:IEEE Press, 1998:263-269.
[23] RONNEBERGER O, FISCHER P, BROX T. U-Net:convolutional networks for biomedical image segmentation[M]. Berlin, Germany:Springer, 2015.
[24] ÇIÇEK Ö, ABDULKADIR A, LIENKAMP S S, et al. 3D U-Net:learning dense volumetric segmentation from sparse annotation[C]//Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin, Germany:Springer, 2016:424-432.
[25] MILDENHALL B, SRINIVASAN P P, TANCIK M, et al. NeRF:representing scenes as neural radiance fields for view synthesis[C]//Proceedings of European Conference on Computer Vision. Berlin, Germany:Springer, 2020:405-421.
[26] ZHU H M, CAO Y, JIN H, et al. Deep Fashion3D:a dataset and benchmark for 3D garment reconstruction from single images[C]//Proceedings of European Conference on Computer Vision. Berlin, Germany:Springer, 2020:512-530.
[27] BERNARDINI F, MITTLEMAN J, RUSHMEIER H, et al. The ball-pivoting algorithm for surface reconstruction[J]. IEEE Transactions on Visualization and Computer Graphics, 1999, 5(4):349-359.
[28] STUTZ D, GEIGER A. Learning 3D shape completion from laser scan data with weak supervision[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C.,USA:IEEE Press, 2018:1955-1964.
[29] LIN H, ZHENG X, LI L, et al. Meta architecure for point cloud analysis[EB/OL].[2023-04-24]. https://arxiv.org/abs/2211.14462.
[30] QIAN G C, HAMMOUD H, LI G H, et al. ASSANet:an anisotropic separable set abstraction for efficient point cloud representation learning[EB/OL].[2023-04-24]. https://arxiv.org/abs/2110.10538v1.
[31] ZHAO H S, JIANG L, JIA J Y, et al. Point transformer[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D. C.,USA:IEEE Press, 2021:16259-16268.
[32] QIAN G, LI Y, PENG H, et al. Pointnext:revisiting pointnet++ with improved training and scaling strategies[C]//Proceedings of NIPS'22. Cambridge,USA:MIT Press, 2022:23192-23204.
[33] GUO M H, CAI J X, LIU Z N, et al. PCT:point cloud transformer[J]. Computational Visual Media, 2021, 7(2):187-199.
[34] BOULCH A, MARLET R. POCO:point convolution for surface reconstruction[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D. C.,USA:IEEE Press, 2022:6302-6314.
[35] ABEWYICKRAMA T, CHEEMA M, TANIAR D. KNN on road networks:a journey in experimentation and in-memory implementation[EB/OL].[2023-04-24]. https://arxiv.org/abs/1601.01549.
[36] QI C R, YI L, SU H, et al. PointNet++:deep hierarchical feature learning on point sets in a metric space[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York,USA:ACM Press, 2017:5105-5114.
[37] KAZHDAN M, BOLITHO M, HOPPE H. Poisson surface reconstruction[C]//Proceedings of the 4th Symposium on Geometry Processing. New York,USA:ACM Press, 2006:61-70.

Please choose a citation manager

Content to export