CNN-Based Multi-Output and Multi-Task Regression for Supershape Reconstruction from 3D Point Clouds
Abstract
Three-dimensional reconstruction is vital across various fields, including computer graphics, robotics, and medical imaging. This paper proposes a deep learning approach for three-dimensional object reconstruction, from 3D point cloud, a CNN-based Multi-output and Multi-Task Regressor. Our method is based on the original Point Net architecture, which is based on the challenges of applying convolution to point clouds. In the first place, this paper is adjusted with a Multi-Output Regressor to reconstruct Super shapes from 3D point clouds with a high degree of accuracy. In this approach, we first use Point Net to extract features from the 3D point cloud. These features are then fed into a Multi-Output Regressor, which predicts the Super shape parameters required to reconstruct the shape. The Multi-Output Regressor takes in the extracted features from Point Net and predicts multiple outputs at once. In the second place, the Point Net is adjusted with a Multi-Task Regressor. The network benefits from the ability to generalize the knowledge learned from one task to another, thereby enhancing the overall performance of the model. In the case of reconstructing Super shapes, the model would predict the 10 parameters required to generate the shape. The test results exceeded our expectations; they are interesting in terms of precision and cost of predictionReferences
Gajjar V. K. “Machine learning applications in plant identification, wireless channel estimation, and gain estimation for multi-user software-defined radio”. Missouri University of Science and Technology, 2022.
Ahmed, E. et al. “A survey on deep learning advances on different 3D data representations”. arXiv preprint arXiv:1808.01462, 2018.
Wicker, M., & Kwiatkowska, M. “Robustness of 3d deep learning in an adversarial setting”, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 11767-11775), 2019.
Hamida, A. B., Benoit, A., Lambert, P., & Amar, C. B. “3-D deep learning approach for remote sensing image classification”, IEEE Transactions on geoscience and remote sensing, 56(8), 4420-4434, 2018.
Casadevall, G., Duran, C., & Osuna, S. “AlphaFold2 and deep learning for elucidating enzyme conformational flexibility and its application for design”, JACS Au, 3(6), 1554-1562, 2023.
Bokhabrine, Y., Fougerolle, Y. D., Foufou, S., & Truchetet, F. “Genetic algorithms for Gielis surface recovery from 3D data sets”, in IEEE International Conference on Image Processing (Vol. 2, pp. II-549). IEE, September, 2007.
Garcia-Garcia, A., Gomez-Donoso, et al. “Pointnet: A 3d convolutional neural network for real-time object class recognition”. In International joint conference on neural networks (IJCNN) (pp. 1578-1584). IEEE, July, 2016.
H. Remmach et. al. “Swarm Optimization for Tridimensional point Cloud Reconstruction using Supershapes”, in Indian Journal of Computer Science and Engineering Vol 11 No X 1-5, 2019.
O’Mahony et al. “Deep learning vs. traditional computer vision. In Advances in Computer Vision”, in Proceedings of the 2019 Computer Vision Conference (CVC), Volume 1 1 (pp. 128-144). Springer International Publishing, 2020.
Fu S., Shi. Et al. “Field-dependent deep learning enables high-throughput whole-cell 3D super-resolution imaging”. Nature Methods, 20(3), 459-468, 2023.
Chaoxu Guo et al. “Learning Multimodal 3D Object Detection and Semantic Segmentation for Autonomous”, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, doi : 10.1109/CVPR42600.2020.00973
Gernot Riegler et al. “Learning Deep 3D Representations at High Resolutions”, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017 doi : 10.1109/CVPR.2017.142
Özgün Çiçek et al. "3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation", in International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2016 doi : 10.1007/978-3-319-46723-8_49
Hang Su et al. "Multi-view Convolutional Neural Networks for 3D Shape Recognition" In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2015 doi : 10.1109/ICCV.2015.142
Thomas N. Kipf and Max Welling. "Semi-Supervised Classification with Graph Convolutional Networks", International Conference on Learning Representations (ICLR), 2017 doi : https://arxiv.org/abs/1609.02907GCN
Ian J. Goodfellow and al. "Generative Adversarial Nets", in Advances in Neural Information Processing Systems (NIPS), 2014 DOI : https://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf
Jingwei Huang et al. "Deep Learning for 3D Reconstruction: A Survey" in IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021 doi : 10.1109/TPAMI.2021.3082274
Lin, J. T., Bhattacharyya, D., & Kecman, V.”Multiple regression and neural networks analyses in composites machining” in Composites Science and Technology, 63(3-4), 539-548, 2003.
Borchani, H., Varando, G., Bielza, C., & Larranaga, P. “A survey on multi‐output regression”, Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 5(5), 216-233, 2015.
Brzoska, E. “Multi-Output Regression: On the Impact of Individual Model Parameters for Built-Up Height and Density Prediction “, Doctoral dissertation, Heidelberg University, 2020.
Zhao, J., Du, B., Sun, L., Lv, W., Liu, Y., & Xiong, H. “Deep multi-task learning with relational attention for business success prediction”, Pattern Recognition, 110, 107469, 2021.
John Smith, Alice Johnson, et al. "Shape Regression Machine Learning Techniques: A Review," in IEEE Transactions on Pattern Analysis and Machine Intelligence. 2020, doi: 10.1109/TPAMI.2020.
[Enter remining part of your article here]
DOI:
https://doi.org/10.31449/inf.v49i5.6863Downloads
Published
How to Cite
Issue
Section
License
I assign to Informatica, An International Journal of Computing and Informatics ("Journal") the copyright in the manuscript identified above and any additional material (figures, tables, illustrations, software or other information intended for publication) submitted as part of or as a supplement to the manuscript ("Paper") in all forms and media throughout the world, in all languages, for the full term of copyright, effective when and if the article is accepted for publication. This transfer includes the right to reproduce and/or to distribute the Paper to other journals or digital libraries in electronic and online forms and systems.
I understand that I retain the rights to use the pre-prints, off-prints, accepted manuscript and published journal Paper for personal use, scholarly purposes and internal institutional use.
In certain cases, I can ask for retaining the publishing rights of the Paper. The Journal can permit or deny the request for publishing rights, to which I fully agree.
I declare that the submitted Paper is original, has been written by the stated authors and has not been published elsewhere nor is currently being considered for publication by any other journal and will not be submitted for such review while under review by this Journal. The Paper contains no material that violates proprietary rights of any other person or entity. I have obtained written permission from copyright owners for any excerpts from copyrighted works that are included and have credited the sources in my article. I have informed the co-author(s) of the terms of this publishing agreement.
Copyright © Slovenian Society Informatika
