An Effective Loss Function for Generating 3D Models from Single 2D Image without Rendering
Papers with code | Paper
University of Novi Sad University of Cambridge
Citation
Besides AIAI 2021, our paper is in a Springer's book entitled "Artificial Intelligence Applications and Innovations": link
Please, cite our paper if you find this code useful for your research.
@article{zubic2021effective,
title={An Effective Loss Function for Generating 3D Models from Single 2D Image without Rendering},
author={Zubi{\'c}, Nikola and Li{\`o}, Pietro},
journal={arXiv preprint arXiv:2103.03390},
year={2021}
}
Prerequisites
-
Download code:
Git clone the code with the following command:git clone https://github.com/NikolaZubic/2dimageto3dmodel.git -
Open the project with Conda Environment (Python 3.7)
-
Install packages:
conda install pytorch torchvision torchaudio cudatoolkit=11.0 -c pytorchThen git clone Kaolin library in the root (2dimageto3dmodel) folder with the following commit and run the following commands:
cd kaolin python setup.py install pip install --no-dependencies nuscenes-devkit opencv-python-headless scikit-learn joblib pyquaternion cachetools pip install packaging
Run the program
Run the following commands from the root/code/ (2dimageto3dmodel/code/) directory:
python main.py --dataset cub --batch_size 16 --weights pretrained_weights_cub --save_results
for the CUB Birds Dataset.
python main.py --dataset p3d --batch_size 16 --weights pretrained_weights_p3d --save_results
for the Pascal 3D+ Dataset.
The results will be saved at 2dimageto3dmodel/code/results/ path.
Continue training
To continue the training process:
Run the following commands (without --save_results) from the root/code/ (2dimageto3dmodel/code/) directory:
python main.py --dataset cub --batch_size 16 --weights pretrained_weights_cub
for the CUB Birds Dataset.
python main.py --dataset p3d --batch_size 16 --weights pretrained_weights_p3d
for the Pascal 3D+ Dataset.
License
MIT
Acknowledgment
This idea has been built based on the architecture of Insafutdinov & Dosovitskiy.
Poisson Surface Reconstruction was used for Point Cloud to 3D Mesh transformation.
The GAN architecture (used for texture mapping) is a mixture of Xian's TextureGAN and Li's GAN.
36 Jan 06, 2023
255 Dec 27, 2022
2k Jan 08, 2023
6 Nov 18, 2022
36 Dec 04, 2022
493 Jan 04, 2023
41 Nov 08, 2022
142 May 16, 2022
12 Nov 22, 2022
144 Dec 26, 2022
255 Sep 07, 2022
19 Nov 28, 2022
5 Dec 21, 2021
176 Jan 05, 2023
98 Dec 30, 2022
33 Nov 27, 2022
20 Dec 31, 2021
2.3k Jan 08, 2023
495 Dec 28, 2022
957 Dec 21, 2022