An Effective Loss Function for Generating 3D Models from Single 2D Image without Rendering
Papers with code | Paper
University of Novi Sad University of Cambridge
Citation
Besides AIAI 2021, our paper is in a Springer's book entitled "Artificial Intelligence Applications and Innovations": link
Please, cite our paper if you find this code useful for your research.
@article{zubic2021effective,
title={An Effective Loss Function for Generating 3D Models from Single 2D Image without Rendering},
author={Zubi{\'c}, Nikola and Li{\`o}, Pietro},
journal={arXiv preprint arXiv:2103.03390},
year={2021}
}
Prerequisites
-
Download code:
Git clone the code with the following command:git clone https://github.com/NikolaZubic/2dimageto3dmodel.git -
Open the project with Conda Environment (Python 3.7)
-
Install packages:
conda install pytorch torchvision torchaudio cudatoolkit=11.0 -c pytorchThen git clone Kaolin library in the root (2dimageto3dmodel) folder with the following commit and run the following commands:
cd kaolin python setup.py install pip install --no-dependencies nuscenes-devkit opencv-python-headless scikit-learn joblib pyquaternion cachetools pip install packaging
Run the program
Run the following commands from the root/code/ (2dimageto3dmodel/code/) directory:
python main.py --dataset cub --batch_size 16 --weights pretrained_weights_cub --save_results
for the CUB Birds Dataset.
python main.py --dataset p3d --batch_size 16 --weights pretrained_weights_p3d --save_results
for the Pascal 3D+ Dataset.
The results will be saved at 2dimageto3dmodel/code/results/ path.
Continue training
To continue the training process:
Run the following commands (without --save_results) from the root/code/ (2dimageto3dmodel/code/) directory:
python main.py --dataset cub --batch_size 16 --weights pretrained_weights_cub
for the CUB Birds Dataset.
python main.py --dataset p3d --batch_size 16 --weights pretrained_weights_p3d
for the Pascal 3D+ Dataset.
License
MIT
Acknowledgment
This idea has been built based on the architecture of Insafutdinov & Dosovitskiy.
Poisson Surface Reconstruction was used for Point Cloud to 3D Mesh transformation.
The GAN architecture (used for texture mapping) is a mixture of Xian's TextureGAN and Li's GAN.
140 Sep 26, 2022
2 Nov 30, 2021
519 Dec 21, 2022
1 Nov 13, 2022
148 Dec 27, 2022
395 Dec 14, 2022
4 Feb 09, 2022
37 Nov 10, 2022
27 Sep 17, 2021
3 Oct 18, 2022
10 Dec 22, 2022
12 Nov 30, 2022
3.6k Dec 22, 2022
43 Oct 03, 2022
614 Dec 31, 2022
12 Nov 02, 2022
12 Dec 21, 2022
332 Jan 03, 2023
20 Feb 13, 2022
6 Sep 23, 2022