An Effective Loss Function for Generating 3D Models from Single 2D Image without Rendering
Papers with code | Paper
University of Novi Sad University of Cambridge
Citation
Besides AIAI 2021, our paper is in a Springer's book entitled "Artificial Intelligence Applications and Innovations": link
Please, cite our paper if you find this code useful for your research.
@article{zubic2021effective,
title={An Effective Loss Function for Generating 3D Models from Single 2D Image without Rendering},
author={Zubi{\'c}, Nikola and Li{\`o}, Pietro},
journal={arXiv preprint arXiv:2103.03390},
year={2021}
}
Prerequisites
-
Download code:
Git clone the code with the following command:git clone https://github.com/NikolaZubic/2dimageto3dmodel.git -
Open the project with Conda Environment (Python 3.7)
-
Install packages:
conda install pytorch torchvision torchaudio cudatoolkit=11.0 -c pytorchThen git clone Kaolin library in the root (2dimageto3dmodel) folder with the following commit and run the following commands:
cd kaolin python setup.py install pip install --no-dependencies nuscenes-devkit opencv-python-headless scikit-learn joblib pyquaternion cachetools pip install packaging
Run the program
Run the following commands from the root/code/ (2dimageto3dmodel/code/) directory:
python main.py --dataset cub --batch_size 16 --weights pretrained_weights_cub --save_results
for the CUB Birds Dataset.
python main.py --dataset p3d --batch_size 16 --weights pretrained_weights_p3d --save_results
for the Pascal 3D+ Dataset.
The results will be saved at 2dimageto3dmodel/code/results/ path.
Continue training
To continue the training process:
Run the following commands (without --save_results) from the root/code/ (2dimageto3dmodel/code/) directory:
python main.py --dataset cub --batch_size 16 --weights pretrained_weights_cub
for the CUB Birds Dataset.
python main.py --dataset p3d --batch_size 16 --weights pretrained_weights_p3d
for the Pascal 3D+ Dataset.
License
MIT
Acknowledgment
This idea has been built based on the architecture of Insafutdinov & Dosovitskiy.
Poisson Surface Reconstruction was used for Point Cloud to 3D Mesh transformation.
The GAN architecture (used for texture mapping) is a mixture of Xian's TextureGAN and Li's GAN.
159 Dec 20, 2022
235 Dec 19, 2022
1 Jun 05, 2022
7 Apr 13, 2022
6 Jul 07, 2022
11 Oct 17, 2022
360 Dec 27, 2022
1.7k Jan 02, 2023
12 Jul 22, 2022
30 Oct 19, 2022
2.1k Jan 02, 2023
280 Dec 23, 2022
1.2k Dec 21, 2022
320 Jan 02, 2023
482 Dec 18, 2022
254 Dec 29, 2022
106 Jan 06, 2023
890 Jan 04, 2023
111 Dec 07, 2022