Self-Supervised Learning of Image Scale and Orientation Estimation (BMVC 2021)

This is the official implementation of the paper "Self-Supervised Learning of Image Scale and Orientation Estimation" by Jongmin Lee [Google Scholar], Yoonwoo Jeong [Google Scholar], and Minsh Cho [Google Scholar]. We introduce a self-supervised framework for learning patch pose. Given a rescaled/rotated pair of image patches, we feed them to the patch pose estimation networks that output scale/orientation histograms for each. We compare the output histogram vectors by the histogram alignment technique and compute the loss.

Requirements

• Ubuntu 18.04
• python 3.8
• pytorch 1.8.1
• torchvision 0.9.1
• wandb 0.10.28

Environment

Clone the Git repository

git clone https://github.com/bluedream1121/SelfScaOri.git

Install dependency

Run the script to install all the dependencies. You need to provide the conda install path (e.g. ~/anaconda3) and the name for the created conda environment.

bash install.sh conda_install_path self-sca-ori

Dataset preparation

You can download the training/test dataset using the following scripts:

cd datasets
bash download.sh

If you want to regenerate the patchPose datasets, please run the following script:

cd datasets/patchpose_dataset_generation
bash generation_script.sh

Trained models

cd trained_models
bash download_ori_model.sh
bash download_sca_model.sh

Test on the patchPose and the HPatches

After download the datasets and the pre-trained models, you can evaluate the patch pose estimation results using the following scripts:

python test.py --load trained_models/_*branchori/best_model.pt  --dataset_type ppa_ppb
python test.py --load trained_models/_*branchsca/best_model.pt  --dataset_type ppa_ppb

python test.py --load trained_models/_*branchori/best_model.pt  --dataset_type hpa
python test.py --load trained_models/_*branchsca/best_model.pt  --dataset_type hpa

Training

You can train the networks for patch scale estimation and orientation estimation using the proposed histogram alignment loss as follows:

python train.py --branch ori --output_ori 36

python train.py --branch sca --output_sca 13

Citation

If you find our code or paper useful to your research work, please consider citing our work using the following bibtex:

@inproceedings{lee2021self,
    author   = {},
    title    = {},
    booktitle= {},
    year     = {2021}
}

Contact

Jongmin Lee ([email protected])

Questions can also be left as issues in the repository.