An Implementation of SiameseRPN with Feature Pyramid Networks

Overview

SiameseRPN with FPN

This project is mainly based on HelloRicky123/Siamese-RPN. What I've done is just add a Feature Pyramid Network method to the original AlexNet structures.

For more details about siameseRPN please refer to the paper : High Performance Visual Tracking with Siamese Region Proposal Network by Bo Li, Junjie Yan,Wei Wu, Zheng Zhu, Xiaolin Hu.

For more details about Feature Pyramid Network please refer to the paper: Feature Pyramid Network for Object Detection by Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie.

Networks

  • Siamese Region Proposal Networks

    image-20210909160951628

  • Feature Pyramid Networks

    image-20210909161336484

  • SimaeseRPN+FPN

    • Template Branch

      0001

    • Detection Branch

      0001

Results

This project can get 0.618 AUC on OTB100, which also achieves overall 1.3% progress than the performance of baseline Siamese-RPN. Additionally, based on the ablation study results, it also shows that it can achieve robust performance different operating systems and GPUs.

Data preparation

I only use pre-trained models to finish my experiments,so here I would post the testing dataset OTB100 I get from http://cvlab.hanyang.ac.kr/tracker_benchmark/

If you don't want to download through the website above, you can just download: https://pan.baidu.com/s/1vWIn8ovCGKmlgIdHdt_MkA key: p8u4

For more details about OTB100 please refer to the paper: Object Tracking Benchmark by Yi Wu, Jongwoo Lim, Ming-Hsuan Yang.

Train phase

I didn't do any training but I still keep the baseline training method in my project. So if you have VID dataset or youtube-bb dataset, I would just post the steps of training here

Create dataset:

python bin/create_dataset_ytbid.py --vid-dir /PATH/TO/ILSVRC2015 --ytb-dir /PATH/TO/YT-BB --output-dir /PATH/TO/SAVE_DATA --num_threads 6

Create lmdb:

python bin/create_lmdb.py --data-dir /PATH/TO/SAVE_DATA --output-dir /PATH/TO/RESULT.lmdb --num_threads 12

Train:

python bin/train_siamrpn.py --data_dir /PATH/TO/SAVE_DATA

Test phase

If want to test the tracker, please first change the project path:

sys.path.append('[your_project_path]')

And then choose the combinations of different layers I putted in the net/network.py

then input your model path and dataset path to run:

python bin/test_OTB.py -ms [your_model_path] -v tb100 -d [your_dataset_path]

Environment

I've exported my anaconda and pip environment into /env/conda_env.yaml and /env/pip_requirements.txt

if you want to use it, just run the command below accordingly

for anaconda:

conda create -n [your_env_name] -f conda_env.yaml

for pip:

pip install -r requirements.txt

Model Download

Model which the baseline uses: https://pan.baidu.com/s/1vSvTqxaFwgmZdS00U3YIzQ keyword: v91k

Model after training 50 epoch: https://pan.baidu.com/s/1m9ISra0B04jcmjW1n73fxg keyword: 0s03

Experimental Environment

(1)

DELL-Precision-7530

OS: Ubuntu 18.04 LTS CPU: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz

Memory: 2*8G DDR4 2666MHZ

GPU: Nvidia Quadro P1000

(2)

HP OMEN

OS: Windows 10 Home Edition

CPU: Intel(R) Core(TM) i7-9750H CPU @ 2.6GHz

Memory: 2*8G DDR4 2666MHZ

GPU: Nvidia Geforce RTX2060

Optimization

On Ubuntu and Quadro P1000

  • AUCs with model siamrpn_38.pth
Layers Results(AUC)
baseline 0.610
2+5 0.618
2+3+5 0.607
2+3+4+5 0.611
  • AUCs with model siamrpn_50.pth
Layers Results(AUC)
baseline 0.600
2+5 0.605
2+3+5 0.594
2+3+4+5 0.605

On Windows 10 and Nvidia Geforce RTX2060

  • AUCs with model siamrpn_38.pth
layers Results(AUC)
baseline 0.610
2+5 0.617
2+3+5 0.607
2+3+4+5 0.612
  • AUCs with model siamrpn_50.pth
Layers Results(AUC)
baseline 0.597
2+5 0.606
2+3+5 0.597
2+3+4+5 0.605

Reference

[1] B. Li, J. Yan, W. Wu, Z. Zhu, X. Hu, High Performance Visual Tracking with Siamese Region Proposal Network, inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pages 8971-8980.

[2] T. Lin, P. Dollar, R. Girshick, K. He, B. Hariharan, S. Belongie, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pages 2117-2125.

[3] Y. Wu, J. Lim, M. Yang, "Object Tracking Benchmark", in IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, pages 1834-1848.

AdaShare: Learning What To Share For Efficient Deep Multi-Task Learning

AdaShare: Learning What To Share For Efficient Deep Multi-Task Learning (NeurIPS 2020) Introduction AdaShare is a novel and differentiable approach fo

94 Dec 22, 2022
Official Code Release for "CLIP-Adapter: Better Vision-Language Models with Feature Adapters"

Official Code Release for "CLIP-Adapter: Better Vision-Language Models with Feature Adapters" Pipeline of CLIP-Adapter CLIP-Adapter is a drop-in modul

peng gao 157 Dec 26, 2022
[ICLR'21] Counterfactual Generative Networks

This repository contains the code for the ICLR 2021 paper "Counterfactual Generative Networks" by Axel Sauer and Andreas Geiger. If you want to take the CGN for a spin and generate counterfactual ima

88 Jan 02, 2023
a generic C++ library for image analysis

VIGRA Computer Vision Library Copyright 1998-2013 by Ullrich Koethe This file is part of the VIGRA computer vision library. You may use,

Ullrich Koethe 378 Dec 30, 2022
Deep Learning Based EDM Subgenre Classification using Mel-Spectrogram and Tempogram Features"

EDM-subgenre-classifier This repository contains the code for "Deep Learning Based EDM Subgenre Classification using Mel-Spectrogram and Tempogram Fea

11 Dec 20, 2022
classify fashion-mnist dataset with pytorch

Fashion-Mnist Classifier with PyTorch Inference 1- clone this repository: git clone https://github.com/Jhamed7/Fashion-Mnist-Classifier.git 2- Instal

1 Jan 14, 2022
Core ML tools contain supporting tools for Core ML model conversion, editing, and validation.

Core ML Tools Use coremltools to convert machine learning models from third-party libraries to the Core ML format. The Python package contains the sup

Apple 3k Jan 08, 2023
This is an example of object detection on Micro bacterium tuberculosis using Mask-RCNN

Mask-RCNN on Mycobacterium tuberculosis This is an example of object detection on Mycobacterium Tuberculosis using Mask RCNN. Implement of Mask R-CNN

Jun-En Ding 1 Sep 16, 2021
Contrastive Learning for Many-to-many Multilingual Neural Machine Translation(mCOLT/mRASP2), ACL2021

Contrastive Learning for Many-to-many Multilingual Neural Machine Translation(mCOLT/mRASP2), ACL2021 The code for training mCOLT/mRASP2, a multilingua

104 Jan 01, 2023
Statistical and Algorithmic Investing Strategies for Everyone

Eiten - Algorithmic Investing Strategies for Everyone Eiten is an open source toolkit by Tradytics that implements various statistical and algorithmic

Tradytics 2.5k Jan 02, 2023
The Python code for the paper A Hybrid Quantum-Classical Algorithm for Robust Fitting

About The Python code for the paper A Hybrid Quantum-Classical Algorithm for Robust Fitting The demo program was only tested under Conda in a standard

Anh-Dzung Doan 5 Nov 28, 2022
PyTorch implementation of Spiking Neural Networks trained on surrogate gradient & BPTT using snntorch.

snn-localization repo PyTorch implementation of Spiking Neural Networks trained on surrogate gradient & BPTT using snntorch. Install Dependencies Orig

Sami BARCHID 1 Jan 06, 2022
Implementation of ICCV 2021 oral paper -- A Novel Self-Supervised Learning for Gaussian Mixture Model

SS-GMM Implementation of ICCV 2021 oral paper -- Self-Supervised Image Prior Learning with GMM from a Single Noisy Image with supplementary material R

HUST-The Tan Lab 4 Dec 05, 2022
Automatic deep learning for image classification.

AutoDL AutoDL automates machine learning tasks enabling you to easily achieve strong predictive performance in your applications. With just a few line

wenqi 2 Oct 12, 2022
Rethinking Portrait Matting with Privacy Preserving

Rethinking Portrait Matting with Privacy Preserving This is the official repository of the paper Rethinking Portrait Matting with Privacy Preserving.

184 Jan 03, 2023
H&M Fashion Image similarity search with Weaviate and DocArray

H&M Fashion Image similarity search with Weaviate and DocArray This example shows how to do image similarity search using DocArray and Weaviate as Doc

Laura Ham 18 Aug 11, 2022
A simple version for graphfpn

GraphFPN: Graph Feature Pyramid Network for Object Detection Download graph-FPN-main.zip For training , run: python train.py For test with Graph_fpn

WorldGame 67 Dec 25, 2022
code for `Look Closer to Segment Better: Boundary Patch Refinement for Instance Segmentation`

Look Closer to Segment Better: Boundary Patch Refinement for Instance Segmentation (CVPR 2021) Introduction PBR is a conceptually simple yet effective

H.Chen 143 Jan 05, 2023
Implementation of the pix2pix model on satellite images

This repo shows how to implement and use the pix2pix GAN model for image to image translation. The model is demonstrated on satellite images, and the

3 May 24, 2022
Pytorch implementation of Zero-DCE++

Zero-DCE++ You can find more details here: https://li-chongyi.github.io/Proj_Zero-DCE++.html. You can find the details of our CVPR version: https://li

Chongyi Li 157 Dec 23, 2022