[ECCVW2020] Robust Long-Term Object Tracking via Improved Discriminative Model Prediction (RLT-DiMP)

Related tags

Deep LearningRLT-DIMP
Overview

Feel free to visit my homepage

Robust Long-Term Object Tracking via Improved Discriminative Model Prediction (RLT-DIMP) [ECCVW2020 paper]


Presentation video

1-minute version (ENG)

Video Label

12-minute version (ENG)

Video Label


Summary

Abstract

We propose an improved discriminative model prediction method for robust long-term tracking based on a pre-trained short-term tracker. The baseline pre-trained short-term tracker is SuperDiMP which combines the bounding-box regressor of PrDiMP with the standard DiMP classifier. Our tracker RLT-DiMP improves SuperDiMP in the following three aspects: (1) Uncertainty reduction using random erasing: To make our model robust, we exploit an agreement from multiple images after erasing random small rectangular areas as a certainty. And then, we correct the tracking state of our model accordingly. (2) Random search with spatio-temporal constraints: we propose a robust random search method with a score penalty applied to prevent the problem of sudden detection at a distance. (3) Background augmentation for more discriminative feature learning: We augment various backgrounds that are not included in the search area to train a more robust model in the background clutter. In experiments on the VOT-LT2020 benchmark dataset, the proposed method achieves comparable performance to the state-of-the-art long-term trackers.


Framework


Baseline

  • We adopt the pre-trained short-term tracker which combines the bounding box regressor of PrDiMP with the standard DiMP classifier
  • This tracker's name is SuperDiMP and it can be downloaded on the DiMP-family's github page [link]

Contribution1: Uncertainty reduction using random erasing


Contribution2: Random search with spatio-temporal constraints


Contribution3: Background augmentation for more discriminative learning


Prerequisites

  • Ubuntu 18.04 / Python 3.6 / CUDA 10.0 / gcc 7.5.0
  • Need anaconda
  • Need GPU (more than 2GB, Sometimes it is a little more necessary depending on the situation.)
  • Unfortunately, "Precise RoI Pooling" included in the Dimp tracker only supports GPU (cuda) implementations.
  • Need root permission
  • All libraries in “install.sh” file (please check “how to install”)

How to install

  • Unzip files in $(tracker-path)
  • cd $(tracker-path)
  • bash install.sh $(anaconda-path) $(env-name) (Automatically create conda environment, If you don’t want to make more conda environments, run “bash install_in_conda.sh” after conda activation)
  • check pretrained model "super_dimp.pth.tar" in $(tracker-path)$/pytracking/networks/ (It should be downloaded by install.sh)
  • conda activate $(env-name)
  • make VOTLT2020 workspace (vot workspace votlt2020 --workspace $(workspace-path))
  • move trackers.ini to $(workspace-path)
  • move(or download) votlt2020 dataset to $(workspace-path)/sequences
  • set the VOT dataset directory ($(tracker-path)/pytracking/evaluation/local.py), vot_path should include ‘sequence’ word (e.g., $(vot-dataset-path)/sequences/), vot_path must be the absolute path (not relative path)
  • modify paths in the trackers.ini file, paths should include ‘pytracking’ word (e.g., $(tracker-path)/pytracking), paths must be absolute path (not relative path)
  • cd $(workspace-path)
  • vot evaluate RLT_DiMP --workspace $(workspace-path)
  • It will fail once because the “precise rol pooling” file has to be compiled through the ninja. Please check the handling error parts.
  • vot analysis --workspace $(workspace-path) RLT_DiMP --output json

Handling errors

  • “Process did not finish yet” or “Error during tracker execution: Exception when waiting for response: Unknown”-> re-try or “sudo rm -rf /tmp/torch_extensions/_prroi_pooling/
  • About “groundtruth.txt” -> check vot_path in the $(tracker-path)/pytracking/evaluation/local.py file
  • About “pytracking/evaluation/local.py” -> check and run install.sh
  • About “permission denied : “/tmp/torch_extensions/_prroi_pooling/” -> sudo chmod -R 777 /tmp/torch_extensions/_prroi_pooling/
  • About “No module named 'ltr.external.PreciseRoiPooling’” or “can not complie Precise RoI Pooling library error” -> cd $(tracker-path) -> rm -rf /ltr/external/PreciseRoiPooling -> git clone https://github.com/vacancy/PreciseRoIPooling.git ltr/external/PreciseRoIPooling
  • If nothing happens since the code just stopped -> sudo rm -rf /tmp/torch_extensions/_prroi_pooling/

Contact

If you have any questions, please feel free to contact [email protected]


Acknowledgments

  • The code is based on the PyTorch implementation of the DiMP-family.
  • This work was done while the first author was a visiting researcher at CMU.
  • This work was supported in part through NSF grant IIS-1650994, the financial assistance award 60NANB17D156 from U.S. Department of Commerce, National Institute of Standards and Technology (NIST) and by the Intelligence Advanced Research Projects Activity (IARPA) via Department of Interior/Interior Business Center (DOI/IBC) contract number D17PC0034. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copy-right annotation/herein. Disclaimer: The views and conclusions contained herein are those of the authors and should not be interpreted as representing the official policies or endorsements, either expressed or implied, of NIST, IARPA, NSF, DOI/IBC, or the U.S. Government.

Citation

@InProceedings{Choi2020,
  author = {Choi, Seokeon and Lee, Junhyun and Lee, Yunsung and Hauptmann, Alexander},
  title = {Robust Long-Term Object Tracking via Improved Discriminative Model Prediction},
  booktitle={Proceedings of the European Conference on Computer Vision (ECCV)},
  pages={0--0},
  year={2020}
}

Reference

  • [PrDiMP] Danelljan, Martin, Luc Van Gool, and Radu Timofte. "Probabilistic Regression for Visual Tracking." arXiv preprint arXiv:2003.12565 (2020).
  • [DiMP] Bhat, Goutam, et al. "Learning discriminative model prediction for tracking." Proceedings of the IEEE International Conference on Computer Vision. 2019.
  • [ATOM] Danelljan, Martin, et al. "Atom: Accurate tracking by overlap maximization." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019.
Owner
Seokeon Choi
I plan to receive a Ph.D. in Aug. 2021. I'm currently looking for a full-time job, residency program, or post-doc. linkedin.com/in/seokeon
Seokeon Choi
Code for the paper "Training GANs with Stronger Augmentations via Contrastive Discriminator" (ICLR 2021)

Training GANs with Stronger Augmentations via Contrastive Discriminator (ICLR 2021) This repository contains the code for reproducing the paper: Train

Jongheon Jeong 174 Dec 29, 2022
StyleTransfer - Open source style transfer project, based on VGG19

StyleTransfer - Open source style transfer project, based on VGG19

Patrick martins de lima 9 Dec 13, 2021
PyTorch framework for Deep Learning research and development.

Accelerated DL & RL PyTorch framework for Deep Learning research and development. It was developed with a focus on reproducibility, fast experimentati

Catalyst-Team 29 Jul 13, 2022
A script helps the user to update Linux and Mac systems through the terminal

Description This script helps the user to update Linux and Mac systems through the terminal. All the user has to install some requirements and then ru

Roxcoder 2 Jan 23, 2022
InterfaceGAN++: Exploring the limits of InterfaceGAN

InterfaceGAN++: Exploring the limits of InterfaceGAN Authors: Apavou Clément & Belkada Younes From left to right - Images generated using styleGAN and

Younes Belkada 42 Dec 23, 2022
Codebase for "Revisiting spatio-temporal layouts for compositional action recognition" (Oral at BMVC 2021).

Revisiting spatio-temporal layouts for compositional action recognition Codebase for "Revisiting spatio-temporal layouts for compositional action reco

Gorjan 20 Dec 15, 2022
This repo is the code release of EMNLP 2021 conference paper "Connect-the-Dots: Bridging Semantics between Words and Definitions via Aligning Word Sense Inventories".

Connect-the-Dots: Bridging Semantics between Words and Definitions via Aligning Word Sense Inventories This repo is the code release of EMNLP 2021 con

12 Nov 22, 2022
Generative vs Discriminative: Rethinking The Meta-Continual Learning (NeurIPS 2021)

Generative vs Discriminative: Rethinking The Meta-Continual Learning (NeurIPS 2021) In this repository we provide PyTorch implementations for GeMCL; a

4 Apr 15, 2022
This is the first released system towards complex meters` detection and recognition, which is implemented by computer vision techniques.

A three-stage detection and recognition pipeline of complex meters in wild This is the first released system towards detection and recognition of comp

Yan Shu 19 Nov 28, 2022
Custom implementation of Corrleation Module

Pytorch Correlation module this is a custom C++/Cuda implementation of Correlation module, used e.g. in FlowNetC This tutorial was used as a basis for

Clément Pinard 361 Dec 12, 2022
A pytorch implementation of faster RCNN detection framework (Use detectron2, it's a masterpiece)

Notice(2019.11.2) This repo was built back two years ago when there were no pytorch detection implementation that can achieve reasonable performance.

Ruotian(RT) Luo 1.8k Jan 01, 2023
Pytorch Implementation of LNSNet for Superpixel Segmentation

LNSNet Overview Official implementation of Learning the Superpixel in a Non-iterative and Lifelong Manner (CVPR'21) Learning Strategy The proposed LNS

42 Oct 11, 2022
Code for Reciprocal Adversarial Learning for Brain Tumor Segmentation: A Solution to BraTS Challenge 2021 Segmentation Task

BRATS 2021 Solution For Segmentation Task This repo contains the supported pytorch code and configuration files to reproduce 3D medical image segmenta

Himashi Amanda Peiris 6 Sep 15, 2022
Code for "Diversity can be Transferred: Output Diversification for White- and Black-box Attacks"

Output Diversified Sampling (ODS) This is the github repository for the NeurIPS 2020 paper "Diversity can be Transferred: Output Diversification for W

50 Dec 11, 2022
This repository contains a pytorch implementation of "StereoPIFu: Depth Aware Clothed Human Digitization via Stereo Vision".

StereoPIFu: Depth Aware Clothed Human Digitization via Stereo Vision | Project Page | Paper | This repository contains a pytorch implementation of "St

87 Dec 09, 2022
Addon and nodes for working with structural biology and molecular data in Blender.

Molecular Nodes 🧬 🔬 💻 Buy Me a Coffee to Keep Development Going! Join a Community of Blender SciVis People! What is Molecular Nodes? Molecular Node

Brady Johnston 456 Jan 08, 2023
WSDM2022 "A Simple but Effective Bidirectional Extraction Framework for Relational Triple Extraction"

BiRTE WSDM2022 "A Simple but Effective Bidirectional Extraction Framework for Relational Triple Extraction" Requirements The main requirements are: py

9 Dec 27, 2022
Code release for the ICML 2021 paper "PixelTransformer: Sample Conditioned Signal Generation".

PixelTransformer Code release for the ICML 2021 paper "PixelTransformer: Sample Conditioned Signal Generation". Project Page Installation Please insta

Shubham Tulsiani 24 Dec 17, 2022
Read number plates with https://platerecognizer.com/

HASS-plate-recognizer Read vehicle license plates with https://platerecognizer.com/ which offers free processing of 2500 images per month. You will ne

Robin 69 Dec 30, 2022
Drone-based Joint Density Map Estimation, Localization and Tracking with Space-Time Multi-Scale Attention Network

DroneCrowd Paper Detection, Tracking, and Counting Meets Drones in Crowds: A Benchmark. Introduction This paper proposes a space-time multi-scale atte

VisDrone 98 Nov 16, 2022